admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-11-09 14:02:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
5,128 Commits 568 Branches 294 Tags
8f4c19daaeb00a2aeccf761cd96879867a4d86ce
Commit Graph
5128 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Aleksei Sviridkin 395af115b5 docs(agents): require one continuous line per paragraph 
Markdown renderers collapse a soft line break into a space, so a
paragraph hardwrapped at ~80 columns renders exactly like the same
paragraph on one line -- while breaking narrow viewports, mangling
nested list and table rendering, and turning a one-word edit into a
diff that reflows the entire block.

The rule covers prose this repository owns, in markdown files and in
every body published to GitHub: PR bodies, issue bodies, review
comments, release notes. It governs new and changed paragraphs; the
existing tree is wrapped in many places and is not a backlog of defects
to go and fix. Vendored upstream chart docs stay verbatim, published
changelogs stay as published, and commit message bodies keep the
~72-column wrap that git log and its tooling expect.

Structure that needs its line breaks is unaffected: code blocks, lists,
tables, headings, blockquotes, front matter, badge stacks, and explicit
hard breaks, which render as <br> and cannot be reflowed without
changing the output.

AGENTS.md opened with a wrapped paragraph of its own; reflow it.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-14 14:34:22 +03:00
Aleksei Sviridkin f24b56327b fix(platform): route OIDC backend calls through the in-cluster Keycloak by default 
authentication.oidc.keycloakInternalUrl defaulted to an empty string, which
left oauth2-proxy performing OIDC discovery against the external Keycloak
hostname. That request resolves through public DNS and lands on the root
ingress controller, which does not serve yet on a fresh install: the root
tenant's IngressClass only exists once tenant-root is ready, and tenant-root
is still reconciling while the dashboard starts. Discovery has nothing to talk
to, the gatekeeper fails, and the dashboard never comes up.

The bundled Keycloak is the only supported provider — the external hostname is
templated as keycloak.<host> and the realm is fixed — so backend-to-backend
calls have no reason to leave the cluster. Default the value to
http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy, the same Service
address keycloak-configure already uses. The dashboard and linstor-gui
gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo
and logout requests to the Service, while browser redirects stay on the
external host. Those calls travel as plaintext HTTP between pods, as
keycloak-configure's admin calls to the same address already do.

The value remains overridable for a Keycloak that does not live at the standard
in-cluster address, and setting it back to an empty string restores external
discovery. Existing OIDC clusters that never set it get a gatekeeper rollout
onto the internal path on upgrade.

Enabling OIDC swaps the dashboard's token-proxy container for oauth2-proxy, so
the E2E install now drives that swap to completion: it waits for the auth-proxy
container to appear, then gates on the gatekeeper rollout. Readiness alone
proves nothing there — the HelmRelease is still Ready from the token-proxy
install, and the Deployment keeps the old ReplicaSet Available while a new pod
crashloops, so only the rollout catches a regression back to external discovery.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-14 14:22:46 +03:00
Aleksei Sviridkin caf4645cf2 docs(harbor): fix SeaweedFS discovery key described as annotation 
The prerequisites section described the resolved SeaweedFS class as being
surfaced on the tenant namespace as the `namespace.cozystack.io/seaweedfs`
annotation. The tenant chart renders that key under `metadata.labels`,
alongside the sibling `/etcd`, `/ingress`, `/gateway`, `/monitoring` and
`/host` keys; no annotation of that name exists.

The wrong noun sends an operator to the wrong field. When a Harbor release
stays unreconciled waiting on `BucketInfo`, the documented check is to
confirm the tenant chain resolved a SeaweedFS class. Looking for it in
`.metadata.annotations` returns nothing even on a correctly configured
tenant, which invites the wrong conclusion that the tenant chain is broken
and sends the investigation away from the actual cause.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-14 13:45:19 +03:00
Aleksei Sviridkin 5d881f111a fix(fluxcd): omit empty distribution.artifact in FluxInstance for guest clusters (#3284) 
## What this PR does

Guest Kubernetes clusters (`apps/kubernetes`) with the `fluxcd` addon
enabled never became
`Ready`. The `flux-instance` template renders
`spec.distribution.artifact` unconditionally,
unlike the sibling optional fields (`artifactPullSecret`,
`imagePullSecret`, `variant`, which
are all `{{- if }}`-guarded). The umbrella default is empty (`artifact:
""`) so the operator
uses its embedded manifests, which is the intended air-gapped default
(#964, and the upstream
recommendation). But an empty value renders as YAML `null`, and the
flux-operator v0.50.0 CRD
types `distribution.artifact` as a non-nullable `string` (pattern
`^oci://.*$`), so the API
rejects the `FluxInstance`.

Result: the addon's HelmRelease looped install/uninstall
(`remediation.retries: -1`), the
parent `kubernetes` release stayed `InProgress`, and the cluster CR was
stuck `Ready=Unknown`.

This guards the field so an empty value is omitted entirely. Behaviour:

- empty (default) -> field omitted -> operator falls back to embedded
manifests: **air-gapped
default preserved**, and the CRD accepts it (artifact is not in
`distribution.required`);
- non-empty -> renders as before; `valuesOverride` still applies.

The template is a vendored subchart (`make update` runs `helm pull`), so
the change is carried
as `patches/guard-distribution-artifact.diff` and applied by the
`update` target, matching the
convention of the sibling `fluxcd-operator` package (`patch -p1 <
patches/*.diff`).

Verified with `helm template packages/system/fluxcd`:

- default values -> no `artifact:` line (before this PR: `artifact:
null`);
- `--set
flux-instance.instance.distribution.artifact=oci://example.com/x:1` ->
renders.

The affected path is off by default (`addons.fluxcd.enabled: false`) and
not exercised by the
in-repo CI values, which is why default e2e stayed green.

Closes #3283

### Release note

```release-note
fix(fluxcd): guest Kubernetes clusters with the fluxcd addon enabled no longer fail to install; an empty distribution artifact is now omitted so the operator uses its embedded manifests (air-gapped default preserved) instead of rendering an invalid null
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Prevented Flux instance configuration from including an empty
distribution artifact value, which could cause validation failures.
* Improved the generated FluxInstance manifest so the artifact field is
omitted entirely when no distribution artifact is provided.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-14 13:18:29 +03:00
Ivan Okhotnikov da47b70e2c fix(fluxcd): omit empty distribution.artifact in FluxInstance 
The flux-instance template rendered spec.distribution.artifact
unconditionally, unlike the sibling optional fields (artifactPullSecret,
imagePullSecret, variant). The umbrella default is empty to use the
operator's embedded manifests (the air-gapped default from #964), but an
empty value renders as YAML null. In the flux-operator v0.50.0 CRD the
field is optional yet typed 'string' with pattern ^oci://.*$ and is not
nullable, so the rendered null (like an empty string) fails schema
validation and the API rejects the FluxInstance.

Guest Kubernetes clusters with the fluxcd addon enabled could therefore
never install it: the sub-release looped install/uninstall, the parent
kubernetes release stayed InProgress, and the cluster never reached Ready.

Guard the field so an empty value is omitted (the operator falls back to
its embedded manifests, preserving the air-gapped default) while a
non-empty artifact still renders. The template lives in the vendored
subchart, so the change is carried as a patch applied by 'make update'.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
 2026-07-14 13:10:59 +03:00
IvanHunters 3380bb4ab6 refactor(hack): replace ls|grep with pure-bash glob (shellcheck SC2010) 
Filenames in packages/core/platform/images/migrations/migrations/ are
numeric-only by convention so the original `ls -1 | grep -E` filter was
safe in practice, but the cleaner pure-bash form is equivalent and
silences SC2010 without an annotation.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-07-14 13:08:54 +03:00
IvanHunters c3d3e2596e feat(ci): guard migrations.targetVersion against silent skip 
Adds hack/check-migrations-target.sh and wires it into unit-tests via the
migrations-target-check Makefile target. The check fails when the highest
numbered file under packages/core/platform/images/migrations/migrations/
is not matched by a bumped migrations.targetVersion in
packages/core/platform/values.yaml.

run-migrations.sh:34 loops `seq $CURRENT_VERSION $((TARGET_VERSION - 1))`,
so a migration whose number equals targetVersion never runs and the
intended state change is silently skipped on cluster upgrade.

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
 2026-07-14 13:08:54 +03:00
Aleksei Sviridkin f478915935 fix(dashboard): unbreak CORS on expired session for k8s API calls (#2788) 
## What this PR does

Fix two related issues that cause the dashboard SPA to break with CORS
errors once the `kc-access` cookie expires.

1. `KeycloakClient` for dashboard had no `webOrigins` set, so the
operator did not touch the field in Keycloak after initial creation.
After any `rootUrl` change the existing `webOrigins` value got stuck at
an old hostname and blocked CORS from the SPA. Setting it to `+` makes
Keycloak derive the allowed origins from `redirectUris`.

2. `oauth2-proxy` answered `302` (redirect to Keycloak `/auth`) for
unauthenticated XHR/fetch on `/api` and `/apis`. The login page does not
emit `Access-Control-Allow-Origin`, so the browser blocked the fetch
with a CORS error and the SPA had no way to recover. Adding
`--api-route` makes `oauth2-proxy` answer `401` without a redirect for
these paths, so the SPA can handle re-auth itself via a top-level
navigation.

### Release note

```release-note
fix(dashboard): dashboard SPA no longer fails with CORS errors after the kc-access cookie expires; oauth2-proxy returns 401 (not 302) for /api and /apis, and the KeycloakClient declares webOrigins: ["+"].
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Chores

* Updated API routing configuration to properly handle requests to API
endpoints
* Enhanced web origins configuration for improved authentication system
compatibility

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-14 13:07:13 +03:00
IvanHunters 0044221d31 fix(apps/vpn): remove invalid foo field from urls Secret (#3281) 
## What this PR does

Removes a leftover debug field `foo` from the `<release>-urls` Secret
template in the VPN app chart.

The `foo` field is not part of the `core/v1` Secret schema (which only
allows `data`, `stringData`, `type`, `immutable`, and `metadata`). It
went unnoticed under client-side apply, but under server-side apply the
stricter schema validation rejects it with:

```text
.foo: field not declared in schema
```

As a result, any VPN application upgrade fails and the release is left
in `UpgradeFailed` state.

### Release note

```release-note
fix(apps/vpn): remove invalid `foo` field from the urls Secret that broke Helm upgrades under server-side apply
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added per-user Shadowsocks connection links to the VPN secret.
* Links include the configured encryption method, host, port, and
connection name for easier client setup.

* **Bug Fixes**
  * Removed obsolete serialized password data from the VPN secret.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-14 12:50:17 +03:00
Andrey Kolkov bf69eff7a5 feat(keycloak): add S3 backup for the CNPG database (#3174) 
## What this PR does

Adds optional scheduled S3 backups for the `keycloak-db` CloudNativePG
cluster.

When `backup.enabled=true` with a `backup.destinationPath`, the Cluster
gets
`spec.backup.barmanObjectStore` (gzip-compressed data + WAL streams, a
configurable `retentionPolicy`) and a `ScheduledBackup` drives recurring
base
backups. S3 credentials are either inlined into a rendered
`keycloak-db-backup-s3-creds` Secret, or referenced via
`backup.existingSecretName` (a Secret carrying `AWS_ACCESS_KEY_ID` /
`AWS_SECRET_ACCESS_KEY`).

Disabled by default — no change to existing deployments; the object
store
coordinates and credentials are supplied per-environment by the
operator.

This is the Keycloak/Postgres half of backing up a Cozystack portal
control
plane. The portal-side etcd backups (system Talos etcd + api-server
etcd) live
in the companion downstream PR aenix-org/cozyportal#1008.

### Release note

```release-note
feat(keycloak): add optional S3 backups (CNPG Barman) for the Keycloak database, configurable via `backup.*` values and disabled by default.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added configurable scheduled backups for the Keycloak CNPG database to
an S3-compatible object store, including destination prefix, optional
endpoint URL, retention policy, and cron schedule.
* Backups and scheduled backup resources are only created when all
required backup settings are provided, avoiding incomplete
configuration.
* Supports backup credentials via inline values (creates the needed
Secret automatically) or via an existing Secret reference.
* **Documentation**
* Added guidance on the cron schedule format required for the scheduler.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-14 11:49:30 +04:00
Ivan Okhotnikov 70a2284551 fix(apps/vpn): remove invalid foo field from urls Secret 
A leftover debug field "foo" in the <release>-urls Secret template
violates the core/v1 Secret schema. It went unnoticed under client-side
apply but breaks Helm upgrades under server-side apply with
".foo: field not declared in schema", leaving the release in
UpgradeFailed state. Drop the debug block.

Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
 2026-07-13 23:32:24 +03:00
Myasnikov Daniil 103b8e16ee test(e2e): diff-driven install selector select-install.sh (#2847) 
Add hack/select-install.sh: given the E2E suites select-e2e.sh picked,
emit the forward dependsOn closure over the PackageSource graph — the
minimal set of packages to install for those suites. Companion to the
reverse (test-selection) walk in select-e2e.sh; keeps the cozystack-engine
edge since install needs it.

Suite->source mapping follows the <suite>-application convention with a
bare-<suite> fallback plus explicit cases for irregular names
(kubernetes/vminstance/securitygroup) and core-platform features that have
no separately-enabled package (serviceexposure). Includes a --validate mode
(reachability + cycle check) and a 13-case unit suite auto-discovered by
make unit-tests.

Track A of #2847; tracked in #3278.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-13 17:47:26 +05:00
myasnikovdaniil eea8b73460 feat(kubernetes): per-node-group nodeHealthCheck overrides (#3053) 
Adds per-node-group overrides for the worker MachineHealthCheck tuning
introduced as a cluster-wide `nodeHealthCheck` by #2931.

**What:** optional `nodeGroups[name].maxUnhealthy` and
`nodeGroups[name].nodeStartupTimeout`; when unset a group inherits the
cluster-wide `nodeHealthCheck.*` (no behaviour change). The per-group
value reuses the existing int-or-string validation; an invalid override
fails the render naming the offending group.

**Why:** the MachineHealthCheck is rendered per node group, so groups
can warrant different remediation tolerances (e.g. a stateful group at
`0%`, stateless at `50%`). This is the per-group granularity of the
standalone proposal (#2752 / #2935), rebuilt on top of #2931's
`nodeHealthCheck`.

**Notes:** stacked on `phase1-talos-migration` (#2931) — retarget to
`main` once it merges. Supersedes #2935 (original author credited as
co-author). helm-unittest covers per-group override (percent + bare
int), `nodeStartupTimeout` override, fallback-when-unset, and
invalid-override rejection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added per-node-group overrides for health-check settings:
`maxUnhealthy` and `nodeStartupTimeout`, with automatic fallback to
cluster-wide `nodeHealthCheck` when unset.
* **Bug Fixes**
* Improved `maxUnhealthy` parsing/rendering to support both bare
integers and percentage strings, including more precise validation
errors for the exact override path.
* **Documentation**
* Updated kubernetes app docs and values schemas (including CRD
chart-values schema) to document the new override fields.
* **Tests**
* Added MachineHealthCheck rendering tests for override, fallback, and
invalid `maxUnhealthy` (e.g., `"abc"`) cases.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-13 15:28:11 +05:00
Myasnikov Daniilandmattia-eleuteri db4db4a6e8 feat(kubernetes): per-node-group nodeHealthCheck overrides 
Add optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout
fields that override the cluster-wide nodeHealthCheck defaults per worker node
group. When unset, a group inherits nodeHealthCheck.{maxUnhealthy,nodeStartupTimeout},
so existing values are unaffected. The MachineHealthCheck render resolves the
per-group value first, reuses the existing int-or-string validation, and names
the offending group in the fail() message when an override is invalid.

Delivers the per-group granularity of the standalone maxUnhealthy proposal on
top of the cluster-wide nodeHealthCheck from the Talos worker bootstrap PR.

Co-authored-by: mattia-eleuteri <mattia@hidora.io>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-13 12:22:38 +05:00
renovate[bot] 8a5bc68b43 chore(deps): update actions/checkout action to v6.0.3 2026-07-12 13:35:59 +00:00
renovate[bot] 9dc63bd412 chore(deps): update github.com/kubevirt/csi-driver digest to 27b52aa 2026-07-12 13:35:55 +00:00
myasnikovdaniil 99a28d82c2 test(e2e): migrate app suite from BATS to Chainsaw and wire into CI (#2826) 
## What this PR does

Migrates the E2E **app** test suite from BATS to [Kyverno
Chainsaw](https://github.com/kyverno/chainsaw) — a declarative,
Kubernetes-native E2E framework (CNCF, part of the Kyverno project;
successor to KUTTL) — and wires it into CI as the replacement for the
per-app BATS loop. This grows the original pilot (postgres + bucket)
into the full suite.

All 22 app suites are ported under `hack/e2e-chainsaw/` and the
`hack/e2e-apps/*.bats` files are removed:

`postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `qdrant`,
`clickhouse`, `kafka`, `etcd`, `openbao`, `harbor`, `foundationdb`,
`external-dns`, `kuberture`, `vminstance`, `gateway`,
`kubernetes-latest`, `kubernetes-previous`, `kubernetes-oidc-system`,
`kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure`.

**Approach**

- **Declarative** suites assert on `status.conditions` and concrete
fields. The `timeout N sh -ec "until kubectl get ..."` + `kubectl wait`
pair that appeared ~100 times collapses into a single `assert` that
polls existence and state together, with a structured diff on failure
and automatic `events`/`describe`/`podLogs` capture via `catch`
(previously only `harbor.bats` did this, by hand).
- **Imperative** suites keep their logic in `script` steps: `openbao`
init/unseal, `kuberture` external-dns split-horizon probes,
`vminstance`, the `gateway` admission/impersonation cases, and
`kubernetes-latest`/`previous`, which wrap the relocated
`hack/e2e-chainsaw/_lib/run-kubernetes.sh` verbatim (Kamaji bring-up,
LB/NFS/ouroboros checks).
- `gateway` tests derive the tenant apex from the namespace
`namespace.cozystack.io/host` label at runtime, so they are
host-independent.

**CI**

- The `e2e` job now runs `chainsaw test hack/e2e-chainsaw/` via a new
`test-chainsaw` target and uploads the JUnit `chainsaw-report.xml`.
- The `chainsaw` binary is added to the e2e-sandbox image.
- `install-cozystack` and `test-openapi` stay BATS (cluster bootstrap +
OpenAPI checks).
- The per-app 3-retry loop is dropped — assertion polling replaces the
fixed-timeout flakiness it papered over.

**Validation**

Ran against a development cluster: the DB/app, storage, and VM suites
pass; the three suites that depend on platform features not present on
that cluster (`gateway`, `kuberture`, `external-dns`) and the two
heavyweight `kubernetes-*` suites are exercised by this PR's CI run on a
freshly installed platform.

Note for reviewers: service-port asserts use the `(ports[*].port)`
projection form rather than a number-literal filter (`` ports[?port ==
`N`] ``), because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal
comparisons.

**2026-07-09: reconciled with main (~520 commits of drift)**

- etcd suite re-ported to the v1alpha2 operator contracts (#2859):
`readyMembers=3` gate, pod-label/`/scale`/WorkloadMonitor/metrics/defrag
contracts, plus the `ETCD_E2E_S3_ROUNDTRIP`-gated backup round-trip
driving `examples/backups/etcd`
- 12 new `ingress-hostname-policy` gateway cases ported (apex derived at
runtime from the tenant-root namespace label)
- post-branch bats drift folded in: `mariadb-single` webhook guard,
kafka `c1.small` presets, bucket/harbor 2m BucketClaim fail-fast, bucket
readonly-denial promoted to a hard fail (cosi-driver v0.3.1), qdrant
PVC-reclaim guard (#3059), kuberture fail-loud negations,
SC-fallback-default test (#2872 B1)
- 4 suites that only exist on main since the branch was cut are ported:
`kubernetes-oidc-system`, `kubernetes-oidc-customconfig`,
`securitygroup`, `serviceexposure`
- CI seams: `nightly.yaml` converted to the chainsaw invocation,
`run-kubernetes.sh` reconciled (Talos/CABPT waits, tenant drain, LINSTOR
pool wait, talos-image-cache under `_lib/`), `e2e-capture-dataplane.sh`
replicated in the Chainsaw global catch

### Release note

```release-note
fix(kubernetes): tenant Kubernetes teardown no longer hangs when the cluster has no working nodes — the pre-delete hook now bounds its wait for the in-tenant HelmReleases and force-clears their Flux finalizers on timeout (#3271)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* End-to-end testing is now driven by a unified Chainsaw suite (platform
apps, Kubernetes, networking, security groups, service exposure).
* Test Impact Analysis selects affected suites, with an option to run
the full suite.
* CI produces a JUnit report artifact and collects suite-scoped
diagnostics on failure.
* **Bug Fixes**
* CI E2E now fails fast and emits clearer Kubernetes/HelmRelease and
sandbox bucket state diagnostics.
* Improved teardown reliability with bounded wait + finalizer handling
to avoid hangs.
* **Documentation**
* Updated E2E testing docs and local guidance to reflect the new
Chainsaw-based workflow and suite paths.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-12 11:58:08 +05:00
Myasnikov DaniilandClaude Opus 4.8 8af4ed543e test(e2e): address review nits — shellcheck directive + ports-workaround TODO 
Non-blocking review notes from #2826:

- Add `# shellcheck shell=bash` to the four sourced `_lib/*.sh` helpers
  (etcd-cleanup, remediation-guard, run-kubernetes, talos-image-cache) so
  shellcheck stops flagging SC2148 (no shell directive) — they are only ever
  `. sourced`, so there is no runtime interpreter to infer from a shebang.
- Add a `# TODO(chainsaw>0.2.15)` marker at the postgres Service asserts
  explaining the `(ports[0].port)`/`(ports[*].port)` projection is a v0.2.15
  number-literal-filter workaround (and that `ports[0]` checks only the first
  port), so it gets reverted on a chainsaw bump; cross-references the same
  workaround in kafka/mariadb/harbor and the e2e-testing.md gotchas section.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-10 17:02:46 +05:00
Myasnikov DaniilandClaude Opus 4.8 18715f8c63 docs(agents): document Chainsaw v0.2.15 gotchas in the e2e-testing guide 
The v0.2.15 syntax/semantics traps were tribal knowledge scattered across PR
review notes (the number-literal caveat lived only in #2826's body, which
evaporates on merge). Consolidate them into a durable subsection so the next
author finds them before hitting each one fresh: filter-as-list conditions,
no number-literals/backticks in JMESPath, error: fails hard on an unknown
GVK, no conditional Test-level skip, step-level-only finally, bounded/blocking
cleanup surfacing latent teardown bugs, projection-style failure diffs, and
the catch-script cwd.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-10 16:16:18 +05:00
Andrei Kvapil b99df11b36 fix(kubernetes): bump tenant worker Talos to v1.13.6 (#3269) 
## What this PR does

Tenant worker VMs were pinned to Talos v1.13.0, which ships Linux
6.18.24. That kernel predates the fixes for two related KVM x86
shadow-paging use-after-free bugs in `arch/x86/kvm/mmu/mmu.c`:
CVE-2026-46113, fixed upstream in
`0cb2af2ea66ad8ff195c156ea690f11216285bdf` and released in 6.18.30, and
CVE-2026-53359, fixed in `81ccda30b4e83d8f5cc4fd50503c44e3a33abfeb` and
released in 6.18.38. A worker's own kernel runs KVM whenever a tenant
enables nested virtualization inside the VM, and either bug lets a
nested guest escape into the worker.

This bumps the worker pin to Talos v1.13.6, which ships Linux 6.18.38.
Stable kernel branches are cumulative, so v1.13.6 is the first v1.13
patch release carrying both fixes. The pinned v1.13.0 ships 6.18.24 and
carries neither; v1.13.5 ships 6.18.36 and carries only CVE-2026-46113.
Nested virtualization stays available to tenants; the kernel bump is the
fix, not a capability removal.

The image-factory schematic is unchanged and resolves for the new
version: both the `openstack-amd64.raw.xz` artifact and the
`installer/<schematicID>:v1.13.6` manifest are served by the public
factory. Talos v1.13 supports Kubernetes 1.31 through 1.36, so the
chart's Talos-to-Kubernetes support window is unaffected and no tenant
`spec.version` needs to move.

Applying this rolls each worker pool once. The new pin changes the
rendered worker spec, which rotates the content hash in the
`KubevirtMachineTemplate` and `talos-reconcile` Job names, and CAPI
replaces the worker VMs to deliver the new kernel. The pinned hashes in
the chart tests are recomputed to match.

A second commit fixes the support-matrix lookup itself. The matrix dict
was keyed on the full patch version, so every Talos patch bump dropped
out of the dict and silently disabled the render-time guard — exactly
the mismatch the guard exists to catch. The lookup now derives the minor
from `talos.version`. The supported-Kubernetes window is unchanged;
unknown minors are still skipped, leaving compatibility to whoever
overrode the pin.

### Screenshots

Not applicable — no UI changes.

### Release note

```release-note
fix(kubernetes): bump tenant worker Talos to v1.13.6, picking up the Linux 6.18.38 kernel fixes for CVE-2026-46113 and CVE-2026-53359 (KVM shadow-paging use-after-free, nested-guest escape into the worker VM). Nested virtualization remains available. Applying this upgrade rolls each tenant worker pool once, as the worker template is renamed to carry the new image.
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Updated the default Talos worker release to v1.13.6 for Kubernetes
cluster provisioning.
- Improved compatibility checks to apply consistently across Talos patch
releases within the same minor version.

- **Bug Fixes**
- Corrected generated resource naming expectations when storage settings
or GPU configuration change.

- **Tests**
- Added coverage confirming Kubernetes clusters render successfully with
Talos v1.13.6.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
v1.6.0-rc.2 		2026-07-10 11:34:48 +02:00
renovate[bot] ebecbd71c9 chore(deps): update altinity/clickhouse-backup docker tag to v2.7.4 2026-07-10 07:24:26 +00:00
myasnikovdaniil c8ee5dc317 chore(deps): update container image & action digests (#3232) 
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [actions/stale](https://redirect.github.com/actions/stale)
([changelog](https://redirect.github.com/actions/stale/compare/eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899..1e223db275d687790206a7acac4d1a11bd6fe629))
| action | digest | `eb5cf3a` → `1e223db` |
| alpine | final | pinDigest |  → `28bd5fe` |
| alpine | stage | pinDigest |  → `28bd5fe` |
|
[altinity/clickhouse-backup](https://redirect.github.com/Altinity/clickhouse-backup)
| final | pinDigest | → `b8a098d` |
| clickhouse/clickhouse-server | final | pinDigest |  → `f21e0f9` |
| debian | final | pinDigest |  → `ee12ffb` |
| debian | final | pinDigest |  → `30482e8` |
| debian | stage | pinDigest |  → `30482e8` |
| debian | final | pinDigest |  → `28de087` |
| docker.io/golang | stage | pinDigest |  → `079e598` |
| docker.io/library/alpine | final | pinDigest |  → `1435830` |
| docker.io/library/alpine | final | pinDigest |  → `28bd5fe` |
| docker.io/library/golang | stage | pinDigest |  → `079e598` |
| [docker/dockerfile](https://redirect.github.com/moby/buildkit) |
syntax | pinDigest | → `e2a8561` |
| gcr.io/distroless/static | final | pinDigest |  → `d29e660` |
| gcr.io/distroless/static | stage | pinDigest |  → `d29e660` |
| golang | final | pinDigest |  → `0178a64` |
| golang | stage | pinDigest |  → `079e598` |
| golang | stage | pinDigest |  → `0178a64` |
| golang | final | pinDigest |  → `079e598` |
| [grafana/grafana](https://redirect.github.com/grafana/grafana) | final
| pinDigest | → `f739273` |
|
[nginxinc/nginx-unprivileged](https://redirect.github.com/nginx/docker-nginx-unprivileged)
| final | pinDigest | → `0c79d56` |
| [node](https://redirect.github.com/nodejs/node) | stage | pinDigest |
→ `16e22a5` |
| quay.io/centos/centos | final | pinDigest |  → `0c9db82` |
| quay.io/cilium/cilium | stage | pinDigest |  → `20fbbc1` |
|
[quay.io/poseidon/matchbox](https://redirect.github.com/poseidon/matchbox)
| final | pinDigest | → `06bcdae` |
| ubuntu | final | pinDigest |  → `4fbb8e6` |
| ubuntu | final | pinDigest |  → `0e0a0fc` |
| ubuntu | stage | pinDigest |  → `0e0a0fc` |

---

### Configuration

📅 **Schedule**: (UTC)

- Branch creation
  - "before 6am on monday"
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/cozystack/cozystack).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI0Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==-->
 2026-07-10 12:22:35 +05:00
Myasnikov DaniilandClaude Opus 4.8 302a72f14a Merge origin/main into feat/e2e-chainsaw-example 
Reconciles the talos-image-cache conflicts created when #3254 merged to
main: keep the Chainsaw _lib/ port (hack/e2e-chainsaw/_lib/talos-image-cache.sh,
a superset of #3254's logic + the node-join diagnose helper), drop the
BATS-era hack/e2e-apps/talos-image-cache.sh, and keep the _lib/-path
versions of the mirror manifest, install bats, and unit test. Also picks
up main's zizmor action-pinning (#3223, unblocks pre-commit/zizmor) and
Talos v1.13.6 (#3240).

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-10 12:00:01 +05:00
Myasnikov DaniilandClaude Opus 4.8 02b87b63f2 fix(apps/kubernetes): bound the pre-delete hook's tenant HelmRelease wait (#3271) 
The pre-delete hook (delete.yaml) deleted the tenant's addon HelmReleases with
an unbounded `kubectl delete … --wait=true`. When the tenant has no working
nodes (LINSTOR/ZFS exhaustion, worker VMs that never joined, or a nodeGroup at
minReplicas: 0), those uninstalls cannot converge — in-tenant cilium stays
InstallFailed because cilium-operator has nothing to schedule on — so the hook
hung indefinitely, the parent Helm uninstall failed on the pre-cleanup Job, and
the Kubernetes CR was stuck deleting.

Bound the wait with --timeout=120s; on expiry, force-clear the Flux finalizer
from any child HelmRelease still present so it can be garbage-collected — safe
because the whole tenant (control plane torn down in the steps below) is being
destroyed, so there is nothing left to orphan. A healthy tenant's children
uninstall well within the timeout and never reach the force path.

Also unblocks the render-side kubernetes-oidc-{system,customconfig} Chainsaw
e2e suites, which deterministically hit this teardown hang under the Chainsaw
cleanup budget.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-10 11:18:32 +05:00
renovate[bot] 5b9b420ffe chore(deps): update container image & action digests 2026-07-10 05:29:23 +00:00
Aleksei Sviridkin a3829ab66c fix(talos): bump Talos to v1.13.6 (CVE-2026-53359) (#3240) 
## What this PR does

Bumps the platform Talos profiles from v1.13.5 to v1.13.6.

CVE-2026-53359 and CVE-2026-46113 are related KVM x86 shadow-paging
use-after-free bugs that let a guest escape to the host, and the fix is
complete only once both upstream patches (`81ccda30b4e8`,
`0cb2af2ea66a`) are applied. On the 6.18 stable line CVE-2026-46113
landed in 6.18.30 and CVE-2026-53359 in 6.18.38. Talos v1.13.5 runs
Linux 6.18.36, so it carries only the first of the two. Talos v1.13.6
runs Linux 6.18.38 and carries both.

Nested virtualization stays enabled. The bugs are fixed in the kernel,
so there is no reason to take the feature away from workloads that rely
on it — bumping the kernel is the complete fix.

Profiles are regenerated with `hack/gen-profiles.sh v1.13.6`. Alongside
the Talos version and the installer reference, this refreshes the pinned
system-extension digests: the firmware extensions move from the 20260519
to the 20260622 build, and `intel-ucode` keeps its `20260512` tag but
resolves to a new digest because upstream re-pushed that tag as an OCI
image index (identical config and layer blobs underneath).

References: [Talos v1.13.6 release
notes](https://github.com/siderolabs/talos/releases/tag/v1.13.6)
(`Linux: 6.18.38`)

### Screenshots

Not applicable, no UI changes.

### Release note

```release-note
fix(talos): bump Talos Linux to v1.13.6, which ships Linux 6.18.38 with the kernel fixes for the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes
```
 2026-07-10 04:30:23 +03:00
myasnikovdaniil 06b17bf7a9 ci: move remaining workflows off the persistent self-hosted runner (#3268) 
## What this PR does

Completes the last step of #2937 — no workflow depends on the persistent
self-hosted runner anymore, so it can be decommissioned. PR builds, the
main cache warmer, nightly, promote-rc, and pull-requests-release
already run on the ephemeral pool; this PR moves the stragglers:

- **`tags.yaml` / `prepare-release`** moves to the large ephemeral shape
(same as the main cache warmer). Release builds now warm-start from the
shared `mode=max` build cache in OCIR (written by `build-main.yaml` on
every push to main) while still pushing images to GHCR. Cache manifests
are registry-portable, and a missing cache ref degrades harmlessly into
a cold build. Without this wiring, every tag build on an ephemeral
runner would start 100% cold — the persistent runner used to warm-start
from local buildkit state.
- **`tags.yaml` / `generate-changelog` and `update-website-docs`**,
**`backport.yaml`**, **`update-releasenotes.yaml`** move to
GitHub-hosted runners: these jobs only need git and the GitHub API, no
docker or repo toolchain.
- **`pull-requests.yaml`**: the `debug` label no longer reroutes jobs to
the self-hosted runner — that path would queue forever once the runner
is gone. The label still gates the SSH breakpoint on e2e failure, which
works from ephemeral runners (the breakpoint connects out to the
rendezvous server).
- **`flux-shard-operator`** was the last image built with the stale
inline/`:latest` cache pattern (always cold since #2711); it now uses
the shared `cache-args` registry cache like every other image.

Operational notes for the actual decommission:

- The breakpoint rendezvous server (`BREAKPOINT_ENDPOINT`) is separate
infrastructure — if it happens to live on the same host as the runner,
it needs a new home before the host is retired.
- `tags.yaml` now uses the `OCIR_USER`/`OCIR_TOKEN` secrets on tag
events (already used there by `nightly.yaml` and `release-e2e` before
it).

### Screenshots

Not applicable — CI-only change.

### Release note

```release-note
ci: all CI jobs now run on ephemeral or GitHub-hosted runners; release tag builds warm-start from the shared registry build cache
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Updated multiple GitHub Actions workflows to use more consistent
hosted runner environments for checks, PR validation, tagging,
backports, and release note updates.
* Improved tag/release build setup for ephemeral runners, including
optional build toolchain setup and container registry authentication.
* Enhanced image build caching for the shard operator to improve
packaging efficiency.
* **Bug Fixes**
* Removed label-driven runner switching in pull request checks to make
build and test execution more predictable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-10 01:27:46 +05:00
Andrei Kvapil 814a47f23e ci: move remaining workflows off the persistent self-hosted runner 
Finishes the runner decommission tracked in #2937. PR builds, the main
cache warmer, nightly, promote-rc and pull-requests-release already run
on the ephemeral pool; this moves the stragglers so no workflow depends
on the persistent runner anymore:

- tags.yaml prepare-release: large ephemeral shape (same as the cache
  warmer), flux CLI bootstrap, and read-only OCIR access so release
  builds warm-start from the shared mode=max build cache while still
  pushing images to GHCR. Without the cache wiring every tag build on
  an ephemeral runner would be 100% cold.
- tags.yaml generate-changelog / update-website-docs, backport.yaml,
  update-releasenotes.yaml: GitHub-hosted runners — these jobs only
  need git and the GitHub API, no docker or repo toolchain.
- pull-requests.yaml: the debug label no longer reroutes jobs to the
  self-hosted runner (that path would queue forever once the runner is
  gone). The label keeps gating the SSH breakpoint on e2e failure,
  which works from ephemeral runners too.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
 2026-07-09 20:50:43 +02:00
Andrei Kvapil 681f4a8313 build(flux-shard-operator): read the shared registry build cache 
flux-shard-operator was the last image built with the pre-#2937 cache
pattern: --cache-from a :latest tag that PR builds no longer publish and
--cache-to type=inline, which never caches multistage builder layers. Use
the shared cache-args macro so it reads (and, on main, writes) the same
mode=max :buildcache ref in CACHE_REGISTRY as every other image.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
 2026-07-09 20:49:43 +02:00
Aleksei Sviridkin 3e9b5600e3 fix(talos): bump Talos to v1.13.6 (CVE-2026-53359) 
Talos v1.13.5 ships Linux 6.18.36, which carries the fix for
CVE-2026-46113 but not the one for CVE-2026-53359. The two are related
KVM x86 shadow-paging bugs allowing a guest to escape to the host, and
both patches are needed for the mitigation to hold. Talos v1.13.6 ships
Linux 6.18.38, the first release on the 6.18 stable line that has both.

Profiles regenerated with hack/gen-profiles.sh v1.13.6. Alongside the
Talos version and the installer reference, this refreshes the pinned
system-extension digests: the firmware extensions move from the 20260519
to the 20260622 build, and intel-ucode stays on its 20260512 tag but
resolves to a new digest because upstream re-pushed that tag.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 20:29:40 +03:00
IvanHunters ca93adc728 test(e2e): make the in-sandbox Talos image cache reachable by tenant workers (#3254) 
## What this PR does

The in-sandbox Talos image cache added in #3244 is unreachable by the
tenant workers it was meant to serve, so it makes the
`kubernetes-latest` / `kubernetes-previous` e2e tests fail
deterministically instead of relieving the intermittent public-factory
flake it targets (#3231). Every branch off current `main` inherits this
— for example #3240 has failed its tenant-Kubernetes e2e repeatedly for
this reason alone, despite being unrelated (it only touches host boot
assets).

Root cause: tenant namespaces run under a default-deny Cilium egress
installed by the tenant chart. A worker's CDI importer is allowed to
reach the outside world (so the public factory works) and kube-dns (so
the name resolves), but nothing permits egress to an arbitrary
`kube-system` Service. The importer therefore resolves
`talos-image-cache.kube-system.svc` yet its TCP connect to the ClusterIP
is silently dropped (`dial tcp <clusterip>:80: i/o timeout`), the
DataVolume import never starts, the worker VM stays `DataVolumeError`,
and no tenant node joins within the 12-minute deadline. #3244's
readiness gate never caught this because it probed the server over
`localhost` inside the serve Pod, which passes regardless of cross-Pod
reachability.

Two changes:

- Ship a tightly-scoped `CiliumClusterwideNetworkPolicy` that lets
importer Pods in `tenant-test` egress to the mirror Pod in
`kube-system`. Cilium unions allow rules, so this only adds a hole;
scoping it to a namespace that is already default-deny egress means it
never flips another namespace's posture, and the mirror Pod needs no
ingress rule because no ingress policy selects it. The policy is applied
at point-of-use (not with the rest of the mirror manifest, which is
applied before Cilium's CRDs exist).
- Replace the localhost range-probe with one that faces the exact same
network path as a real importer: a throwaway Pod in the tenant
namespace, labelled `cdi.kubevirt.io=importer`, that fetches the seeded
image from the Service ClusterIP with a Range request and must get a
`206`. Tenants are pointed at the mirror only when that end-to-end check
passes; otherwise the harness falls back to the public factory. This
restores the invariant #3244 intended — the mirror can only help, never
make CI worse.

Not a retry/timeout bump: it fixes the real network path and adds a
genuine reachability gate.

### Screenshots

N/A — CI-only change, no user-facing surface.

### Release note

```release-note
test(e2e): make the in-sandbox Talos image cache reachable by tenant Kubernetes workers, fixing deterministic kubernetes-* e2e failures (#3231)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Updated Talos image cache e2e selection to run a tenant-scoped
reachability probe to the local mirror and require HTTP byte-range
support (`206`) before using it.
* If the probe fails or does not confirm byte-range behavior, the flow
now reliably falls back to the public Talos image source.
* Added CI-only tenant egress controls to allow importer Pods to reach
the mirror Service.
* **Documentation**
* Expanded e2e guidance to clarify the mirror is best-effort and that
fallback happens at probe time.
* **Tests**
* Added a new Bats suite covering manifest splitting, policy wiring,
probe overrides, and strict `206` gating/fallback scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 20:18:57 +03:00
myasnikovdaniil 6b1e170e44 feat(release): add Cut Pre-release Tag workflow (#2677) (#3253) 
## What this PR does

Adds `cut-prerelease.yaml` — a `workflow_dispatch` that becomes the
**sole entry point for creating a pre-release tag** (`vX.Y.Z-alpha.N` /
`-beta.N` / `-rc.N`), replacing the manual `git push origin
HEAD:refs/tags/<tag>`.

## Why

#2677 makes stable tags immutable, but tag *creation* was still only
convention-enforced for **pre-releases**. Repo rulesets **can** already
restrict *stable* `vX.Y.Z` creation to CI — a "Restrict creations"
ruleset targeting `v*` with an exclude pattern for `v*-*` covers stable
tags without any new workflow. What that approach **cannot** cover is
pre-releases: `-rc` / `-alpha` / `-beta` tags were hand-pushed, so they
can't be locked to CI while they still have a human entry point.

This workflow closes that gap by making CI the sole creator of
pre-release tags too. With it in place, an admin can extend "Restrict
creations" (CI app as sole bypass) to **all** of `v*` — pre-release and
stable alike — instead of only the stable subset.

_(Correction: an earlier version of this description claimed a ruleset
couldn't express "stable-only" creation. It can, via include + exclude
patterns — thanks @IvanHunters. The workflow's real value is extending
creation-control to pre-release tags, plus the audit trail, single entry
point, guaranteed `base_ref`, and app-identity push that makes
`tags.yaml` fire.)_

## How it works

- Generates the CozyStack CI app token (same app as `promote-rc.yaml`)
and pushes the tag **as the app** — so `tags.yaml` fires (a tag pushed
with the default `GITHUB_TOKEN` would not) and the push event's
`base_ref` is populated (`tags.yaml`'s *Get base branch* step requires
it).
- Validates `^v\d+\.\d+\.\d+-(alpha|beta|rc)\.\d+$` and **rejects stable
`vX.Y.Z`** — stable tags are cut write-once by the promote flow at a PR
merge commit, never here.
- Gates on the dispatcher having **write access** to the repo
(`workflow_dispatch` already requires it; the explicit check is
auditable defense-in-depth that fails closed).
- Enforces dispatch from `main` (new minor's `vX.Y.0-*` only) or the
matching `release-X.Y` branch; write-once (refuses an existing tag) with
a stale-tip guard so a branch that advanced since dispatch fails cleanly
instead of burning the tag name.
- Pushes `HEAD:refs/tags/<tag>` from the branch tip — the exact form
`docs/release.md` prescribes. Runs on `ubuntu-latest` (no build runner
needed).

Also updates `tags.yaml`'s two human-facing error messages and
`docs/release.md` (the RC and patch flows, the workflow list, and a new
repo-side tag-ruleset enforcement note) to point at the workflow instead
of a manual push. Completes SHA-pinning on the touched release workflows
(`nightly.yaml`, `promote-rc.yaml`, `retention.yaml`) — the zizmor
"Audit workflows" gate was red on `main` for those.

## Rollout ordering

The two `v*` tag rulesets have a hard order (now documented in
`docs/release.md`):

1. **Immutability** (*Restrict updates* + *deletions*, no bypass) — safe
to enable at any time; nothing in the flow ever moves or deletes a `v*`
tag.
2. **Creation control** (*Restrict creations*, CI app sole bypass) —
enable **only after this PR ships**, since this workflow is what makes
CI the sole creator of pre-release tags. Enabling it earlier would lock
maintainers out of cutting rc/alpha/beta.

## Validation

- `actionlint` — clean.
- `act -l` — job graph resolves (`cut` / `workflow_dispatch`).
- `zizmor --offline` — no findings across the whole workflows dir
(SHA-pinned actions, least-privilege `contents: read`).
- Run blocks `bash -n` syntax-checked.
- **Untested locally:** the workflow runtime (app-token auth, the real
tag push, `base_ref` population, the `getCollaboratorPermissionLevel`
gate) needs a live run with the CI secrets — the same "orchestration is
unproven until it runs" caveat #3017 carried.

## Review follow-up

Addressed the non-blocking findings from @IvanHunters' review (commit
`fix(ci): address review findings`):

1. **[MINOR]** `main` dispatch now requires patch `Z == 0` (a new
minor's pre-release), so a patch-line rc can't be cut from `main`
content.
2. **[MINOR]** Added a stale-tip guard: if the dispatch branch advanced
between dispatch and push, fail with a re-run hint instead of tagging a
stale commit / burning the write-once name on an empty-`base_ref`
failure.
3. **[MINOR]** Corrected the PR-body rationale above (rulesets can
express stable-only creation via include + exclude).
4. **[NIT]** `ls-remote` now distinguishes "tag absent" (exit 2) from a
transport/auth failure.
5. **[NIT]** Shallow checkout (`fetch-depth: 1`, no `fetch-tags`);
dropped the redundant local tag pre-check.
6. **[NIT]** Pure-input validation now runs before the token mint +
checkout (fail-fast).

Plus a **write-access gate** on the dispatcher, requested separately.

Part of #2677.

## Release note

```release-note
NONE
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a controlled workflow to create pre-release tags (alpha/beta/rc)
with write-once behavior.
* Added stronger enforcement so pre-releases are cut from the correct
release line.
* **Bug Fixes**
* Improved guidance when stable tags are rejected or when base branch
info is missing.
* Added safeguards to prevent cutting tags from an out-of-date branch
tip.
* **Documentation**
* Updated release process docs, including new tag protection rulesets
for `v*` tags.
* **Chores**
* Pinned multiple CI workflow actions to fixed versions for reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 18:45:46 +05:00
Myasnikov DaniilandClaude Opus 4.8 ff0d0d77a7 test(e2e): make the talos image cache reachable by tenant workers (#3254) 
The in-sandbox Talos image mirror (#3244) is unreachable by the tenant
workers it serves, so the kubernetes-* e2e tests fail deterministically at
the 12m node-join wait instead of relieving the public-factory flake (#3231).

Root cause (from #3254 by @lexfrei, verified against the failing run's
crust-gather snapshot): tenant namespaces run a default-deny Cilium egress
(allow-internal/external-communication + the <tenant>-egress CCNP) that
permits the world entity and the tenant tree but NOT an arbitrary kube-system
Service, so a worker's CDI importer resolves talos-image-cache.kube-system.svc
yet its TCP connect to the ClusterIP is silently dropped and no disk imports.
#3244's gate never caught this because it probed the server over localhost
inside the serve pod, which passes regardless of cross-pod reachability.

Port of #3254 to the Chainsaw layout (hack/e2e-chainsaw/_lib/):
- ship a tightly-scoped CiliumClusterwideNetworkPolicy (importer pods in
  tenant-test -> mirror pod in kube-system), applied at point-of-use once
  Cilium CRDs exist; excluded from the pre-Cilium install apply via yq.
- replace the readiness probe with a throwaway pod in tenant-test labelled
  cdi.kubevirt.io=importer that curls the ClusterIP with a range request and
  must get 206 - the exact egress path a real importer faces; tenants use the
  mirror only when that end-to-end check passes, else fall back to the public
  factory, so it can only help, never make CI worse.
- unit test pinning the manifest split, probe/policy label agreement, the
  strict 206 gate, and the --overrides JSON builder.

Also adds (a2) DataVolume import-stage diagnostics to the node-join failure
block in run-kubernetes.sh (importer logs + a ClusterIP re-probe via
talos_image_cache_diagnose) so the import-stage failure mode is legible.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 18:21:07 +05:00
Aleksei Sviridkin b73860d4c9 fix(kubernetes): bump tenant worker Talos to v1.13.6 
Tenant worker VMs were pinned to Talos v1.13.0, which ships Linux 6.18.24.
That kernel predates both fixes for a pair of related KVM x86 shadow-paging
use-after-free bugs: CVE-2026-46113, fixed in 6.18.30, and CVE-2026-53359,
fixed in 6.18.38. A worker's own kernel runs KVM whenever a tenant enables
nested virtualization inside the VM, and either bug lets a nested guest
escape into the worker.

Talos v1.13.6 ships Linux 6.18.38. Stable kernel branches are cumulative, so
it is the first v1.13 patch release carrying both fixes: the pinned v1.13.0
ships 6.18.24 and carries neither, while v1.13.5 ships 6.18.36 and carries
only CVE-2026-46113. Nested virtualization stays available to tenants — the
kernel bump is the fix.

The new pin changes the rendered worker spec, so the content hash in the
KubevirtMachineTemplate and talos-reconcile Job names rotates and each
worker pool rolls once on upgrade. Pinned hashes in the chart tests are
recomputed accordingly.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 15:59:08 +03:00
Aleksei Sviridkin 1a0680b778 test(e2e): allow tenant CDI importers to reach the image cache 
Tenant namespaces run under a default-deny Cilium egress installed by
the tenant chart, which lets a worker's CDI importer reach the outside
world and kube-dns but nothing in kube-system. The in-sandbox image
mirror is a kube-system Service, so without an explicit allow the
importer's connect to its ClusterIP is dropped and the worker disk
import never starts.

Ship a tightly-scoped CiliumClusterwideNetworkPolicy in the mirror
manifest that lets importer Pods in tenant-test egress to the mirror
Pod in kube-system. Cilium unions allow rules, so this only adds a
hole; scoping it to tenant-test (already under default-deny egress)
means it never flips another namespace's posture, and the mirror Pod
needs no ingress rule because it is selected by no ingress policy. The
policy is kept out of the pre-install apply because that runs before
Cilium's CRDs exist; the resolver in hack/e2e-apps/talos-image-cache.sh
applies it at point-of-use once Cilium is up.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 15:56:02 +03:00
Aleksei Sviridkin f5ec885d8d test(e2e): gate the talos image cache on real tenant reachability 
The in-sandbox Talos image mirror was pointed at by tenant Kubernetes
CRs whenever its Deployment was Available, but "Available" only proves
the seed finished and the server answers on localhost. It does not
prove a worker's CDI importer can reach the Service. Tenant namespaces
run under a default-deny Cilium egress that permits the outside world
and kube-dns but not an arbitrary kube-system Service, so the importer
resolves the name yet its TCP connect to the ClusterIP is silently
dropped. The disk import then never starts, no tenant node joins within
the 12-minute deadline, and the kubernetes-* tests fail
deterministically -- worse than the intermittent public-factory flake
the mirror was meant to relieve.

Replace the localhost range-probe with one that faces the exact same
network path as a worker importer: a throwaway Pod in the tenant
namespace, labelled cdi.kubevirt.io=importer, that fetches the seeded
image from the Service ClusterIP with a Range request and expects a
206. Point tenants at the mirror only when that end-to-end check
passes; otherwise fall back to the public factory, restoring the
guarantee that the mirror can only help, never make CI worse. The
resolver also installs the importer egress allow at point-of-use,
because Cilium's CRDs do not exist yet when the mirror manifest is
first applied before install.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 15:49:59 +03:00
Aleksei Sviridkin b420d3de02 fix(kubernetes): key Talos support matrix on minor version 
The Talos<->Kubernetes support window is declared per Talos minor release,
and patch releases never move it. Keying the chart's matrix dict on the full
patch version meant every Talos patch bump fell out of the dict and silently
disabled the render-time guard — precisely the mismatch the guard exists to
catch.

Derive the minor from talos.version before the lookup, so the guard keeps
firing across patch releases of a known minor. Unknown minors are still
skipped, leaving compatibility to the operator who overrode the pin.

Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 15:47:22 +03:00
Myasnikov DaniilandClaude Opus 4.8 4a2e43969b fix(ci): polish cut-prerelease per re-review nits (#3253) 
Cosmetic follow-ups from the re-review (all fail-closed, no behavior change):

- Write-access gate: drop the inert 'maintain' from the allowed list.
  getCollaboratorPermissionLevel's `permission` field returns only the legacy
  roles admin/write/read/none (maintain collapses to write), so admin+write is
  the full "can push" set; 'maintain' never matched.
- Stale-tip guard: resolve the remote tip without a pipe masking ls-remote's
  exit code, mirroring the write-once query — a transport failure now reports a
  distinct error instead of "branch not found".
- tags.yaml: the hand-pushed-stable rejection message now says stable tags are
  finalized by the promote flow at the merge commit (pull-requests-release.yaml),
  not "come from promote-rc.yaml" — promote-rc opens the PR; finalize cuts the tag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 17:33:32 +05:00
Myasnikov DaniilandClaude Opus 4.8 31cda39dcd fix(ci): address review findings on cut-prerelease (#3253) 
Non-blocking review notes from #3253:

- Require write access to dispatch. workflow_dispatch already needs write
  access to trigger, but add an explicit getCollaboratorPermissionLevel gate
  (admin/maintain/write) as auditable defense-in-depth that fails closed.
- Constrain a `main` dispatch to a new minor's pre-release (patch Z == 0). A
  patch-line pre-release built from main content would produce a wrong-tree
  release-X.Y.Z-rc.N staging branch; it must be cut from its release-X.Y line.
- Guard against a stale checkout tip. workflow_dispatch pins the dispatch-time
  SHA, not the push-time branch tip (the gap includes the concurrency-queue
  wait); if the branch advanced, the push would tag a stale commit and yield an
  empty base_ref, failing tags.yaml after burning the write-once name. Compare
  local HEAD to the remote branch tip and fail with a re-run hint instead.
- Distinguish ls-remote "tag absent" (exit 2) from a transport/auth failure
  (any other non-zero), so an infra blip cannot read as "free to create".
- Shallow checkout (fetch-depth: 1, no fetch-tags) instead of a full-history
  clone to push one ref; drop the redundant local tag pre-check.
- Run pure-input validation (tag + dispatch branch) before minting the token
  or checking out, so a bad dispatch fails in seconds.

actionlint + zizmor clean; run blocks bash -n checked.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 17:00:14 +05:00
Myasnikov DaniilandClaude Opus 4.8 bece285678 ci: pin action refs flagged by the zizmor gate (#3223) 
The zizmor "Audit workflows" gate (added in #3223) is red on main: five
actions/* references in the release workflows from #3017 are pinned to a
floating tag instead of a commit SHA (blanket unpinned-uses policy, High
severity):

- nightly.yaml (x3)  actions/checkout@v4
- promote-rc.yaml     actions/checkout@v4
- retention.yaml      actions/create-github-app-token@v1

Pin each to the SHA the rest of the repo already standardizes on:
- actions/checkout                -> 34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
- actions/create-github-app-token -> d72941d797fd3113feb6b93fd0dec494b13a2547 # v1

`zizmor --offline .github/workflows/` now reports no findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 13:15:39 +05:00
Myasnikov DaniilandClaude Opus 4.8 47eec0157f feat(release): add Cut Pre-release Tag workflow (#2677) 
Pre-release tags (vX.Y.Z-alpha.N / -beta.N / -rc.N) were cut by a manual
`git push origin HEAD:refs/tags/<tag>`. That leaves the immutable-tag work in
#2677 only partly enforceable: a repo ruleset cannot restrict `v*` tag
*creation* to CI while still letting maintainers hand-push pre-releases (fnmatch
has no way to match "v* except v*-rc.*"), so "do not hand-cut stable tags" stays
a matter of discipline backed only by an in-workflow guard.

Add cut-prerelease.yaml — a workflow_dispatch that becomes the sole entry point
for creating a pre-release tag:

- Generates the CozyStack CI app token (same app as promote-rc.yaml) and pushes
  the tag as the app, so tags.yaml fires (a tag pushed with the default
  GITHUB_TOKEN would not trigger it) and the push-event base_ref is populated
  (tags.yaml's Get-base-branch step needs it).
- Validates ^v\d+\.\d+\.\d+-(alpha|beta|rc)\.\d+$ and rejects stable vX.Y.Z —
  stable tags are cut write-once by the promote flow at a PR merge commit.
- Enforces dispatch from main or the matching release-X.Y branch, and refuses to
  move an existing tag (write-once).
- Pushes HEAD:refs/tags/<tag> from the branch tip — the exact form
  docs/release.md prescribes — on ubuntu-latest (no build runner needed).

With every pre-release cut by CI, repo admins can then enable a "Restrict
creations" tag ruleset on `v*` with the CI app as the sole bypass actor, turning
tag-creation discipline into hard enforcement.

Also update tags.yaml's two human-facing error messages and docs/release.md (the
RC and patch flows, the workflow list, and a new repo-side tag-ruleset
enforcement note) to point at the workflow instead of a manual push.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 12:21:58 +05:00
Myasnikov DaniilandClaude Fable 5 1f84d2d49e Merge origin/main into feat/e2e-chainsaw-example (post-#3244) 
Fold the in-sandbox Talos image cache work into the Chainsaw layout:
the new shared helper relocates to hack/e2e-chainsaw/_lib/
talos-image-cache.sh and run-kubernetes.sh sources it from there (the
ouroboros DNS-check no-retry rework merges clean). The OIDC bats are
resolved as deleted again; their Chainsaw fixtures deliberately skip
the talos.imageFactoryURL injection — the helper only emits it when
the cache is up, and the zero-worker render-side tests instantiate no
DataVolume imports for it to redirect.

Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
 2026-07-09 10:38:35 +05:00
scooby87 f91841fce6 fix(monitoring): surface missing VictoriaLogs and gate dead-end vlinsert ExternalName (#3215) 
## What this PR does

Addresses #3181, where platform log ingestion could fail **silently**:
fluent-bit shipped to `vlinsert-generic` while the monitoring
`HelmRelease` reported `Ready`, and a `vlinsert-generic` `ExternalName`
in `cozy-monitoring` dead-ended at a `tenant-root` VictoriaLogs that did
not exist when the root tenant had monitoring disabled. Logs were
dropped with no failing signal.

Three complementary layers make the failure loud instead of silent:

1. **Readiness gate** —
`packages/extra/monitoring/templates/helmrelease.yaml` now sets a
`poller` `waitStrategy` and a `healthCheckExprs` entry on the
`VLCluster` `status.updateStatus`. A missing/unhealthy log store makes
the `monitoring-system` release **not-ready** instead of flipping
`Ready` the moment the CR is applied. Mirrors the existing seaweedfs
CNPG pattern.
2. **No empty-log-store hole** —
`packages/system/monitoring/templates/vlogs/vlogs.yaml` fails the render
on an empty `logsStorages`. With no entry no `VLCluster` exists, the
readiness gate has nothing to match, and fluent-bit would silently drop
logs — so this is now a loud misconfiguration.
3. **No silent fallback** — the `cozy-monitoring → tenant-root`
`vlinsert`/`vminsert` `ExternalName` redirects in `cozystack-basics` are
created only when the root tenant hosts monitoring. The template reads
the live `Tenant/root` `spec.monitoring` via `lookup` (source of truth,
so it cannot drift from the tenant's real state) and falls back to a new
platform value `monitoring.rootEnabled` (via
`_cluster.monitoring-enabled`) when `lookup` is empty (dry-run / first
render). The default preserves the historical always-create behaviour.

Covered by helm-unittest across all four affected charts (141 tests):
readiness-gate fields, empty-vs-populated `logsStorages` contract,
`ExternalName` present/absent fallback contract, and the platform wiring
into the `cozystack-values` Secret. Also adds the missing `test:` target
to `packages/extra/monitoring/Makefile` so its suite runs in CI.

**Known limitation / follow-up:** the legacy platform monitoring release
installed directly into `cozy-monitoring` via the `cozystack.monitoring`
PackageSource cannot carry the readiness gate, because the engine's
`ComponentInstall`/HelmRelease builder and the vendored
`helm-controller` API v1.4.3 lack `waitStrategy`/`healthCheckExprs`.
That source is not referenced by any active bundle; full coverage there
needs a helm-controller bump plus engine support.

### Release note

```release-note
fix(monitoring): surface missing platform VictoriaLogs as not-ready and stop the dangling vlinsert-generic ExternalName from silently dropping logs (#3181)
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a platform setting (`monitoring.rootEnabled`) to control whether
root-level monitoring is enabled.
* Monitoring redirect ExternalName services are now rendered only when
root monitoring is enabled (defaulting to enabled).
* **Bug Fixes**
* Ensures an explicitly set `monitoring.rootEnabled=false` is preserved
when generating platform configuration.
* Prevents incomplete monitoring deployments by failing rendering when
`logsStorages` is empty.
* Improves VictoriaMetrics readiness by delaying “ready” until the
operator reports an operational status.
* **Tests**
* Added/expanded Helm-unittest suites covering monitoring wiring,
redirects, readiness gating, and `logsStorages` requirements.
  * Added a `make test` target to run chart unittests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 01:55:01 +03:00
Aleksei Sviridkin a53d64d67a ci(e2e): move install gate to 32vCPU runner to end CPU oversubscription (#3251) 
## What this PR does

The E2E `Install Cozystack` gate flakes intermittently: a platform
HelmRelease/Deployment sticks `InProgress` → `Failed` and its dependents
cascade to `DependencyNotReady`, on a **different component each run**
(observed: opensearch-operator-controller-manager,
monitoring-agents-kube-state-metrics, kamaji readiness "context deadline
exceeded"). That non-determinism is the signature of host resource
thrash, not a product bug.

**Measured envelope.** The sandbox boots three QEMU guests
(`hack/e2e-prepare-cluster.bats`), each `-smp 8 -m 24576`:

| Resource | Guests demand | 24-vCPU runner (current) | 32-vCPU runner
(this PR) |
| --- | --- | --- | --- |
| vCPU | 3 × 8 = 24 | 24 → 0 host headroom | 32 → 8 (25%) headroom |
| RAM | 3 × 24 = 72 GiB | 96 GiB → 24 GiB headroom | 128 GiB → 56 GiB
headroom |

On the 24-vCPU runner the guests alone pin all 24 host vCPUs, leaving
nothing for QEMU's own emulation/IO threads, vhost-net, the host
`iptables MASQUERADE` every guest image pull traverses, containerd, the
kernel and the runner agent. Under the peak concurrency of the platform
install the guest vCPUs contend with those host threads and lose cycles
to CPU steal, so in-guest kubelet/apiserver readiness/liveness probes
miss their deadlines and the HelmRelease flips. Across sampled failed
runs the symptom is always a probe/readiness timeout with
`DependencyNotReady`, never `OOMKilled`/`Evicted`/`MemoryPressure` — CPU
steal, not memory, is the binding constraint (RAM already keeps 24 GiB
of headroom).

**Fix.** Move the job to the `oracle-vm-32cpu-128gb-x86-64` class
(already referenced in-repo as the commented alternative). Guest sizing
is unchanged (24 vCPU / 72 GiB), so the CPU-bound install does not slow
down, while the host keeps 8 vCPU + 56 GiB of headroom that absorbs the
emulation/network overhead the probes were starving against.

This is deliberately **not** a retry or timeout bump (forbidden by
`docs/agents/e2e-testing.md`): the install gate keeps its teeth, runs
once, and still fails loud on a genuine product bug. It removes the
harness oversubscription that manufactured false failures, and is
complementary to the per-component probe-hardening work and to the
in-sandbox image-mirror change (which relieves the distinct
storage-import axis).

**Trade-off / caveat.** The 32-vCPU class costs more per E2E run and
adds load to that runner pool — an infra/cost decision for maintainers.
A zero-cost alternative (drop guest `-smp` to 6 → 18 vCPU / 6 host
headroom on the existing runner) was considered and rejected as primary:
it cuts guest compute 25% on an already CPU-bound install and could
trade steal-flakes for wall-clock-timeout flakes, unverifiable without a
CI run.

### Screenshots

N/A — CI-only change, no user-facing surface.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Increased the default runner size for end-to-end pull request checks.
* Added clearer workflow guidance to reduce flaky readiness and liveness
failures during installation steps.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 01:49:09 +03:00
Aleksei Sviridkin 0814586f0f feat(apps/kubernetes): overridable Talos image factory URL + in-sandbox e2e mirror (#3244) 
## Problem

`kubernetes-*` e2e tests fail intermittently and each take 18–22 min.
Root cause: every tenant worker VM imports its Talos OS disk directly
from the **public** `factory.talos.dev`. That endpoint has no byte-range
support (CDI falls back to a full copy-to-scratch) and its egress
stream-resets / stalls mid-transfer from the CI runner, so a worker
DataVolume import hangs past the chart's 12-minute node-join deadline →
0 tenant nodes join → test fails.

Seen on run `28924455578` (`kubernetes-previous`) and, on an unrelated
branch, `28929095889` (`kubernetes-latest`) — so it's a shared
external-dependency flake, not a per-PR bug.

## Changes

**1. Overridable image source (chart).** Two new `talos.*` values, both
defaulting to the public factory so rendered output is
**byte-identical** and behaviour is unchanged:
- `talos.imageFactoryURL` (default `https://factory.talos.dev`) — base
URL for the worker OS disk image streamed in by CDI.
- `talos.installerRepository` (default `factory.talos.dev/installer`) —
OCI repository prefix for the in-guest `talos-reconcile` installer
image.

This closes the README's Phase-2 "not currently overridable" gap for
air-gapped / mirrored deployments.

**2. In-sandbox e2e mirror.** A best-effort `talos-image-cache`
Deployment seeded once at install time with an aggressive `curl --retry`
loop, then serving the image locally over HTTP with range support.
Tenant Kubernetes CRs point `spec.talos.imageFactoryURL` at its Service
via a shared helper that **gates on the mirror being Available and falls
back to the public factory otherwise** — so the mirror can only help,
never make CI worse. Deployed early so the seed overlaps the Cozystack
install; readiness is resolved once and cached across bats files.

## Verification

- `helm unittest` — 173/173 pass (added override + trailing-slash
cases).
- `helm template` — default render byte-identical to pre-PR; override
redirects both the disk URL and the installer image.
- Regenerated `values.schema.json`, aggregated-API Go types,
`kubernetes-rd` cozyrds schema and README via `make generate` (clean, no
key-reorder churn).
- Manifest YAML + shell syntax validated; CR heredoc renders valid YAML
in both mirror and fallback cases.

## Test plan

- [ ] CI e2e: `kubernetes-latest` / `kubernetes-previous` import from
the local mirror and join well within the node-join deadline. The mirror
half can only be fully validated in a real CI run (busybox httpd / range
serving / readiness gating); the fallback keeps it safe regardless.

```release-note
feat(apps/kubernetes): the Talos worker OS image and installer are now overridable via `talos.imageFactoryURL` and `talos.installerRepository` (both default to the public factory.talos.dev), enabling air-gapped / mirrored deployments and an in-sandbox caching mirror for the e2e suite.
```

Refs #3231 — **do not auto-close**: this addresses the
external-image-pull flake; the `MachineDeployment.spec.replicas` /
LINSTOR-pressure concern in #3231 is a separate change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added configurable Talos worker image and installer settings
(`talos.imageFactoryURL`, `talos.installerRepository`) for Kubernetes
deployments.
* Added support for using a local Talos image cache during end-to-end
runs when available.
* **Bug Fixes**
* Reduced e2e flakiness by avoiding reliance on public Talos image
downloads.
  * Normalized Talos factory URLs to handle trailing slashes correctly.
* **Documentation**
* Updated Kubernetes app docs to describe the new Talos options and
Phase 1 behavior changes.
* **Tests**
* Added/expanded e2e and template rendering tests for the new Talos
image URL behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2026-07-09 01:24:52 +03:00
Aleksei Sviridkin c32ac68ab8 test(apps/kubernetes): anchor the trailing-slash imageFactoryURL assertion 
Match the full rendered worker-disk URL shape (version + artifact + end anchor) like the sibling override assertion, so the trailing-slash normalization case cannot pass on a partial match.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 00:53:06 +03:00
Aleksei Sviridkin 336f873b07 fix(e2e): serve the Talos image cache with a range-capable HTTP server 
The talos-image-cache serve container ran `busybox httpd`, but the pinned alpine/k8s image ships no `httpd` applet and no standalone httpd, so the container exited 127 and crash-looped and the Deployment never became Available. The resolver then burned its full 12-minute rollout wait before falling back to the public factory — a fixed latency tax on the first kubernetes-* test while the flake the mirror targets stayed live.

Serve the seeded image with a small range-capable server on the image's bundled python3 instead (stock `python3 -m http.server` ignores Range headers and answers 200, not 206). It needs no extra package pull, so the mirror comes up deterministically and keeps CDI's fast range import. The seed curl now also aborts and retries a stalled transfer (--speed-time/--speed-limit), and the resolver verifies a 206 range response before pointing tenants at the mirror.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 00:52:58 +03:00
Aleksei Sviridkin f60fe2270b Merge remote-tracking branch 'origin/main' into feat/kubernetes-talos-image-factory-override 
Signed-off-by: Aleksei Sviridkin <f@lex.la>

# Conflicts:
#	packages/system/kubernetes-rd/cozyrds/kubernetes.yaml
 2026-07-09 00:46:33 +03:00
Aleksei Sviridkin 41eb6f912b fix(e2e): gate LINSTOR node-list probe on a ready controller endpoint 
The post-install-prep node-list loop dialed the linstor-controller Service
(linstor+ssl://linstor-controller:3371) as soon as the controller Deployment
reported Available once. That gate goes stale: the controller carries
reloader.stakater.com/auto=true and mounts the cert-manager-issued API-TLS
Secret, so when cert-manager (re)writes that Secret during bring-up, reloader
rolls the single-replica controller. The roll drops the Service to zero ready
endpoints after Available already passed, and a dial into that window fails
with "[Errno 113] No route to host" (EHOSTUNREACH). The blind until-loop then
churned the CLI on that error instead of waiting for a routable controller.

Gate every probe on the linstor-controller Service having a ready endpoint, so
the CLI only dials a reachable controller and a mid-loop reload re-waits for
the endpoint instead of erroring. The "Online == 3" satellite-convergence
assertion and the dedicated 300s budget are preserved; a timeout now dumps
controller endpoints and pods for diagnostics before failing.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 00:44:34 +03:00
Aleksei Sviridkin 08ac063d6d ci(e2e): move install gate to 32vCPU runner to end CPU oversubscription 
The three sandbox QEMU guests each run with `-smp 8`, demanding 24 vCPU
in total. On the 24-vCPU runner that is a 1:1 CPU commit with no host
headroom for QEMU's emulation/IO threads, vhost-net, the host NAT every
guest image pull traverses, containerd and the kernel. Under the peak
concurrency of "Install Cozystack" the guests lose cycles to CPU steal,
so in-guest readiness/liveness probes miss their deadlines and a
HelmRelease intermittently flips InProgress -> Failed with dependents
cascading to DependencyNotReady — on a different component each run, the
signature of resource thrash rather than a product bug.

Move the job to the 32-vCPU / 128 GiB runner class. Guest sizing is
unchanged (24 vCPU / 72 GiB), so the install does not slow down, while
the host keeps 8 vCPU and 56 GiB of headroom that absorbs the emulation
and network overhead the probes were starving against.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
 2026-07-09 00:39:43 +03:00