mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 14:02:56 +09:00
e160a0dc8799fcd30abaf8523ff77c66a4e01895
4898
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Myasnikov DaniilandClaude Opus 4.8
|
e160a0dc87 |
fix(backups): derive default S3 endpoint from the provisioned bucket
The default backupStorage.endpoint is http://seaweedfs-s3...svc:8333, but Cozystack ships SeaweedFS with global.seaweedfs.enableSecurity=true, so its in-cluster S3 serves TLS on :8333 behind the self-signed "SeaweedFS CA". Every cozy-default backup (etcd/mariadb/velero/fdb/cnpg) therefore hits a TLS listener over plaintext and fails the handshake. This is fatal for the etcd v1alpha2 adoption migration: its mandatory pre-upgrade safety snapshot writes to this endpoint, so migration 50 hard-fails and blocks the 1.5->1.6 upgrade on any cluster with a legacy etcd. The Etcd Strategy S3 schema has no caCert/insecureSkipVerify field, so it cannot target the self-signed in-cluster endpoint at all. Add a "backupstrategy-controller.endpoint" helper that, for a provisioned bucket, resolves the endpoint from the COSI bucket's system credentials Secret (backupStorage.systemSecretName) — the external S3 ingress with an ACME cert, the same trusted endpoint COSI advertises and every backup operator can verify — and forces the https:// scheme. All Strategy CRs, the Velero BackupStorageLocation and the controller Deployment env are routed through it. It falls back to .Values.backupStorage.endpoint for external S3 (provisionBucket=false) and for offline `helm template`/unit renders and the pre-reconcile first install, where the Secret lookup returns nothing (Flux re-renders on spec.interval once the Secret exists). helm-unittest extended to assert every consumer picks up the resolved endpoint (CNPG/Etcd/Velero keep the full URL, MariaDB/FDB strip the scheme and derive their secure flag) plus the external-S3 verbatim fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
47d239c519 |
fix(platform): address review on migration 50 (exact SAN match, Secret-gated wait, IPv6 kubeconfig)
Review feedback on #3261 (gemini-code-assist, coderabbitai, myasnikovdaniil): - Exact SAN match. _san_present read the Certificate SANs as {.spec.dnsNames} (a bracketed JSON blob that `tr ','` cannot split) and matched with a substring `grep -qF`, so a wildcard that is a substring of a longer SAN (e.g. *.etcd.<ns>.svc inside *.etcd.<ns>.svc.cluster.local) could false-positive and skip the re-issue. Read {.spec.dnsNames[*]} (space separated) and match exactly with `grep -qxF`, matching the Secret-annotation branch. - Wait on the re-issued Secret, not the patched spec. The post-patch wait loop called _san_present, which returns as soon as the Certificate spec.dnsNames contains the wildcard -- but the patch just added it there, so the loop broke on the first iteration before cert-manager re-issued the Secret, and the "re-issued" log printed unconfirmed. Add _secret_has_san (Secret alt-names annotation only, same empty-read retry + exact match) and gate the wait on it. - IPv6-safe kubeconfig server. The synthesized kubeconfig used https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT, which is an invalid URL on an IPv6-only cluster (a bare IPv6 host must be bracketed). Use https://kubernetes.default.svc -- IP-family-agnostic, validated by the mounted SA CA, and the same server URL kubectl synthesizes in-cluster. Tests (the review blocker): extend the bats suite (and its fake kubectl) with two cases pinning these contracts -- a superstring dnsName must NOT skip the re-issue patch, and etcd-migrate must be invoked with a synthesized in-cluster kubeconfig whose server is kubernetes.default.svc (not the bare IPv6 host), authenticating via the SA token file + CA. Both the kubeconfig path and the SA directory are now env-overridable (ETCD_MIGRATE_KUBECONFIG, ETCD_ADOPT_SA_DIR) so the synthesis is exercisable off-cluster; production defaults are unchanged. Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
c2539ca935 |
fix(platform): make etcd v1alpha2 adoption (migration 50) robust in-cluster
Migration 50 (etcd.aenix.io -> etcd-operator.cozystack.io/v1alpha2 adoption) had two defects that blocked every in-cluster 1.5 -> 1.6 upgrade on a cluster with an existing etcd: 1. Cert-SAN wait treated transient kubectl failures as "SAN absent". ensure_wildcard_sans checked/awaited the wildcard SAN with `kubectl get ... 2>/dev/null | grep`, so any transient GET failure (API discovery refresh, apiserver blip, throttling) produced an empty string indistinguishable from a genuine absence -> false miss, and the 120s wait never recovered. Replace the two ad-hoc checks with a _san_present helper that retries on an empty read (a real Certificate/Secret never has empty dnsNames/alt-names) and accepts the native wildcard from EITHER the issued Secret's cert-manager.io/alt-names annotation OR the Certificate spec.dnsNames (the source of truth for what cert-manager will issue). 2. etcd-migrate had no kubeconfig in-cluster. etcd-migrate only reads a kubeconfig file (-k/--kubeconfig, default /root/.kube/config) and, unlike kubectl, does not fall back to the mounted in-cluster ServiceAccount. The hook Job set no KUBECONFIG and passed no --kubeconfig, so both the dry-run and --apply aborted with "error building kubeconfig: stat /root/.kube/config: no such file". Synthesize an in-cluster kubeconfig from the mounted ServiceAccount and pass --kubeconfig to both etcd-migrate invocations. Verified end-to-end on a 1.5.2 -> 1.6.0-rc.1 upgrade: the adoption now completes in-place (pods never restarted, data intact) and the cluster reaches readyMembers=3 / Available=True. Refs: #3243, #3255 Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
ca4f99411f |
fix(etcd): harden v1alpha2 transition — hook runAsUser, digest-pin kubectl, tests
Addresses review on #3265: - pre-upgrade-selector-fix hook: add runAsUser: 65532. clastix/kubectl's image user is the non-numeric name `nonroot`, which the kubelet cannot verify against runAsNonRoot: true — so the Pod fails admission and the hook never runs, silently blocking the very 1.5->1.6 upgrade it exists to unblock. Matches postgres-operator's webhook-ready hook (same image). - Digest-pin the kubectl image (the comment already claimed digest-pinning but shipped a floating v1.32 tag). Reuse the digest postgres-operator already vendors, add the renovate annotation, and template repo:tag@digest. - Tests (both charts have CI helm-unittest suites, none previously covered these): - etcd-operator/tests/selector-fix-hook_test.yaml: hook wiring, namespaced least-privilege RBAC, numeric-non-root securityContext (guards the runAsUser regression above), digest-pinned image. - etcd-operator/tests/deployment_test.yaml: assert the 256Mi cold-start memory floor so it can't silently drop back to an OOMKilling value. - extra/etcd/tests/etcd-cluster_test.yaml: assert the transitional etcd-headless Service (headless, publishNotReadyAddresses, member selector, client/peer ports). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrey Kolkov
|
9daaa9ff5a |
fix(etcd): fix etcd-operator 1.5->1.6 upgrade (immutable selector + OOM floor)
Two defects that break the etcd-operator on an in-cluster 1.5 -> 1.6 upgrade (#2859 swapped the upstream chart for the cozystack-authored one): - #3242: the controller Deployment's spec.selector.matchLabels changed ({instance,name} -> {name,control-plane}). spec.selector is immutable, so `helm upgrade` cannot patch the existing Deployment and the whole HelmRelease upgrade fails ("field is immutable") -- the operator and the v1alpha2 CRDs it serves never come up, which blocks the etcd v1alpha2 adoption. Fresh installs use the new selector directly, so fresh-install CI does not catch it. Add a pre-upgrade hook that deletes the Deployment ONLY when its live selector is the pre-1.6 one (lacks control-plane=controller-manager), so Helm recreates it cleanly. No-op when the selector already matches (rc.1 -> later) and never runs on a fresh install (pre-upgrade only). Keeping the selector stable in the chart is not an option: rc.1 already shipped the new selector, so aligning it back would merely move the immutable break to rc.1 -> next. - Raise the manager's cold-start memory limit floor 128Mi -> 256Mi (and the VPA minAllowed to match). The steady-state working set is ~250Mi (the VPA's own recommendation); at 128Mi a Pod that starts before the VPA admission webhook rewrites it (e.g. a Deployment recreated out of band, or the webhook briefly unavailable during upgrade) OOMKills into a crash loop. Defense in depth so the operator never depends on VPA timing merely to avoid crashing; the VPA still scales it further under load up to maxAllowed. Verified live on a 1.5 -> 1.6 adoption: the delete-Deployment step plus the VPA re-applying 256Mi+ let the operator come up and the adoption complete. Refs: #3242, #3243 Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey Kolkov
|
d51d6914d7 |
fix(etcd): keep legacy etcd-headless Service alive during v1alpha2 adoption
etcd-migrate adopts legacy clusters IN PLACE, so the Pods keep their original spec.subdomain: etcd-headless and are dialed at etcd-<i>.etcd-headless.<ns>.svc until they eventually roll onto the operator's native <member>.etcd.<ns>.svc domain. The v1alpha2 operator only creates the native `etcd` Service, and the legacy `etcd-headless` Service is pruned during the transition, so those names stop resolving (no such host), the operator's MemberList fails, and status.readyMembers never populates -- the EtcdCluster never goes Ready even though the etcd processes are healthy and in quorum. Ship a chart-managed transitional headless `etcd-headless` Service (selector mirrors the operator's native `etcd` Service via etcd-operator.cozystack.io/cluster, publishNotReadyAddresses: true). This is the DNS counterpart of the legacy *.etcd-headless.<ns>.svc wildcard already kept in the server/peer cert SANs for the same transition window -- the TLS half of the compat was done, the DNS half was missing. Safe to remove, together with that SAN, once the members have rolled onto the native `etcd` subdomain. Verified live: recreating this Service on an adopted 3-node cluster restored per-pod DNS and the cluster went readyMembers=3 / Available=True. Refs: #3243 Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
myasnikovdaniil
|
215bd430a9 |
test(e2e): diff-driven install selector select-install.sh (#3279)
## What this PR does Adds `hack/select-install.sh` — the install-side companion to `hack/select-e2e.sh`. Where `select-e2e.sh` picks *which* Chainsaw suites run for a diff (a reverse dependency walk), `select-install.sh` picks *what must be installed* for those suites: the forward `dependsOn` closure over the `PackageSource` graph. Given the suite names `select-e2e.sh` emits, it maps each back to its `*-application` `PackageSource` and walks `spec.variants[].dependsOn` forward to the transitive closure — the minimal set of packages to enable instead of provisioning the full `isp-full` platform. A `postgres` chart change, for example, resolves to 9 of 97 sources. Design notes: - Reuses the real `PackageSource` graph in `packages/core/platform/sources/` — no parallel dependency-declaration files to keep in sync. - Unlike `select-e2e.sh`'s reverse walk, the forward walk **keeps** the `cozystack.cozystack-engine` edge: for test *selection* it is noise (it would fan every app out to every other), but for *install* it is a genuine prerequisite (the app's `*-rd` HelmRelease needs the engine to register the ApplicationDefinition CRD). Because that edge lives only on `*-application` sources, a suite backed by a **system** package (e.g. `securitygroup`, whose `SecurityGroup` in `sdn.cozystack.io` is served by the aggregated `cozystack-api` — a `cozystack-engine` component) can't reach the engine via the walk, so the engine is **seeded explicitly for any non-empty closure** (and a graph missing it is a hard error). - `--validate` mode checks the whole graph — every `dependsOn` target resolves to a real `PackageSource` (reachability) and there are no dependency cycles — **and** the suite mapping: every `hack/e2e-chainsaw/*/` suite resolves to a real `PackageSource` or to `-` (a missing suites dir is itself a failure). The graph lives next to the packages and can't drift from the suite dirs; the hand-maintained mapping is the only part that can, so this is the cheap CI guard. - Suite→source mapping follows the `<suite>-application` convention with a bare-`<suite>` fallback, plus explicit cases for the irregular names (`kubernetes`/`vminstance`/`securitygroup`) and for core-platform features that have no separately-enabled package (`serviceexposure`, whose `ExposureClass`/`ServiceExposure` CRDs live in `packages/core/platform`). A suite may reference other app kinds by selector without standing them up (e.g. `securitygroup`'s `SecurityGroup` names `Postgres`/`Kubernetes` as selectors only) — only the primary package is mapped here; the forward closure and the seeded engine cover its prerequisites, and full multi-package modelling is left to the `test-minimal` follow-up. A suite with no known mapping is a **hard error** (fail closed), not a silent skip. Includes a 17-case unit suite (`hack/select-install_test.bats`) covering single-app closure, transitivity, multi-suite union, stdin input, the kubernetes/irregular/core-platform suite mappings, the `securitygroup` engine-seed regression, empty input, fail-closed unmapped suites (all reported, no partial output), the missing-engine hard error, and the `--validate` failure modes (dangling dep, cycle, unmapped suite dir, missing suites dir). It is auto-discovered by `make unit-tests` (`BATS_UNIT_FILES` globs `hack/*.bats`). Verified end-to-end against real git diffs across six path types (app chart, shared operator, docs-only, per-suite Chainsaw edit, library, shared e2e helper): a `postgres` chart change scopes the install to 9 of 97 sources; the full-suite fallback to 39 of 97. This is Track A of #2847 (diff-driven & scenario-based E2E); tracked in #3278. Consuming the output — a `test-minimal` platform variant/bundle plus wiring into the `pull-requests.yaml` install step — is a deliberate follow-up, since it depends on a platform-variant design decision. ### Screenshots Not applicable — CI/tooling-only change. ### Release note ```release-note test(e2e): add hack/select-install.sh to compute the minimal package install set for a diff-scoped E2E run (forward PackageSource dependency closure), with a --validate graph-consistency mode ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a tool to select the minimal package set required for one or more end-to-end test suites. * Supports combining multiple suites, resolving transitive dependencies, and handling unmapped or empty suite selections. * Added validation for missing dependencies and circular dependency definitions. * **Tests** * Added coverage for package selection, dependency resolution, suite mappings, empty selections, and validation failures. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
29e5a55473 |
fix(kubernetes): re-pin deleted centos:stream9 digest for kubevirt-csi-driver (#3303)
## What this PR does `packages/apps/kubernetes/images/kubevirt-csi-driver/Dockerfile` pins its runtime base image to `quay.io/centos/centos:stream9` by digest. `stream9` is a rolling tag, and upstream deleted the pinned digest (`sha256:0c9db821…`) when it published a fresh stream9 build. As a result `make image-kubevirt-csi-driver` now fails with `not found` on every build — open PRs and `main` alike (the `Build packages/apps/kubernetes` job goes red before E2E even starts). Re-pin to the current live `stream9` digest (`sha256:a93e3316…`, verified pullable and multi-arch: amd64/arm64/ppc64le/s390x). No Dockerfile logic changes. ## Known limitation and follow-up This is an emergency CI unblock, **not** a durable fix. `stream9` is a rolling, garbage-collected tag: upstream deletes old digests on every rebuild, so the same `not found` breakage will recur the next time CentOS rolls the tag faster than the pin is bumped. This is the same operation Renovate's weekly (non-automerged) digest PR performs. A durable fix is out of scope for this hotfix and left as follow-up: - mirror the base image into cozystack's own registry so old digests are not GC'd, or - move to an immutable-digest base such as UBI9 (`registry.access.redhat.com/ubi9`), or - enable automerge / tighten the schedule for the digest-only Renovate group so future rolls are absorbed automatically. ### Downstream repositories - [x] No downstream repository is affected by this change <!-- Runtime base-image digest of an internal component image; no API, values, CRD, or chart-interface surface changes. --> ### Release note ```release-note fix(kubernetes): restore the kubevirt-csi-driver image build after upstream deleted the pinned centos:stream9 base image digest ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated the Kubernetes CSI driver’s runtime base image to a newer verified version. * Runtime behavior and application functionality remain unchanged. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ivan Okhotnikov
|
ce4b263ca6 |
fix(kubernetes): re-pin deleted centos:stream9 digest for kubevirt-csi-driver
quay.io/centos/centos:stream9 is a rolling tag; upstream deleted the pinned digest (sha256:0c9db821...) when it published a new stream9 build, so every build of image-kubevirt-csi-driver fails with "not found" — on open PRs and on main. Re-pin to the current live stream9 digest (sha256:a93e3316...). This is an emergency CI unblock, not a durable fix: stream9 is garbage-collected, so upstream will delete this digest too on its next rebuild and the same "not found" breakage recurs until the pin is bumped again — the same operation Renovate's weekly, non-automerged digest PR performs. A durable fix (mirror the base image into cozystack's own registry, or move to an immutable-digest base such as UBI9) is out of scope for this hotfix and left as follow-up. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Myasnikov Daniil
|
1f084cecc8 |
test(e2e): harden select-install.sh per review
Address the review blockers on the diff-driven install selector: - B1: seed cozystack.cozystack-engine for any non-empty closure. Suites backed by a system package (e.g. securitygroup, which applies a sdn.cozystack.io resource served by the aggregated cozystack-api) carry no dependsOn edge to the engine, so the forward walk could never reach it and the emitted set could not run the suite. Hard-error if the engine source is absent from the graph. - B2: an unmapped suite is now a hard error in closure mode (every bad suite reported in one run, no partial output) instead of failing open. --validate additionally asserts every hack/e2e-chainsaw suite resolves to a real PackageSource or "-", and fails when the suites dir is missing. Also correct the inaccurate "securitygroup also stands up a Kubernetes app" comment (its fixtures reference Postgres/Kubernetes as selectors only), note the variant-union over-selection in build_forward_deps, drop the unnecessary source copies from the tests, and add coverage for stdin input, the engine seed, fail-closed mapping, the --validate suite guard, and the missing-suites-dir / missing-engine failure modes. Assisted-By: Claude <noreply@anthropic.com> Assisted-By: GPT-5 <noreply@openai.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
77389d6e99 |
chore(deps): update github.com/kubevirt/csi-driver digest to 27b52aa (#3066)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/kubevirt/csi-driver](https://redirect.github.com/kubevirt/csi-driver) | replace | digest | `51c6bca` → `27b52aa` | --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cozystack/cozystack). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI1OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==--> |
||
|
myasnikovdaniil
|
a269eb3965 |
chore(deps): update actions/checkout action to v6.0.3 (#3133)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [actions/checkout](https://redirect.github.com/actions/checkout) | action | patch | `v6.0.2` → `v6.0.3` | --- ### Release Notes <details> <summary>actions/checkout (actions/checkout)</summary> ### [`v6.0.3`](https://redirect.github.com/actions/checkout/blob/HEAD/CHANGELOG.md#v603) [Compare Source](https://redirect.github.com/actions/checkout/compare/v6.0.2...v6.0.3) - Fix checkout init for SHA-256 repositories by [@​yaananth](https://redirect.github.com/yaananth) in [#​2439](https://redirect.github.com/actions/checkout/pull/2439) - fix: expand merge commit SHA regex and add SHA-256 test cases by [@​yaananth](https://redirect.github.com/yaananth) in [#​2414](https://redirect.github.com/actions/checkout/pull/2414) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cozystack/cozystack). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI0Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==--> |
||
|
myasnikovdaniil
|
1d1c7c2c38 |
chore(deps): update altinity/clickhouse-backup docker tag to v2.7.4 (#3226)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [altinity/clickhouse-backup](https://redirect.github.com/Altinity/clickhouse-backup) | final | patch | `2.7.2` → `2.7.4` | --- ### Release Notes <details> <summary>Altinity/clickhouse-backup (altinity/clickhouse-backup)</summary> ### [`v2.7.4`](https://redirect.github.com/Altinity/clickhouse-backup/blob/HEAD/ChangeLog.md#v274) [Compare Source](https://redirect.github.com/Altinity/clickhouse-backup/compare/v2.7.3...v2.7.4) NEW FEATURES - add `clickhouse.parts_columns_max_bytes_before_external_group_by` (env `CLICKHOUSE_PARTS_COLUMNS_MAX_BYTES_BEFORE_EXTERNAL_GROUP_BY`, default `100000000`) and `clickhouse.parts_columns_max_memory_usage` (env `CLICKHOUSE_PARTS_COLUMNS_MAX_MEMORY_USAGE`, default `200000000`) — make the memory caps injected into the `system.parts_columns` check query configurable instead of hardcoded, `0` means don't inject the corresponding setting (pre-2.7.2 behavior), fix [#​1420](https://redirect.github.com/Altinity/clickhouse-backup/issues/1420) BUG FIXES - fix `--restore-database-mapping`/`--restore-table-mapping` losing unrelated comma-separated `--tables` pattern items, or applying the mapping to the wrong item when the same source database/table name was repeated across items — each comma-separated pattern item is now matched and substituted independently, fix [#​1421](https://redirect.github.com/Altinity/clickhouse-backup/issues/1421) ### [`v2.7.3`](https://redirect.github.com/Altinity/clickhouse-backup/blob/HEAD/ChangeLog.md#v273) [Compare Source](https://redirect.github.com/Altinity/clickhouse-backup/compare/v2.7.2...v2.7.3) NEW FEATURES - add `general.max_broken_part_ratio` (env `MAX_BROKEN_PART_RATIO`, default `0`) — when >0, `create`/`create_remote` skips broken data parts instead of aborting the whole backup as long as the broken/total part ratio stays at or below the configured fraction; skipped parts are logged as warnings and the backup is marked successful but partial; `0` preserves the legacy any-broken-part-fails behavior, fix [#​1418](https://redirect.github.com/Altinity/clickhouse-backup/issues/1418) - add `clickhouse.rebind_replica_path_if_exists` (env `CLICKHOUSE_REBIND_REPLICA_PATH_IF_EXISTS`, CLI `--rebind-replica-path-if-exists`, also accepted by the REST API `POST /backup/restore` endpoint) — opt-in rebind of the replicated table ZK path to `default_replica_path` during `restore` when the resolved path already has leftover children but no local table uses it (async-stale state after DROP); MUST stay `false` during a concurrent multi-replica restore to avoid split-brain, fix [#​849](https://redirect.github.com/Altinity/clickhouse-backup/issues/849) - add `gcs.disable_http2` (env `GCS_DISABLE_HTTP2`) — force HTTP/1.1 for the GCS client to avoid single-TCP-connection HTTP/2 multiplexing becoming the bottleneck for high-throughput parallel downloads, fix [#​1437](https://redirect.github.com/Altinity/clickhouse-backup/pull/1437) IMPROVEMENTS - switch the S3 client retry mode from `aws.RetryModeStandard` to `aws.RetryModeAdaptive`, so upload/download retries back off with a client-side rate limiter under throttling (`SlowDown`/503) instead of a fixed token bucket - support tables with `ENGINE=Alias` during `restore` — strip the explicit column list (Alias inherits columns from the target table) and add `allow_experimental_alias_table_engine=1` for ClickHouse 25.11+, fix [#​1426](https://redirect.github.com/Altinity/clickhouse-backup/issues/1426) - skip tables from databases with engine `DataLakeCatalog` (read-only external catalogs) during backup, in addition to `MySQL`/`PostgreSQL`/`MaterializedPostgreSQL`, fix [#​1417](https://redirect.github.com/Altinity/clickhouse-backup/issues/1417) - speed up `create` on servers with many tables: batch the in-progress `system.mutations` lookup instead of issuing one query per table (O(N^2) -> O(N)), fix [#​1431](https://redirect.github.com/Altinity/clickhouse-backup/pull/1431) - speed up `restore`/`download` table list resolution: eliminate the O(N^2) linear duplicate scan in `getTableListByPatternLocal`/`getTableListByPatternRemote` via a shared O(1) dedup keyed on `metadata.TableTitle`, fix [#​1430](https://redirect.github.com/Altinity/clickhouse-backup/pull/1430) - extend TestFlows FIPS 140-3 test coverage: dedicated test steps, stronger `--fips-info`/version-output and TLS-handshake assertions, updated requirements documentation, fix [#​1443](https://redirect.github.com/Altinity/clickhouse-backup/pull/1443) BUG FIXES - restore the default `general.compression_use_multi_thread` to `true`: [#​1378](https://redirect.github.com/Altinity/clickhouse-backup/issues/1378) defaulted it to `false`, which silently dropped gzip/zstd compression from multi-threaded (pre-1378 gzip always used `pgzip`) to single-threaded and caused \~30% slower upload throughput for backups dominated by one large table (where `upload_concurrency` provides no per-stream parallelism); the option is now silently ignored instead of failing config validation for formats other than gzip/zstd - fix v2.7.1 regression where `restore --schema` on a second replica rebound the ZK replica path even while the first replica was active and healthy — a foreign/renamed local table occupying the resolved ZK path is still detected automatically via `system.replicas` and rebound to `default_replica_path`, but a non-empty ZK path without a local table now defaults to the HA-safe join and requires the new opt-in `rebind_replica_path_if_exists`, fix [#​1428](https://redirect.github.com/Altinity/clickhouse-backup/issues/1428) - fix inconsistent `--rbac` behavior: `restore --rbac` no longer returns an error when the backup contains no RBAC `access` directory, matching the `download --rbac` behavior, fix [#​1432](https://redirect.github.com/Altinity/clickhouse-backup/issues/1432) - inject `ON CLUSTER` for `LIVE VIEW` and `WINDOW VIEW` restored with explicit column lists when `restore_schema_on_cluster` is set — `enrichQueryWithOnCluster` missed these statements and the views were created only on the local replica, fix [#​1433](https://redirect.github.com/Altinity/clickhouse-backup/pull/1433) - mask sensitive values (passwords, storage credentials, encryption keys) when logging `--env` config overrides, and in the API server action logs and `/backup/status` output, fix [#​1441](https://redirect.github.com/Altinity/clickhouse-backup/pull/1441) </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cozystack/cozystack). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI0Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==--> |
||
|
Aleksei Sviridkin
|
249f7b5a5e |
feat(platform, ingress): propagate operator wildcard certificate to per-tenant termination points (#2990)
## What this PR does The operator-provided wildcard certificate (`publishing.certificates.wildcardSecretName`) currently serves the root tenant only: everything is same-namespace, so the root ingress controller and Gateway read the operator's TLS Secret directly. Child tenants run their own ingress controller / Gateway in their own namespace under a default-deny policy and cannot read that Secret across namespaces, so they fall back to per-host ACME. This propagates the operator wildcard to per-tenant termination points by replicating the Secret into each tenant namespace that owns one, keeping the root-MVP invariant intact: the wildcard is deliberately distributed to tenant namespaces, but no tenant RBAC is widened — every consumer reads only its own-namespace copy. A new controller in `cozystack-controller` does the replication with no extra operator input. It reads the same platform values channel the consumers read (`cozy-system/cozystack-values`): the wildcard Secret name from `_cluster.wildcard-secret-name` and the publishing namespace from `_cluster.expose-ingress`. It mirrors that Secret into every tenant namespace whose `namespace.cozystack.io/ingress` or `namespace.cozystack.io/gateway` label equals its own name — i.e. the namespace runs its own controller / Gateway. Because the source is derived from the same value that makes the consumers reference it, the replica is created whenever the consumers expect it — no manual labelling and no upgrade hazard where a child controller references a Secret that nothing creates. Replicas are refreshed on rotation and pruned when a namespace stops terminating TLS. Teardown of every replica happens on exactly one trigger: clearing `wildcardSecretName` (explicit disable). A source that is merely absent or mistyped, or a transiently-missing values channel, leaves existing replicas in place, so a brief gap, a delete+recreate rotation, or a misconfigured publishing namespace never drops tenant TLS. The platform controller's own ServiceAccount gains Secret writes; tenant RBAC is untouched. The controller does not cache every Secret in the cluster: the manager's Secret informer is scoped to managed replicas and the values channel only, and the operator source / foreign collisions are read through the uncached APIReader. The source is not watched (its name is dynamic), so an in-place rotation is picked up by a bounded periodic resync while the publishing tenant serves the source with no lag. On the consumer side, each tenant ingress controller now sets `--default-ssl-certificate` to the wildcard Secret in its own namespace (`.Release.Namespace/<name>`), so a child tenant serves the replica instead of minting per-host ACME. The Gateway path already renders `certMode: existingSecret` with a same-namespace `wildcardSecretRef` for every tenant, so it consumes the replica with no chart change. ### Why not clustersecret-operator The issue proposed the shipped `clustersecret-operator`. It materializes inline template data into selected namespaces; it cannot mirror an existing source Secret, and it is not installed in any default bundle. Driving it would mean embedding the certificate and private key as literal data in a cluster-scoped object rendered from a Helm `lookup` — that broadcasts key material the design keeps off any shared channel, and a `lookup` is invisible to the Flux helm-controller digest so it would not re-render on rotation. Its namespace selector also cannot express "the namespace whose owner label equals its own name", so it would over-replicate the key into every tenant namespace. A small dedicated reconciler targets exactly the termination namespaces and keeps the key off the values channel. ### Coverage caveat Replication delivers the certificate bytes, not SAN coverage: a single `*.<root-host>` does not match `*.<child-apex>`. Operators serving child-tenant subdomains supply a certificate whose SANs cover those apexes; the docs spell this out. The replica carries the wildcard private key into each terminating tenant namespace — the same exposure as the per-host ACME Secret it replaces, now shared across tenants. ### Tests - Go fake-client tests for the reconciler: termination-only replication (ingress-owning and gateway-owning namespaces only, never inheriting ones, never over the source), copy markers, rotation, prune on namespace demotion, the non-destructive source-deletion contract, replica self-heal after out-of-band deletion, the publishing-namespace self-replication guard, disable-via-config teardown, source rename, foreign-Secret collision safety (no overwrite, no requeue, does not block siblings), transient-error requeue, non-TLS source ignored, absent-channel keeps replicas, publishing-namespace mismatch keeps and requeues, the cache-scope pin, the enqueue-wiring map, an `ownsTerminationPoint` table, and the active-only requeue contract. - helm-unittest for the ingress `default-ssl-certificate` wiring (including a child-tenant controller pointing at its own-namespace replica) and for the cozystack-controller ClusterRole's new Secret-write rule. ### Screenshots N/A — no UI changes. ### Release note ```release-note feat(platform): the operator-provided wildcard TLS certificate (`publishing.certificates.wildcardSecretName`) is now served by per-tenant ingress controllers and Gateways automatically — the platform controller replicates it into each tenant namespace that terminates TLS, with no extra operator input and no cross-namespace Secret read ``` Closes #2820 Part of #2811 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **New Features** * Wildcard TLS Secret now replicates from the platform to tenant namespaces that terminate TLS via their own ingress controller or Gateway, including child-tenant scenarios. * **Documentation** * Clarified configuration, timing, and SAN matching requirements for wildcard replication. * Updated gateway and ingress-nginx default certificate documentation for replica mirroring behavior. * **Bug Fixes** * Improved handling for missing/invalid source secrets (no unnecessary pruning) and safe collision behavior. * **Tests** * Added comprehensive reconciliation/replication test coverage. * Added Helm unittest target for the controller chart. * **Chores** * Updated controller RBAC to support wildcard TLS Secret replication. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
6b1ad96093 |
fix(e2e): gate LINSTOR node-list probe on a ready controller endpoint (#3250)
## What this PR does The E2E `Install Cozystack` step runs `hack/e2e-post-install-prep.sh` in the background, which waits for 3 LINSTOR satellites to come Online by dialing the `linstor-controller` Service (`linstor+ssl://linstor-controller:3371`) from the controller pod. That probe was gated only by a one-shot `kubectl wait deployment/linstor-controller --for=condition=available`, which goes stale. The controller carries `reloader.stakater.com/auto=true` and mounts the cert-manager-issued API-TLS Secret (`packages/system/linstor/templates/cluster.yaml`), so when cert-manager (re)writes that Secret during bring-up, reloader rolls the single-replica controller. The roll drops the Service to zero ready endpoints *after* `Available` already passed, and a dial into that window fails with `[Errno 113] No route to host` (EHOSTUNREACH — the state of a ClusterIP whose Service has no ready backend). The blind `until` loop then churned the CLI on that error, tolerating a real un-Ready-controller race instead of gating on it. This gates every probe on the `linstor-controller` Service actually having a ready endpoint (a non-empty `Endpoints .subsets[].addresses`), so the CLI only dials a routable controller and a mid-loop reload re-waits for the endpoint instead of erroring. The `Online == 3` satellite-convergence check stays the real readiness assertion, and the dedicated 300s budget is preserved from the previous `timeout 300`; on timeout the loop now dumps the controller endpoints and pods for diagnostics before failing, per the E2E diagnostics convention. Evidence for the mechanism: the same single `Error: Unable to connect to linstor+ssl://linstor-controller:3371: [Errno 113] No route to host` line appears exactly once per install across multiple runs and is immediately recovered by the retry — consistent with one reloader-driven controller roll during initial bring-up, not a persistent outage. Scoped to the LINSTOR readiness path in `hack/e2e-post-install-prep.sh` only; no change to install ordering, dependencies, or timeouts. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved the LINSTOR startup readiness check to wait more reliably for the service to become reachable before verifying node status. * Made the node-online verification more resilient by handling transient connectivity errors and enforcing a clearer timeout. * Added extra diagnostic output when readiness does not complete in time, helping with troubleshooting failed installs. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
3763c528da |
fix(seaweedfs): soften master anti-affinity for small clusters (#3197)
## What this PR does Overrides the seaweedfs master anti-affinity in the package values from a hard `requiredDuringSchedulingIgnoredDuringExecution` hostname term to a strong preference (`preferredDuringSchedulingIgnoredDuringExecution`, weight 100). The selector and topologyKey are unchanged. The vendored subchart pins the master to a hard hostname anti-affinity — its own comment notes that emptying it is what allows single-node deployment. With the master StatefulSet at three replicas, the surplus master Pods stay Pending forever on clusters with fewer than three nodes, so `helm install --wait` never returns and the S3 stack never comes up. Softening to a strong preference keeps the masters spread across distinct nodes whenever the cluster has them (the scheduler only co-locates when there is no alternative — exactly the small-cluster case), while letting single- and dual-node clusters bring the master up. This mirrors the posture already applied to the s3 component in the same values file. A helm-unittest pins the softened term (asserts the hard term is gone and the preferred term carries the right weight, topologyKey, and component selector) so a future subchart bump cannot silently re-harden it. ### Screenshots N/A — no UI changes. ### Release note ```release-note fix(seaweedfs): soften the master anti-affinity to a preference so the S3 stack installs on clusters with fewer than three nodes ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * SeaweedFS master pods now use a softer scheduling preference for hostname separation, improving deployment success on smaller clusters. * **Tests** * Added a Helm chart validation to confirm the anti-affinity behavior is now expressed as a weighted preference (not a hard requirement), including the expected selector and topology key. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
6c9edf5038 |
fix(platform): route OIDC backend calls through the in-cluster Keycloak by default (#3289)
## What this PR does Defaults `authentication.oidc.keycloakInternalUrl` to `http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy` instead of an empty string. With the empty default, oauth2-proxy performed OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once `tenant-root` is ready, and `tenant-root` is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up — with no obvious diagnosis for a first-time installer. The bundled Keycloak is the only supported provider (the external hostname is templated as `keycloak.<host>` and the realm is fixed), so backend-to-backend calls have no reason to leave the cluster. The new default is the same Service address `keycloak-configure` already uses for its own backend calls. The dashboard and linstor-gui gatekeepers then run with `--skip-oidc-discovery` and send token, jwks, userinfo and logout requests straight to the Service, while browser-facing redirects stay on the external host. Both gatekeeper templates already branch on the value being non-empty, so neither needed a change. The value stays overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to `""` restores the previous external-discovery behaviour. Token validation is unaffected: Keycloak pins `KC_HOSTNAME` to the external URL, so tokens redeemed over the internal Service still carry the external `iss` and match the unchanged `--oidc-issuer-url`. ### Upgrade behaviour Existing OIDC clusters that never set `keycloakInternalUrl` pick up the new default on upgrade: their gatekeepers roll once and move to the internal path. Clusters that set the value explicitly — including to `""` — keep exactly what they configured. ### Security posture The backend leg (token redemption, which carries the gatekeeper's Keycloak client secret, plus jwks/userinfo/logout) now travels as plaintext HTTP between pods rather than HTTPS to the ingress. This is stated in the `values.yaml` comment rather than left implicit. The `keycloak-http` Service has no TLS listener, so in-cluster HTTPS is not an available alternative, and `keycloak-configure` already sends Keycloak *admin* credentials over that identical address — anyone positioned to read this traffic can already reach a strictly more powerful secret. The externally reachable front door (browser to oauth2-proxy) is unchanged. ### Testing helm-unittest coverage across all three affected charts: the default landing on the `_cluster` channel, both gatekeepers rendering the in-cluster backend URLs while keeping browser URLs external, an explicit override winning, the empty-string escape hatch still rendering the external-discovery shape, and OIDC-disabled clusters (now the common case of "OIDC off, internal URL set") ignoring the value entirely. The E2E OIDC case previously applied this exact URL as a manual workaround. It now sets only `oidc.enabled` and asserts the consequence: it waits for the `auth-proxy` container to appear in the gatekeeper Deployment, then gates on `kubectl rollout status`. Readiness alone would prove nothing there — the HelmRelease is still `Ready` from the token-proxy install, and with `replicas: 1` and `maxUnavailable: 25%` (which rounds down to 0) the Deployment keeps the old ReplicaSet `Available` while a new pod crashloops. Only the rollout catches a regression back to external discovery. Follow-up: the OIDC documentation in the website repo still frames `keycloakInternalUrl` as an optional tweak "for self-signed certificates or restricted external access". It is now the default path and needs a separate docs PR. Closes #3288 ### Screenshots No UI changes. ### Release note ```release-note fix(platform): `authentication.oidc.keycloakInternalUrl` now defaults to the in-cluster Keycloak Service (`http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy`). OIDC backend calls from the dashboard and linstor-gui gatekeepers no longer leave the cluster, so enabling OIDC on a fresh install no longer leaves the dashboard down while the root ingress is not yet serving. Existing clusters that never set the value pick up the new default and roll their gatekeepers once. Set the value explicitly to point at a non-standard Keycloak placement, or to an empty string to restore external OIDC discovery. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - OIDC-enabled applications now use the internal Keycloak service for backend authentication by default, improving reliability and avoiding unnecessary external discovery. - Browser redirects and login flows continue using the external Keycloak address. - Operators can provide a custom internal Keycloak URL or set it to an empty value to restore external OIDC discovery. - **Bug Fixes** - Dashboard and Linstor GUI authentication now correctly apply internal Keycloak settings only when OIDC is enabled. - Dashboard upgrades now wait for authentication components to become ready before completing. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
5826d5fd7d |
docs(harbor): fix SeaweedFS discovery key described as annotation (#3293)
## What this PR does Corrects a factual error in the Harbor app's prerequisites documentation. The prerequisites section described the resolved SeaweedFS class as being surfaced on the tenant namespace as the `namespace.cozystack.io/seaweedfs` **annotation**. It is a **label**. `packages/apps/tenant/templates/namespace.yaml` renders that key under `metadata.labels` — in the block commented "Labels for network policies", alongside the sibling `namespace.cozystack.io/etcd`, `/ingress`, `/gateway`, `/monitoring` and `/host` keys. No annotation of that name exists anywhere in the tree. Why the wrong noun matters: it sends an operator to the wrong field at exactly the moment they are already debugging. The README documents that a Harbor release which cannot provision its registry bucket stays unreconciled waiting on `BucketInfo`, and the natural check is to confirm the tenant chain actually resolved a SeaweedFS class. Following the text and inspecting `.metadata.annotations` returns nothing even on a perfectly healthy tenant, which invites the conclusion that the tenant chain is misconfigured and sends the investigation away from the real cause. The change is one word. I checked the rest of the tree for the same error class — grepping for `namespace.cozystack.io/` together with "annotation" across all of `packages/` and then across the whole repository — and this README was the only occurrence. `packages/extra/gateway/README.md` already describes the sibling keys correctly as labels. Note on the file being partially generated: `packages/apps/harbor/README.md` has a `generate:` target in its Makefile that runs `cozyvalues-gen ... -r README.md`, but that generator only rewrites the `## Parameters` tables from the `@param` comments in `values.yaml`. The corrected sentence lives in the hand-maintained prose section, which the generator preserves verbatim — confirmed by running cozyvalues-gen v1.6.0 (the version CI pins) against a copy of the package: the corrected wording survives regeneration byte-for-byte, so this introduces no codegen drift. Editing the README directly is the correct fix here; there is no separate upstream source to change. Downstream consequence worth flagging for reviewers: the cozystack website generates the Harbor app page from this README, so the published v1.5 Harbor page currently carries the same error. It will be corrected when the site re-syncs from this file. ### Screenshots Not applicable — documentation-only change, no UI surface. ### Release note ```release-note docs(harbor): fix SeaweedFS discovery key in prerequisites — it is the `namespace.cozystack.io/seaweedfs` namespace label, previously documented as an annotation ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated Harbor prerequisites to clarify that the resolved SeaweedFS class is exposed through a namespace label. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
ce1b74da46 |
docs(agents): add downstream repository checklist to the PR template (#3298)
## What this PR does Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here can break them in silence. It has already happened: `packages/apps/opensearch` ships but has no reference page on the website, the Terraform provider still offers a Kubernetes version this repo removed, and the plugin repo still sends operators to a `hack/` script that was deleted. This adds a **Downstream Repositories** trigger map to `docs/agents/contributing.md` — per repository, which change here forces a change over there and which file to touch — and a checklist in the PR template that links to it. The template addresses AI agents directly, since they author a large share of PRs, and tells them to open the follow-up rather than tick a box that claims work nobody did. A unit test (`hack/downstream-trigger-map.bats`) keeps the map from rotting: it pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the CI carve-out that keeps the test running when the map is edited alone. Its header is explicit about what it cannot check — whether a coupling is real, and whether it still holds in the other repo, stays a human's job. ### Downstream repositories - [x] No downstream repository is affected by this change ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Expanded contributor guidance for downstream repository changes and follow-up links. * Clarified how to preserve PR template checklists when creating pull requests. * Added detailed downstream repository trigger-map instructions. * **CI** * Documentation changes affecting the trigger map now run unit and controller checks. * **Tests** * Added validation to ensure trigger-map paths, workflow wiring, and repository lists remain consistent. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
44507d2cd0 |
docs(agents): add downstream repository checklist to the PR template
Cozystack is upstream for repositories that nothing keeps in sync with it, and no CI job compares the two sides, so a change here breaks them in silence. Only two narrow paths are automated, and both fire on a release rather than on a PR. It has already happened. The website docs generator works from an app list hardcoded in its own Makefile, and packages/apps/opensearch was never added to it, so an application we ship has no reference page at all. The Terraform provider still offers a Kubernetes version this repo removed. The plugin repo still sends operators to hack/ scripts that were deleted, one of them rewritten as a Go command. Add a Downstream Repositories trigger map to the contributing guide, naming per repository which change here forces a change over there and which file to touch, and a checklist in the PR template that links to it. The template addresses AI agents directly: they author a large share of PRs and will otherwise tick a box without opening the follow-up it claims. The coupling is wider than the obvious ones, and mostly invisible. talm restates the node contract, overlapping ours but already diverged, so it must be diffed rather than overwritten. cozyhr reads an annotation the operator writes and silently merges the wrong values if it is renamed. cozy-proxy is vendored in from its own repo, yet consumes labels produced here and quietly stops handling external VM services if they move. The telemetry server queries metric names emitted here, and a rename just empties its dashboards. None of these fail loudly. Guard the map with a unit test, because a map about silent drift must not drift silently itself. It pins every in-repo path the map cites to a file that exists, pins the map and the checklist to the same repository list, and pins the workflow carve-out below to the map's location, to the job that reads the flag, and to the output that carries it: cutting any one of those three leaves the other two looking wired up. The header states plainly what the suite cannot check, so a green tick is not mistaken for a correct map: whether a coupling is real, and whether it still holds over there, stays a human's job. Editing the map alone is a docs-only PR, which skips the unit tests and would exempt the map from its own guard. Rather than widen the docs-only rule, which also gates the Talos image build, the plan step exports a separate flag that only the unit-test job reads: a docs PR now runs the tests and still builds nothing. It detects the map by its old path too, since a rename is reported as the new path alone. Document that gh pr create --body and --body-file replace the body wholesale, which would otherwise drop the checklist from every CLI-authored PR. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6f5889816b |
feat(ci): guard migrations.targetVersion against silent skip (#2771)
## Summary - Add `hack/check-migrations-target.sh` that fails when the latest numbered file under `packages/core/platform/images/migrations/migrations/` is not matched by a bumped `migrations.targetVersion` in `packages/core/platform/values.yaml`. - Wire it into `unit-tests` via a new `migrations-target-check` Makefile target so PR CI catches the regression mechanically. `run-migrations.sh:34` iterates `seq $CURRENT_VERSION $((TARGET_VERSION - 1))`, so a migration whose number equals the configured `targetVersion` is silently skipped. The cost is state corruption on upgrade — a class of bug worth eliminating with a 10-line check rather than relying on reviewer vigilance. ## Test plan - [x] `make migrations-target-check` passes on `main` HEAD (`targetVersion=43`, latest migration `42`, `43 >= 43`). - [x] Synthetic failure: `touch packages/core/platform/images/migrations/migrations/99` makes the script exit 1 with a clear error pointing at `packages/core/platform/values.yaml` and the required bump. - [x] Tested on macOS (BSD `ls` + mikefarah `yq`). ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Added a CI validation step to unit tests to ensure `migrations.targetVersion` is kept in sync with newly added database migrations, preventing upgrade scripts from skipping new migration steps. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
9223f2ed0e |
feat(agents): block hardwrapped prose in markdown and GitHub bodies (#3297)
## What this PR does
Markdown renderers collapse a soft line break into a space, so a
paragraph hardwrapped at ~80 columns renders exactly like the same
paragraph on one line — while breaking narrow viewports, mangling nested
list and table rendering, and turning a one-word edit into a diff that
reflows the entire block. `AGENTS.md` now states the rule for markdown
files and for every body an agent publishes to GitHub, and names what
keeps its line breaks: code blocks, lists, tables, headings,
blockquotes, front matter, badge stacks, and explicit `<br>` breaks.
Commit message bodies keep their ~72-column wrap.
A documented rule alone does not survive the reflex it fights: an
~80-column wrap is *correct* in Go comments and commit bodies, where
coding agents spend most of their time, and it leaks from there into
documentation and GitHub. So the rule is also checked mechanically at
the moment of writing. `.claude/hooks/md-no-hardwrap.py` is a PreToolUse
hook that refuses a `Write`/`Edit`/`MultiEdit` adding a hardwrapped
paragraph, list item or blockquote to a markdown file, and a `gh`
command publishing one via `--body`, `--body-file`, a heredoc, or an
`api` body field.
Judging what an edit *adds* is the whole difficulty. Comparing the text
of each wrapped block refuses a typo fix inside a paragraph that was
already wrapped; counting the blocks lets a file be rewritten as
entirely fresh hardwrapped prose so long as the total does not rise. The
hook therefore asks about the break rather than the block: a soft break
is the edit's doing only when both lines it joins are lines the edit
wrote.
Whatever it cannot read with certainty, it allows — a body the shell
assembles at runtime, an unsplittable command line, a `--body-file` the
same command has yet to write. A guard that blocks legitimate work gets
switched off, and then it guards nothing. Vendored charts, published
changelogs and Go templates are out of scope, so
`hack/upload-releasenotes.sh` keeps publishing its historical release
notes.
The rule is scoped honestly: it governs prose this repository owns, and
new or changed paragraphs. Much of the tree predates it and is wrapped;
that is not a backlog to go and fix.
`hack/md-no-hardwrap.bats` covers both surfaces with 62 cases and is
picked up by `make unit-tests` automatically; it requires `python3`.
Note for anyone with a personal `.claude/settings.json`: `.gitignore`
previously ignored `.claude/` wholesale, and this PR starts tracking the
hook and its registration by name. Git overwrites a previously-ignored
file on pull without warning, so move personal overrides to
`.claude/settings.local.json`.
Named in honour of IvanHunters 🙂
### Screenshots
Not applicable — no UI changes.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added automated checks to prevent newly introduced hardwrapped
Markdown prose during content edits and publishing commands.
* Reports detected formatting issues with affected line ranges and
guidance.
* Supports Markdown edits and relevant command-line publication
workflows while excluding code and other valid formatting patterns.
* **Documentation**
* Added Markdown prose-formatting guidance for coding assistants.
* **Tests**
* Added comprehensive coverage for valid, invalid, excluded, and
malformed-input scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
a4bbc74b61 |
feat(agents): add md-no-hardwrap hook enforcing the prose rule
An ~80-column wrap is correct in Go comments and commit bodies, where coding agents spend most of their time, and the reflex leaks from there into markdown files and GitHub bodies. A documented rule alone does not survive that reflex, so check it mechanically at the moment of writing. The hook refuses a Write/Edit/MultiEdit that would add a hardwrapped paragraph, list item or blockquote to a markdown file, and a gh command that would publish one to GitHub via --body, --body-file, a heredoc, or an api body field. Judging what an edit ADDS is the whole difficulty. Comparing the text of each wrapped block refuses a typo fix inside a paragraph that was already wrapped; counting the blocks lets a file be rewritten as entirely fresh hardwrapped prose so long as the total does not rise. So the hook asks about the break rather than the block: a soft break is the edit's doing only when both lines it joins are lines the edit wrote. Whatever it cannot read with certainty, it allows -- a body the shell assembles at runtime, an unsplittable command line, a path that is not a regular file, a --body-file the same command has yet to write. A guard that blocks legitimate work gets switched off, and then it guards nothing. Vendored charts, published changelogs and Go templates are out of scope entirely, so hack/upload-releasenotes.sh keeps publishing its historical release notes. Line breaks are only rejected inside prose. Code blocks, tables, HTML, front matter, link reference definitions, footnotes, badge stacks and explicit hard breaks -- two trailing spaces or a trailing backslash, which render as <br> -- keep their breaks, because reflowing those would change the rendered output. A heredoc is attributed to the command that opened it, so a YAML manifest piped to kubectl is not read as a PR body. The hook and its registration are tracked by name so contributors get them on clone, while personal agent state stays ignored; personal overrides belong in .claude/settings.local.json. hack/md-no-hardwrap.bats covers both surfaces with 60 cases and make unit-tests picks it up automatically. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
395af115b5 |
docs(agents): require one continuous line per paragraph
Markdown renderers collapse a soft line break into a space, so a paragraph hardwrapped at ~80 columns renders exactly like the same paragraph on one line -- while breaking narrow viewports, mangling nested list and table rendering, and turning a one-word edit into a diff that reflows the entire block. The rule covers prose this repository owns, in markdown files and in every body published to GitHub: PR bodies, issue bodies, review comments, release notes. It governs new and changed paragraphs; the existing tree is wrapped in many places and is not a backlog of defects to go and fix. Vendored upstream chart docs stay verbatim, published changelogs stay as published, and commit message bodies keep the ~72-column wrap that git log and its tooling expect. Structure that needs its line breaks is unaffected: code blocks, lists, tables, headings, blockquotes, front matter, badge stacks, and explicit hard breaks, which render as <br> and cannot be reflowed without changing the output. AGENTS.md opened with a wrapped paragraph of its own; reflow it. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
f24b56327b |
fix(platform): route OIDC backend calls through the in-cluster Keycloak by default
authentication.oidc.keycloakInternalUrl defaulted to an empty string, which left oauth2-proxy performing OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once tenant-root is ready, and tenant-root is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up. The bundled Keycloak is the only supported provider — the external hostname is templated as keycloak.<host> and the realm is fixed — so backend-to-backend calls have no reason to leave the cluster. Default the value to http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy, the same Service address keycloak-configure already uses. The dashboard and linstor-gui gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo and logout requests to the Service, while browser redirects stay on the external host. Those calls travel as plaintext HTTP between pods, as keycloak-configure's admin calls to the same address already do. The value remains overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to an empty string restores external discovery. Existing OIDC clusters that never set it get a gatekeeper rollout onto the internal path on upgrade. Enabling OIDC swaps the dashboard's token-proxy container for oauth2-proxy, so the E2E install now drives that swap to completion: it waits for the auth-proxy container to appear, then gates on the gatekeeper rollout. Readiness alone proves nothing there — the HelmRelease is still Ready from the token-proxy install, and the Deployment keeps the old ReplicaSet Available while a new pod crashloops, so only the rollout catches a regression back to external discovery. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
caf4645cf2 |
docs(harbor): fix SeaweedFS discovery key described as annotation
The prerequisites section described the resolved SeaweedFS class as being surfaced on the tenant namespace as the `namespace.cozystack.io/seaweedfs` annotation. The tenant chart renders that key under `metadata.labels`, alongside the sibling `/etcd`, `/ingress`, `/gateway`, `/monitoring` and `/host` keys; no annotation of that name exists. The wrong noun sends an operator to the wrong field. When a Harbor release stays unreconciled waiting on `BucketInfo`, the documented check is to confirm the tenant chain resolved a SeaweedFS class. Looking for it in `.metadata.annotations` returns nothing even on a correctly configured tenant, which invites the wrong conclusion that the tenant chain is broken and sends the investigation away from the actual cause. Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5d881f111a |
fix(fluxcd): omit empty distribution.artifact in FluxInstance for guest clusters (#3284)
## What this PR does
Guest Kubernetes clusters (`apps/kubernetes`) with the `fluxcd` addon
enabled never became
`Ready`. The `flux-instance` template renders
`spec.distribution.artifact` unconditionally,
unlike the sibling optional fields (`artifactPullSecret`,
`imagePullSecret`, `variant`, which
are all `{{- if }}`-guarded). The umbrella default is empty (`artifact:
""`) so the operator
uses its embedded manifests, which is the intended air-gapped default
(#964, and the upstream
recommendation). But an empty value renders as YAML `null`, and the
flux-operator v0.50.0 CRD
types `distribution.artifact` as a non-nullable `string` (pattern
`^oci://.*$`), so the API
rejects the `FluxInstance`.
Result: the addon's HelmRelease looped install/uninstall
(`remediation.retries: -1`), the
parent `kubernetes` release stayed `InProgress`, and the cluster CR was
stuck `Ready=Unknown`.
This guards the field so an empty value is omitted entirely. Behaviour:
- empty (default) -> field omitted -> operator falls back to embedded
manifests: **air-gapped
default preserved**, and the CRD accepts it (artifact is not in
`distribution.required`);
- non-empty -> renders as before; `valuesOverride` still applies.
The template is a vendored subchart (`make update` runs `helm pull`), so
the change is carried
as `patches/guard-distribution-artifact.diff` and applied by the
`update` target, matching the
convention of the sibling `fluxcd-operator` package (`patch -p1 <
patches/*.diff`).
Verified with `helm template packages/system/fluxcd`:
- default values -> no `artifact:` line (before this PR: `artifact:
null`);
- `--set
flux-instance.instance.distribution.artifact=oci://example.com/x:1` ->
renders.
The affected path is off by default (`addons.fluxcd.enabled: false`) and
not exercised by the
in-repo CI values, which is why default e2e stayed green.
Closes #3283
### Release note
```release-note
fix(fluxcd): guest Kubernetes clusters with the fluxcd addon enabled no longer fail to install; an empty distribution artifact is now omitted so the operator uses its embedded manifests (air-gapped default preserved) instead of rendering an invalid null
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Prevented Flux instance configuration from including an empty
distribution artifact value, which could cause validation failures.
* Improved the generated FluxInstance manifest so the artifact field is
omitted entirely when no distribution artifact is provided.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Ivan Okhotnikov
|
da47b70e2c |
fix(fluxcd): omit empty distribution.artifact in FluxInstance
The flux-instance template rendered spec.distribution.artifact unconditionally, unlike the sibling optional fields (artifactPullSecret, imagePullSecret, variant). The umbrella default is empty to use the operator's embedded manifests (the air-gapped default from #964), but an empty value renders as YAML null. In the flux-operator v0.50.0 CRD the field is optional yet typed 'string' with pattern ^oci://.*$ and is not nullable, so the rendered null (like an empty string) fails schema validation and the API rejects the FluxInstance. Guest Kubernetes clusters with the fluxcd addon enabled could therefore never install it: the sub-release looped install/uninstall, the parent kubernetes release stayed InProgress, and the cluster never reached Ready. Guard the field so an empty value is omitted (the operator falls back to its embedded manifests, preserving the air-gapped default) while a non-empty artifact still renders. The template lives in the vendored subchart, so the change is carried as a patch applied by 'make update'. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
3380bb4ab6 |
refactor(hack): replace ls|grep with pure-bash glob (shellcheck SC2010)
Filenames in packages/core/platform/images/migrations/migrations/ are numeric-only by convention so the original `ls -1 | grep -E` filter was safe in practice, but the cleaner pure-bash form is equivalent and silences SC2010 without an annotation. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
IvanHunters
|
c3d3e2596e |
feat(ci): guard migrations.targetVersion against silent skip
Adds hack/check-migrations-target.sh and wires it into unit-tests via the migrations-target-check Makefile target. The check fails when the highest numbered file under packages/core/platform/images/migrations/migrations/ is not matched by a bumped migrations.targetVersion in packages/core/platform/values.yaml. run-migrations.sh:34 loops `seq $CURRENT_VERSION $((TARGET_VERSION - 1))`, so a migration whose number equals targetVersion never runs and the intended state change is silently skipped on cluster upgrade. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Aleksei Sviridkin
|
f478915935 |
fix(dashboard): unbreak CORS on expired session for k8s API calls (#2788)
## What this PR does Fix two related issues that cause the dashboard SPA to break with CORS errors once the `kc-access` cookie expires. 1. `KeycloakClient` for dashboard had no `webOrigins` set, so the operator did not touch the field in Keycloak after initial creation. After any `rootUrl` change the existing `webOrigins` value got stuck at an old hostname and blocked CORS from the SPA. Setting it to `+` makes Keycloak derive the allowed origins from `redirectUris`. 2. `oauth2-proxy` answered `302` (redirect to Keycloak `/auth`) for unauthenticated XHR/fetch on `/api` and `/apis`. The login page does not emit `Access-Control-Allow-Origin`, so the browser blocked the fetch with a CORS error and the SPA had no way to recover. Adding `--api-route` makes `oauth2-proxy` answer `401` without a redirect for these paths, so the SPA can handle re-auth itself via a top-level navigation. ### Release note ```release-note fix(dashboard): dashboard SPA no longer fails with CORS errors after the kc-access cookie expires; oauth2-proxy returns 401 (not 302) for /api and /apis, and the KeycloakClient declares webOrigins: ["+"]. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Chores * Updated API routing configuration to properly handle requests to API endpoints * Enhanced web origins configuration for improved authentication system compatibility <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
0044221d31 |
fix(apps/vpn): remove invalid foo field from urls Secret (#3281)
## What this PR does Removes a leftover debug field `foo` from the `<release>-urls` Secret template in the VPN app chart. The `foo` field is not part of the `core/v1` Secret schema (which only allows `data`, `stringData`, `type`, `immutable`, and `metadata`). It went unnoticed under client-side apply, but under server-side apply the stricter schema validation rejects it with: ```text .foo: field not declared in schema ``` As a result, any VPN application upgrade fails and the release is left in `UpgradeFailed` state. ### Release note ```release-note fix(apps/vpn): remove invalid `foo` field from the urls Secret that broke Helm upgrades under server-side apply ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added per-user Shadowsocks connection links to the VPN secret. * Links include the configured encryption method, host, port, and connection name for easier client setup. * **Bug Fixes** * Removed obsolete serialized password data from the VPN secret. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrey Kolkov
|
bf69eff7a5 |
feat(keycloak): add S3 backup for the CNPG database (#3174)
## What this PR does Adds optional scheduled S3 backups for the `keycloak-db` CloudNativePG cluster. When `backup.enabled=true` with a `backup.destinationPath`, the Cluster gets `spec.backup.barmanObjectStore` (gzip-compressed data + WAL streams, a configurable `retentionPolicy`) and a `ScheduledBackup` drives recurring base backups. S3 credentials are either inlined into a rendered `keycloak-db-backup-s3-creds` Secret, or referenced via `backup.existingSecretName` (a Secret carrying `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`). Disabled by default — no change to existing deployments; the object store coordinates and credentials are supplied per-environment by the operator. This is the Keycloak/Postgres half of backing up a Cozystack portal control plane. The portal-side etcd backups (system Talos etcd + api-server etcd) live in the companion downstream PR aenix-org/cozyportal#1008. ### Release note ```release-note feat(keycloak): add optional S3 backups (CNPG Barman) for the Keycloak database, configurable via `backup.*` values and disabled by default. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable scheduled backups for the Keycloak CNPG database to an S3-compatible object store, including destination prefix, optional endpoint URL, retention policy, and cron schedule. * Backups and scheduled backup resources are only created when all required backup settings are provided, avoiding incomplete configuration. * Supports backup credentials via inline values (creates the needed Secret automatically) or via an existing Secret reference. * **Documentation** * Added guidance on the cron schedule format required for the scheduler. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ivan Okhotnikov
|
70a2284551 |
fix(apps/vpn): remove invalid foo field from urls Secret
A leftover debug field "foo" in the <release>-urls Secret template violates the core/v1 Secret schema. It went unnoticed under client-side apply but breaks Helm upgrades under server-side apply with ".foo: field not declared in schema", leaving the release in UpgradeFailed state. Drop the debug block. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Myasnikov Daniil
|
103b8e16ee |
test(e2e): diff-driven install selector select-install.sh (#2847)
Add hack/select-install.sh: given the E2E suites select-e2e.sh picked, emit the forward dependsOn closure over the PackageSource graph — the minimal set of packages to install for those suites. Companion to the reverse (test-selection) walk in select-e2e.sh; keeps the cozystack-engine edge since install needs it. Suite->source mapping follows the <suite>-application convention with a bare-<suite> fallback plus explicit cases for irregular names (kubernetes/vminstance/securitygroup) and core-platform features that have no separately-enabled package (serviceexposure). Includes a --validate mode (reachability + cycle check) and a 13-case unit suite auto-discovered by make unit-tests. Track A of #2847; tracked in #3278. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
eea8b73460 |
feat(kubernetes): per-node-group nodeHealthCheck overrides (#3053)
Adds per-node-group overrides for the worker MachineHealthCheck tuning introduced as a cluster-wide `nodeHealthCheck` by #2931. **What:** optional `nodeGroups[name].maxUnhealthy` and `nodeGroups[name].nodeStartupTimeout`; when unset a group inherits the cluster-wide `nodeHealthCheck.*` (no behaviour change). The per-group value reuses the existing int-or-string validation; an invalid override fails the render naming the offending group. **Why:** the MachineHealthCheck is rendered per node group, so groups can warrant different remediation tolerances (e.g. a stateful group at `0%`, stateless at `50%`). This is the per-group granularity of the standalone proposal (#2752 / #2935), rebuilt on top of #2931's `nodeHealthCheck`. **Notes:** stacked on `phase1-talos-migration` (#2931) — retarget to `main` once it merges. Supersedes #2935 (original author credited as co-author). helm-unittest covers per-group override (percent + bare int), `nodeStartupTimeout` override, fallback-when-unset, and invalid-override rejection. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added per-node-group overrides for health-check settings: `maxUnhealthy` and `nodeStartupTimeout`, with automatic fallback to cluster-wide `nodeHealthCheck` when unset. * **Bug Fixes** * Improved `maxUnhealthy` parsing/rendering to support both bare integers and percentage strings, including more precise validation errors for the exact override path. * **Documentation** * Updated kubernetes app docs and values schemas (including CRD chart-values schema) to document the new override fields. * **Tests** * Added MachineHealthCheck rendering tests for override, fallback, and invalid `maxUnhealthy` (e.g., `"abc"`) cases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniilandmattia-eleuteri
|
db4db4a6e8 |
feat(kubernetes): per-node-group nodeHealthCheck overrides
Add optional nodeGroups[name].maxUnhealthy and nodeGroups[name].nodeStartupTimeout
fields that override the cluster-wide nodeHealthCheck defaults per worker node
group. When unset, a group inherits nodeHealthCheck.{maxUnhealthy,nodeStartupTimeout},
so existing values are unaffected. The MachineHealthCheck render resolves the
per-group value first, reuses the existing int-or-string validation, and names
the offending group in the fail() message when an override is invalid.
Delivers the per-group granularity of the standalone maxUnhealthy proposal on
top of the cluster-wide nodeHealthCheck from the Talos worker bootstrap PR.
Co-authored-by: mattia-eleuteri <mattia@hidora.io>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
renovate[bot]
|
8a5bc68b43 | chore(deps): update actions/checkout action to v6.0.3 | ||
|
renovate[bot]
|
9dc63bd412 | chore(deps): update github.com/kubevirt/csi-driver digest to 27b52aa | ||
|
myasnikovdaniil
|
99a28d82c2 |
test(e2e): migrate app suite from BATS to Chainsaw and wire into CI (#2826)
## What this PR does Migrates the E2E **app** test suite from BATS to [Kyverno Chainsaw](https://github.com/kyverno/chainsaw) — a declarative, Kubernetes-native E2E framework (CNCF, part of the Kyverno project; successor to KUTTL) — and wires it into CI as the replacement for the per-app BATS loop. This grows the original pilot (postgres + bucket) into the full suite. All 22 app suites are ported under `hack/e2e-chainsaw/` and the `hack/e2e-apps/*.bats` files are removed: `postgres`, `bucket`, `mariadb`, `mongodb`, `redis`, `qdrant`, `clickhouse`, `kafka`, `etcd`, `openbao`, `harbor`, `foundationdb`, `external-dns`, `kuberture`, `vminstance`, `gateway`, `kubernetes-latest`, `kubernetes-previous`, `kubernetes-oidc-system`, `kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure`. **Approach** - **Declarative** suites assert on `status.conditions` and concrete fields. The `timeout N sh -ec "until kubectl get ..."` + `kubectl wait` pair that appeared ~100 times collapses into a single `assert` that polls existence and state together, with a structured diff on failure and automatic `events`/`describe`/`podLogs` capture via `catch` (previously only `harbor.bats` did this, by hand). - **Imperative** suites keep their logic in `script` steps: `openbao` init/unseal, `kuberture` external-dns split-horizon probes, `vminstance`, the `gateway` admission/impersonation cases, and `kubernetes-latest`/`previous`, which wrap the relocated `hack/e2e-chainsaw/_lib/run-kubernetes.sh` verbatim (Kamaji bring-up, LB/NFS/ouroboros checks). - `gateway` tests derive the tenant apex from the namespace `namespace.cozystack.io/host` label at runtime, so they are host-independent. **CI** - The `e2e` job now runs `chainsaw test hack/e2e-chainsaw/` via a new `test-chainsaw` target and uploads the JUnit `chainsaw-report.xml`. - The `chainsaw` binary is added to the e2e-sandbox image. - `install-cozystack` and `test-openapi` stay BATS (cluster bootstrap + OpenAPI checks). - The per-app 3-retry loop is dropped — assertion polling replaces the fixed-timeout flakiness it papered over. **Validation** Ran against a development cluster: the DB/app, storage, and VM suites pass; the three suites that depend on platform features not present on that cluster (`gateway`, `kuberture`, `external-dns`) and the two heavyweight `kubernetes-*` suites are exercised by this PR's CI run on a freshly installed platform. Note for reviewers: service-port asserts use the `(ports[*].port)` projection form rather than a number-literal filter (`` ports[?port == `N`] ``), because Chainsaw v0.2.15 mis-evaluates JMESPath number-literal comparisons. **2026-07-09: reconciled with main (~520 commits of drift)** - etcd suite re-ported to the v1alpha2 operator contracts (#2859): `readyMembers=3` gate, pod-label/`/scale`/WorkloadMonitor/metrics/defrag contracts, plus the `ETCD_E2E_S3_ROUNDTRIP`-gated backup round-trip driving `examples/backups/etcd` - 12 new `ingress-hostname-policy` gateway cases ported (apex derived at runtime from the tenant-root namespace label) - post-branch bats drift folded in: `mariadb-single` webhook guard, kafka `c1.small` presets, bucket/harbor 2m BucketClaim fail-fast, bucket readonly-denial promoted to a hard fail (cosi-driver v0.3.1), qdrant PVC-reclaim guard (#3059), kuberture fail-loud negations, SC-fallback-default test (#2872 B1) - 4 suites that only exist on main since the branch was cut are ported: `kubernetes-oidc-system`, `kubernetes-oidc-customconfig`, `securitygroup`, `serviceexposure` - CI seams: `nightly.yaml` converted to the chainsaw invocation, `run-kubernetes.sh` reconciled (Talos/CABPT waits, tenant drain, LINSTOR pool wait, talos-image-cache under `_lib/`), `e2e-capture-dataplane.sh` replicated in the Chainsaw global catch ### Release note ```release-note fix(kubernetes): tenant Kubernetes teardown no longer hangs when the cluster has no working nodes — the pre-delete hook now bounds its wait for the in-tenant HelmReleases and force-clears their Flux finalizers on timeout (#3271) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * End-to-end testing is now driven by a unified Chainsaw suite (platform apps, Kubernetes, networking, security groups, service exposure). * Test Impact Analysis selects affected suites, with an option to run the full suite. * CI produces a JUnit report artifact and collects suite-scoped diagnostics on failure. * **Bug Fixes** * CI E2E now fails fast and emits clearer Kubernetes/HelmRelease and sandbox bucket state diagnostics. * Improved teardown reliability with bounded wait + finalizer handling to avoid hangs. * **Documentation** * Updated E2E testing docs and local guidance to reflect the new Chainsaw-based workflow and suite paths. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
8af4ed543e |
test(e2e): address review nits — shellcheck directive + ports-workaround TODO
Non-blocking review notes from #2826: - Add `# shellcheck shell=bash` to the four sourced `_lib/*.sh` helpers (etcd-cleanup, remediation-guard, run-kubernetes, talos-image-cache) so shellcheck stops flagging SC2148 (no shell directive) — they are only ever `. sourced`, so there is no runtime interpreter to infer from a shebang. - Add a `# TODO(chainsaw>0.2.15)` marker at the postgres Service asserts explaining the `(ports[0].port)`/`(ports[*].port)` projection is a v0.2.15 number-literal-filter workaround (and that `ports[0]` checks only the first port), so it gets reverted on a chainsaw bump; cross-references the same workaround in kafka/mariadb/harbor and the e2e-testing.md gotchas section. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
18715f8c63 |
docs(agents): document Chainsaw v0.2.15 gotchas in the e2e-testing guide
The v0.2.15 syntax/semantics traps were tribal knowledge scattered across PR review notes (the number-literal caveat lived only in #2826's body, which evaporates on merge). Consolidate them into a durable subsection so the next author finds them before hitting each one fresh: filter-as-list conditions, no number-literals/backticks in JMESPath, error: fails hard on an unknown GVK, no conditional Test-level skip, step-level-only finally, bounded/blocking cleanup surfacing latent teardown bugs, projection-style failure diffs, and the catch-script cwd. Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
b99df11b36 |
fix(kubernetes): bump tenant worker Talos to v1.13.6 (#3269)
## What this PR does Tenant worker VMs were pinned to Talos v1.13.0, which ships Linux 6.18.24. That kernel predates the fixes for two related KVM x86 shadow-paging use-after-free bugs in `arch/x86/kvm/mmu/mmu.c`: CVE-2026-46113, fixed upstream in `0cb2af2ea66ad8ff195c156ea690f11216285bdf` and released in 6.18.30, and CVE-2026-53359, fixed in `81ccda30b4e83d8f5cc4fd50503c44e3a33abfeb` and released in 6.18.38. A worker's own kernel runs KVM whenever a tenant enables nested virtualization inside the VM, and either bug lets a nested guest escape into the worker. This bumps the worker pin to Talos v1.13.6, which ships Linux 6.18.38. Stable kernel branches are cumulative, so v1.13.6 is the first v1.13 patch release carrying both fixes. The pinned v1.13.0 ships 6.18.24 and carries neither; v1.13.5 ships 6.18.36 and carries only CVE-2026-46113. Nested virtualization stays available to tenants; the kernel bump is the fix, not a capability removal. The image-factory schematic is unchanged and resolves for the new version: both the `openstack-amd64.raw.xz` artifact and the `installer/<schematicID>:v1.13.6` manifest are served by the public factory. Talos v1.13 supports Kubernetes 1.31 through 1.36, so the chart's Talos-to-Kubernetes support window is unaffected and no tenant `spec.version` needs to move. Applying this rolls each worker pool once. The new pin changes the rendered worker spec, which rotates the content hash in the `KubevirtMachineTemplate` and `talos-reconcile` Job names, and CAPI replaces the worker VMs to deliver the new kernel. The pinned hashes in the chart tests are recomputed to match. A second commit fixes the support-matrix lookup itself. The matrix dict was keyed on the full patch version, so every Talos patch bump dropped out of the dict and silently disabled the render-time guard — exactly the mismatch the guard exists to catch. The lookup now derives the minor from `talos.version`. The supported-Kubernetes window is unchanged; unknown minors are still skipped, leaving compatibility to whoever overrode the pin. ### Screenshots Not applicable — no UI changes. ### Release note ```release-note fix(kubernetes): bump tenant worker Talos to v1.13.6, picking up the Linux 6.18.38 kernel fixes for CVE-2026-46113 and CVE-2026-53359 (KVM shadow-paging use-after-free, nested-guest escape into the worker VM). Nested virtualization remains available. Applying this upgrade rolls each tenant worker pool once, as the worker template is renamed to carry the new image. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Updated the default Talos worker release to v1.13.6 for Kubernetes cluster provisioning. - Improved compatibility checks to apply consistently across Talos patch releases within the same minor version. - **Bug Fixes** - Corrected generated resource naming expectations when storage settings or GPU configuration change. - **Tests** - Added coverage confirming Kubernetes clusters render successfully with Talos v1.13.6. <!-- end of auto-generated comment: release notes by coderabbit.ai -->v1.6.0-rc.2 |
||
|
renovate[bot]
|
ebecbd71c9 | chore(deps): update altinity/clickhouse-backup docker tag to v2.7.4 | ||
|
myasnikovdaniil
|
c8ee5dc317 |
chore(deps): update container image & action digests (#3232)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [actions/stale](https://redirect.github.com/actions/stale) ([changelog](https://redirect.github.com/actions/stale/compare/eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899..1e223db275d687790206a7acac4d1a11bd6fe629)) | action | digest | `eb5cf3a` → `1e223db` | | alpine | final | pinDigest | → `28bd5fe` | | alpine | stage | pinDigest | → `28bd5fe` | | [altinity/clickhouse-backup](https://redirect.github.com/Altinity/clickhouse-backup) | final | pinDigest | → `b8a098d` | | clickhouse/clickhouse-server | final | pinDigest | → `f21e0f9` | | debian | final | pinDigest | → `ee12ffb` | | debian | final | pinDigest | → `30482e8` | | debian | stage | pinDigest | → `30482e8` | | debian | final | pinDigest | → `28de087` | | docker.io/golang | stage | pinDigest | → `079e598` | | docker.io/library/alpine | final | pinDigest | → `1435830` | | docker.io/library/alpine | final | pinDigest | → `28bd5fe` | | docker.io/library/golang | stage | pinDigest | → `079e598` | | [docker/dockerfile](https://redirect.github.com/moby/buildkit) | syntax | pinDigest | → `e2a8561` | | gcr.io/distroless/static | final | pinDigest | → `d29e660` | | gcr.io/distroless/static | stage | pinDigest | → `d29e660` | | golang | final | pinDigest | → `0178a64` | | golang | stage | pinDigest | → `079e598` | | golang | stage | pinDigest | → `0178a64` | | golang | final | pinDigest | → `079e598` | | [grafana/grafana](https://redirect.github.com/grafana/grafana) | final | pinDigest | → `f739273` | | [nginxinc/nginx-unprivileged](https://redirect.github.com/nginx/docker-nginx-unprivileged) | final | pinDigest | → `0c79d56` | | [node](https://redirect.github.com/nodejs/node) | stage | pinDigest | → `16e22a5` | | quay.io/centos/centos | final | pinDigest | → `0c9db82` | | quay.io/cilium/cilium | stage | pinDigest | → `20fbbc1` | | [quay.io/poseidon/matchbox](https://redirect.github.com/poseidon/matchbox) | final | pinDigest | → `06bcdae` | | ubuntu | final | pinDigest | → `4fbb8e6` | | ubuntu | final | pinDigest | → `0e0a0fc` | | ubuntu | stage | pinDigest | → `0e0a0fc` | --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - "before 6am on monday" - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/cozystack/cozystack). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yNDIuMiIsInVwZGF0ZWRJblZlciI6IjQzLjI0Mi4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJhdXRvbWF0ZWQiLCJkby1ub3QtbWVyZ2UvaG9sZCJdfQ==--> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
302a72f14a |
Merge origin/main into feat/e2e-chainsaw-example
Reconciles the talos-image-cache conflicts created when #3254 merged to main: keep the Chainsaw _lib/ port (hack/e2e-chainsaw/_lib/talos-image-cache.sh, a superset of #3254's logic + the node-join diagnose helper), drop the BATS-era hack/e2e-apps/talos-image-cache.sh, and keep the _lib/-path versions of the mirror manifest, install bats, and unit test. Also picks up main's zizmor action-pinning (#3223, unblocks pre-commit/zizmor) and Talos v1.13.6 (#3240). Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.8
|
02b87b63f2 |
fix(apps/kubernetes): bound the pre-delete hook's tenant HelmRelease wait (#3271)
The pre-delete hook (delete.yaml) deleted the tenant's addon HelmReleases with
an unbounded `kubectl delete … --wait=true`. When the tenant has no working
nodes (LINSTOR/ZFS exhaustion, worker VMs that never joined, or a nodeGroup at
minReplicas: 0), those uninstalls cannot converge — in-tenant cilium stays
InstallFailed because cilium-operator has nothing to schedule on — so the hook
hung indefinitely, the parent Helm uninstall failed on the pre-cleanup Job, and
the Kubernetes CR was stuck deleting.
Bound the wait with --timeout=120s; on expiry, force-clear the Flux finalizer
from any child HelmRelease still present so it can be garbage-collected — safe
because the whole tenant (control plane torn down in the steps below) is being
destroyed, so there is nothing left to orphan. A healthy tenant's children
uninstall well within the timeout and never reach the force path.
Also unblocks the render-side kubernetes-oidc-{system,customconfig} Chainsaw
e2e suites, which deterministically hit this teardown hang under the Chainsaw
cleanup budget.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
renovate[bot]
|
5b9b420ffe | chore(deps): update container image & action digests | ||
|
Aleksei Sviridkin
|
a3829ab66c |
fix(talos): bump Talos to v1.13.6 (CVE-2026-53359) (#3240)
## What this PR does Bumps the platform Talos profiles from v1.13.5 to v1.13.6. CVE-2026-53359 and CVE-2026-46113 are related KVM x86 shadow-paging use-after-free bugs that let a guest escape to the host, and the fix is complete only once both upstream patches (`81ccda30b4e8`, `0cb2af2ea66a`) are applied. On the 6.18 stable line CVE-2026-46113 landed in 6.18.30 and CVE-2026-53359 in 6.18.38. Talos v1.13.5 runs Linux 6.18.36, so it carries only the first of the two. Talos v1.13.6 runs Linux 6.18.38 and carries both. Nested virtualization stays enabled. The bugs are fixed in the kernel, so there is no reason to take the feature away from workloads that rely on it — bumping the kernel is the complete fix. Profiles are regenerated with `hack/gen-profiles.sh v1.13.6`. Alongside the Talos version and the installer reference, this refreshes the pinned system-extension digests: the firmware extensions move from the 20260519 to the 20260622 build, and `intel-ucode` keeps its `20260512` tag but resolves to a new digest because upstream re-pushed that tag as an OCI image index (identical config and layer blobs underneath). References: [Talos v1.13.6 release notes](https://github.com/siderolabs/talos/releases/tag/v1.13.6) (`Linux: 6.18.38`) ### Screenshots Not applicable, no UI changes. ### Release note ```release-note fix(talos): bump Talos Linux to v1.13.6, which ships Linux 6.18.38 with the kernel fixes for the CVE-2026-53359 and CVE-2026-46113 KVM guest-to-host escapes ``` |
||
|
myasnikovdaniil
|
06b17bf7a9 |
ci: move remaining workflows off the persistent self-hosted runner (#3268)
## What this PR does Completes the last step of #2937 — no workflow depends on the persistent self-hosted runner anymore, so it can be decommissioned. PR builds, the main cache warmer, nightly, promote-rc, and pull-requests-release already run on the ephemeral pool; this PR moves the stragglers: - **`tags.yaml` / `prepare-release`** moves to the large ephemeral shape (same as the main cache warmer). Release builds now warm-start from the shared `mode=max` build cache in OCIR (written by `build-main.yaml` on every push to main) while still pushing images to GHCR. Cache manifests are registry-portable, and a missing cache ref degrades harmlessly into a cold build. Without this wiring, every tag build on an ephemeral runner would start 100% cold — the persistent runner used to warm-start from local buildkit state. - **`tags.yaml` / `generate-changelog` and `update-website-docs`**, **`backport.yaml`**, **`update-releasenotes.yaml`** move to GitHub-hosted runners: these jobs only need git and the GitHub API, no docker or repo toolchain. - **`pull-requests.yaml`**: the `debug` label no longer reroutes jobs to the self-hosted runner — that path would queue forever once the runner is gone. The label still gates the SSH breakpoint on e2e failure, which works from ephemeral runners (the breakpoint connects out to the rendezvous server). - **`flux-shard-operator`** was the last image built with the stale inline/`:latest` cache pattern (always cold since #2711); it now uses the shared `cache-args` registry cache like every other image. Operational notes for the actual decommission: - The breakpoint rendezvous server (`BREAKPOINT_ENDPOINT`) is separate infrastructure — if it happens to live on the same host as the runner, it needs a new home before the host is retired. - `tags.yaml` now uses the `OCIR_USER`/`OCIR_TOKEN` secrets on tag events (already used there by `nightly.yaml` and `release-e2e` before it). ### Screenshots Not applicable — CI-only change. ### Release note ```release-note ci: all CI jobs now run on ephemeral or GitHub-hosted runners; release tag builds warm-start from the shared registry build cache ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated multiple GitHub Actions workflows to use more consistent hosted runner environments for checks, PR validation, tagging, backports, and release note updates. * Improved tag/release build setup for ephemeral runners, including optional build toolchain setup and container registry authentication. * Enhanced image build caching for the shard operator to improve packaging efficiency. * **Bug Fixes** * Removed label-driven runner switching in pull request checks to make build and test execution more predictable. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |