mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-10 06:24:09 +09:00
11bac54f931682dd94de4081bb9e8a7eef60bef1
78
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Kirill Ilin
|
c96fe3d3ef |
fix(platform): refuse master as the OIDC realm name
The realm named here is where keycloak-configure creates every platform client and the cluster-admin role, and where dashboard users sign in. Naming master puts all of that inside the realm that holds the Keycloak super-admin, and publishes its login page wherever the platform realm is published — and the guard belongs where the value is set, since the Keycloak chart only reads it once a separate admin hostname is configured. Assisted-By: LLM Signed-off-by: Kirill Ilin <stitch14@yandex.ru> |
||
|
Alexey Artamonov
|
142a14cb69 |
fix(platform): reject an empty realm name instead of coercing it to cozy
realmName resolved through `| default "cozy"`, which fires on both null and "". Unset (null) legitimately means the default, but an explicit "" is a meaningless value and coercing it hid the mistake. Split the two: null falls back to cozy, an empty string reaches the charset guard and is rejected — the same null-vs-"" distinction keycloakInternalUrl already draws a few lines below. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
ff637ed9a2 |
fix(platform): validate the realm-name charset and regenerate kubernetes artifacts
The realm-name guard checked only the type and echoed the parsed value back, so its hint told an operator who wrote `on` to set realmName: "true" — the token YAML had already discarded. Drop the echo and add a slug-charset check: the name is spliced verbatim into URL path segments and a shell token, so a "/" or a space must be rejected at the source rather than split a URL or the hook script downstream. The description edit in the previous commit was hand-applied to the generated kubernetes README and never reached the Go types, leaving pre-commit red; regenerate both with cozyvalues-gen. Finish the oidc-tenant System-mode prose that still read cozy as a fixed name. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
949096694b |
fix(platform): reject non-string realm names and shell-quote the hook realm
A realmName that YAML resolves to an int or a bool (a bare 123, or a reserved word like on/off) was coerced with toString, silently renaming the realm to something the operator never typed (on -> "true"). Fail the render with a quoting hint at the source instead of surfacing it later as a broken issuer URL. The update-email hook Job interpolated the realm into its shell script unquoted, so a name carrying a space or a shell metacharacter could break or inject into the script; single-quote it. The Grafana and managed-cluster issuer helpers build the URL with printf %s straight off the channel value, so guard them with toString for the same non-string reason. Name the configured realm in the tenant kube-apiserver oidc.mode description and the Grafana identity-mode comment, which still read cozy as if it were fixed. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
80d32b3ab7 |
fix(keycloak): complete realm-name coverage and guard non-string names
The enable-update-email hook Job in keycloak-configure still pinned REALM=cozy, so on a custom realmName it waited on a realm the chart never provisions and the cold install timed out — the one seventh consumer left behind, and the only value nobody could set without breaking the install. A bare-integer or YAML-reserved realm name reached the printf %s that builds the derived keycloakInternalUrl as a Go int, rendering a "%!s(...)" format error into the URL; coerce the name to a string once at the source. Document that renaming the realm on a running cluster provisions a fresh empty realm and orphans the accounts in the old one, and note in the OIDC tenant and grafana guides that the realm name is now the configurable default. info/Makefile carried NAME=etcd, so cozyhr drove the wrong release from that package; correct it while touching the file. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
44a9a8e735 |
feat(platform): make the Keycloak realm name configurable
The realm name "cozy" was hardcoded in keycloak-configure (which owns the realm, its clients and the cozystack-cluster-admin roles), in the dashboard and linstor-gui gatekeepers, in the info kubeconfig, and in the Grafana and managed-Kubernetes apiserver OIDC issuers. Operators could not rename the realm without patching vendored templates, which `make update` reverts. The apiserver OIDC issuer is already configurable via talm oidcIssuerUrl, so pointing it at a custom realm left every SSO component on "cozy" and broke login with an issuer mismatch. authentication.oidc.realmName (default "cozy") now rides the _cluster channel so every consumer resolves the same realm, and the default keycloakInternalUrl is derived from it so the realm is named in one place. Assisted-by: LLM Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Yan Bondarenko
|
5ed076cff8 |
feat(kubevirt): expose migration configuration through platform values
Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com> |
||
|
Aleksei Sviridkin
|
b083d8e9e6 |
test(platform): pin the opensearch-operator DNS base wiring
Two ends of the same path. The platform suite pins that the shared values Secret carries networking.clusterDomain to the operator under the subchart key, and the package suite pins that a domain set on that key reaches the manager container as DNS_BASE, with the upstream cluster.local default intact when nothing sets it. Every pattern in the platform suite anchors dnsBase to its opensearch-operator and manager parents rather than matching the key anywhere in the payload. An unanchored pattern is satisfied by any sibling block that happens to carry the same domain, which leaves the assertion green after the block it is meant to pin has been deleted. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
scooby87
|
afc003c464 |
feat(postgres): database read-replica autoscaling via KEDA (#3954)
## What this PR does Adds automatic horizontal autoscaling of a PostgreSQL database's **read replicas** in response to load, implementing the merged design proposal [Database Horizontal Autoscaler](https://github.com/cozystack/community/tree/main/design-proposals/database-horizontal-autoscaling) (community#44). The mechanism is entirely stock — no bespoke operator, no new CRD: - **KEDA as an optional platform component** (`packages/system/keda`, wired into `core/platform` as a source + system-bundle entry). It claims the `external.metrics.k8s.io` APIService that nothing served before. - **`cozy-lib.keda.scaledObject` helper** (`packages/library/cozy-lib/templates/_keda.tpl`) renders a KEDA `ScaledObject`, fail-closed on missing/invalid arguments. - **`packages/apps/postgres` autoscaling** — an `autoscaling` values block (`enabled`, `minReplicas`, `maxReplicas`, `metric`, `target`, `maxReplicationLagSeconds`, `dryRun`, `transition`) validated by `values.schema.json`; under active autoscaling the chart renders a `ScaledObject` and seeds `spec.instances` to a constant floor (`max(replicas, effectiveMin)`), while the postgres release is forced to client-side (two-way) Helm apply so the seed is a merge no-op on upgrade and the KEDA-managed HPA — the runtime writer via the CNPG `Cluster` scale subresource — is not reverted. - **Single-value metric (§1)** — one PromQL query returns `Σ(read load over standbys) + target`, so a stock HPA computes `desired = ceil((Σ+target)/target) = 1 + ceil(Σ/target) = primary + desiredRead`, with no per-pod emission. - **The four static brakes (§5)** — quorum floor `max(minReplicas, maxSyncReplicas+1, 2)` (quorum wins over `maxReplicas`), scale-down pacing (`Pods/1/600s`), a replication-lag clamp folded into the query (freeze both directions while lag exceeds the threshold and the primary is writing, with `max_over_time` hysteresis), and dry-run (render paused). - **Two-phase enable/disable migration** via a `transition` sub-flag: phase 1 stages `spec.instances` to the live count and brings KEDA up with its HPA paused in both directions; flipping to phase 2 hands the runtime count to KEDA, and because the seed re-renders the same count the flip is a merge no-op with no dip. - **Observability** — a database-autoscaling Grafana dashboard and PrometheusRule alerts. **Live validation (dev cluster, CNPG 1.30.0).** The full path works end-to-end: the CNPG `Cluster` scale subresource serves `status.selector`, KEDA renders the HPA, the metric reaches it (`ScalingActive=ValidMetricFound`, `TARGETS 151/150`, `desiredReplicas=2 = 1 primary + 1 replica` — the §1 arithmetic), and the lag-clamp query returns the base value on live VictoriaMetrics. Two packaging fixes were needed and are included: KEDA must run on the cozystack `cozy.local` cluster domain (the upstream `cluster.local` default makes the metrics gRPC target NXDOMAIN and every HPA hangs on `FailedGetExternalMetric`), and the operator's client-go throttle is raised so a single-replica operator does not lose its leader lease under scale-subresource polling. Implements #3953. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. Two repositories are affected; both are follow-ups to open once this lands (and after its dependency #3951 merges and the values surface is final), left unticked here for a maintainer to confirm rather than claimed with a speculative link: - **cozystack/terraform-provider-cozystack** — this adds the nested `autoscaling` block to `packages/apps/postgres/values.schema.json`. The provider hand-models each app's fields (schema + model + expand/flatten), so it needs the new block to expose it in Terraform. - **cozystack/website** — KEDA is a new platform component (`guides/platform-stack`), and the postgres reference page gains the `autoscaling` params. The managed-app reference is regenerated from `README.md` by the release bot; the platform-stack component list is hand-maintained. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: to open (new platform component + postgres autoscaling params) - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: to open (new `autoscaling` block in postgres values.schema.json) ### Depends on - #3951 — bump CloudNativePG to 1.30.0. The HPA can only drive the CNPG `Cluster` scale subresource on CNPG ≥ 1.28.4, where it serves `status.selector`; without it the HPA fails with `InvalidSelector`. This PR should not merge before #3951. ### Release note ```release-note feat(postgres): add optional horizontal autoscaling of read replicas via KEDA, configured through the new `autoscaling` values block; KEDA ships as a new optional platform component. Requires CloudNativePG >= 1.28.4 for the Cluster scale subresource selector. Note: every Postgres release now applies client-side (`release.cozystack.io/helm-server-side-apply: "false"`) so the autoscaler's live `/scale` count is not reverted by Helm; a kind-wide side effect is that chart-rendered fields on Postgres releases are no longer re-asserted by server-side-apply drift correction (driftDetection is unset), a documented trade-off with a per-instance apply strategy as follow-up. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added configurable PostgreSQL read-replica autoscaling with KEDA. * Added CPU and read-connection metrics, replica bounds, replication-lag safeguards, dry-run mode, and transition handling. * Added optional KEDA platform support, Grafana autoscaling dashboard, and Prometheus alerts. * **Documentation** * Clarified integer thresholds, prerequisites, configuration, and transition guidance. * **Bug Fixes** * Improved validation and actionable errors for unavailable KEDA or invalid settings. * **Tests** * Expanded coverage for autoscaling, observability, package selection, and validation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
2055341ba9 |
fix(opensearch-operator): raise vm.max_map_count on every node from a DaemonSet (#4152)
## What this PR does OpenSearch will not start below `vm.max_map_count=262144` when the store may use mmap, and nothing in the platform sets it. The Talos profiles declare no sysctls and no documented node-preparation path sets one, so nodes sit at the kernel default of 65530. The app chart asks the operator to cover that with `spec.general.setVMMaxMapCount`, which appends a privileged init container to every OpenSearch pod. Tenant namespaces carry no PodSecurity label of their own, so they take the cluster default, which Talos ships as baseline through the apiserver's `PodSecurityConfiguration`. That default refuses the container and the StatefulSet never produces a pod. On vanilla kubeadm, kind or k3s the default is `privileged` and the same release comes up, which is why this only bites on a Talos install. The value cannot come from the pod at all. The kernel does not namespace `vm.*`, so kubelet rejects it in `securityContext.sysctls`, and refuses to start if it appears in `--allowed-unsafe-sysctls`. That leaves node configuration or a privileged pod, and a privileged pod belongs in a system namespace rather than in every tenant. This adds a DaemonSet to the `opensearch-operator` package that sets the value once per node from a privileged init container, and marks the component privileged so the platform labels `cozy-opensearch-operator` with `enforce=privileged`. The init container writes to `/proc/sys` and nothing else, with no hostPath. The pod restarts with the node and reapplies the value, so there is nothing to persist. It raises the limit to a floor rather than assigning it, which matters more than it looks. `vm.max_map_count` is node-global, and a stock Ubuntu 24.04 node already reads 1048576 from a `sysctl.d` drop-in. An unconditional write would drop that to 262144 for every workload on the node, on every boot, and undo whatever the operator set it for. So the init container reads the current value first and writes only when it is below the threshold. The idle main container runs its sleep under a shell that traps SIGTERM, so a pod exits on its own when the DaemonSet rolls or a node drains, instead of ignoring the signal and being killed after the full grace period. This does not fix OpenSearch on its own. The app chart still requests the operator's init container through `setVMMaxMapCount: true`, so a release stays blocked until the flip to `false` lands with #2682, after this merges. The DaemonSet has to be in place first: flipping it without one would trade the admission failure for a failed bootstrap check. Carrying the sysctl in the node contract instead, across the Talos profiles and the downstream node-prep paths, is tracked in #4144; that is what would let this DaemonSet be retired. Two things change on upgrade, and neither is visible from the diff. One small DaemonSet pod appears per node in `cozy-opensearch-operator`, and that namespace gains `pod-security.kubernetes.io/enforce=privileged`. That label has no finer granularity than the namespace, so the operator Deployment sitting there leaves the cluster default too, not just the DaemonSet. Keeping the manager at baseline would mean a namespace of its own and another component entry, which did not look worth it, but it is a real cost rather than an accident. The operator is in the `paas` bundle's always-on list, so this lands on every node of every paas installation and raises a node-global kernel tunable there, whether or not anyone ever creates an OpenSearchCluster. That is the price of a node-level sysctl having no cheaper carrier, and it is worth an explicit decision rather than being noticed later. The second is `cozypkg add`, which gates privileged components behind a confirmation or `--allow-privileged`. The gate runs over the resolved dependency closure, and `opensearch-application` dependsOn the operator, so installing the app that way now asks one more question. This is not a new break for scripts: `add` already prompts for a variant on every package it resolves and fails on EOF, so it was never usable non-interactively in the first place. Platform-bundle installs go through the reconciler and are unaffected. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map in `docs/agents/contributing.md` against the diff. It adds a template, a test and a values key under `packages/system/opensearch-operator/`, plus one field in `packages/core/platform/sources/opensearch-operator.yaml`. No new app package, no app schema or version enum, no `ApplicationDefinition`, no installer value, no namespace rename, no variant or bundle, no `hack/` path or make target, and no node prerequisite in `hack/e2e-prepare-cluster.bats`. Nothing in the map matches. - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(opensearch-operator): set `vm.max_map_count` on every node from a DaemonSet in the operator's namespace, so OpenSearch no longer depends on a privileged init container that PodSecurity baseline refuses in a tenant namespace. Upgrading adds one small DaemonSet pod per node and labels `cozy-opensearch-operator` with `pod-security.kubernetes.io/enforce=privileged`; installing the package with `cozypkg add` now requires confirmation or `--allow-privileged`. Existing OpenSearch releases are unchanged until a follow-up stops the app chart requesting the operator's own init container. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - OpenSearch Operator installation now runs with the required privileged configuration. - OpenSearch deployments automatically ensure nodes meet the required `vm.max_map_count` setting, improving startup reliability. - Existing higher kernel values are preserved. - Added support for configuring the sysctl image, including use with air-gapped registries. - **Tests** - Added coverage to verify privileged installation, node configuration, security settings, scheduling behavior, and image customization. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a6caad7c57 |
fix(platform): correct what disabledPackages does to ouroboros (#4098)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does `bundles.disabledPackages` does not uninstall anything. Two comments in the platform chart said it does, offered next to `kubectl delete package.cozystack.io cozystack.ouroboros` as if the two were interchangeable. A name in that list only stops `_helpers.tpl` emitting the Package document, which is the same suppression you already get from leaving `publishing.proxyProtocol` false. Every emitted Package carries `helm.sh/resource-policy: keep`, and that annotation is there precisely so disabling a package stops deleting it, so Helm leaves the Package CR alone. The CR stays, the HelmRelease it owns stays, ouroboros keeps running, and the pre-delete hook never fires because nothing uninstalls the release. Deleting the Package CR is what gets there: GC takes the HelmRelease, Flux uninstalls, the hook runs on the way out. There was a second cost. The guard in the system bundle looks up the live Package CR and never reads `disabledPackages`, so an operator who followed the old advice hit the guard anyway with no idea why. New test case pins that. Comments only in the templates, so rendered output is unchanged. Chart suite green at 156 tests. ### Screenshots Not a UI change. <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> Walked the trigger map against the diff. One repo is reached, and I left its box empty rather than tick it without a link. `cozystack/website` has the same wrong instruction verbatim in three files: `content/en/docs/next/networking/hairpin-proxy-protocol.md:122`, and the identical line under `v1.4/` and `v1.5/`. It tells the reader that removing the Package CR "(or add it to `bundles.disabledPackages`)" triggers helm uninstall and fires the pre-delete hook. It also contradicts `operations/configuration/components.md:72` on the same site, which says `disabledPackages` will not remove components that are already installed. The site fix is cozystack/website#683. It turned out to be four files rather than three: `origin/main` there also carries a `v1.6` tree, and all four copies were byte-identical. Nothing else is reached: no package added or renamed, no `values.schema.json`, no version enum, no default, no key the Ansible role sets, nothing under `hack/`. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/683 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(platform): the platform chart no longer says `bundles.disabledPackages` uninstalls a package. A name in that list only stops the Package document being rendered. `helm.sh/resource-policy: keep` leaves the existing Package CR on the cluster and the component keeps running, so deleting the Package CR is what uninstalls it. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Documentation** - Clarified proxy protocol safety guidance, including the effects of disabling the Ouroboros package and the limitations of using `bundles.disabledPackages`. - Documented that disabling package rendering does not remove an existing live package or stop its associated resources. - **Tests** - Added regression coverage confirming that disabling the package does not bypass the proxy protocol safety guard when the live package remains active. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
675163399d |
fix(platform): accept the container gpu-operator variant in iaas (#4101)
## What this PR does `bundles.iaas.gpuOperatorVariant: container` failed the whole platform render. The `cozystack.gpu-operator` PackageSource ships that variant, `values-container.yaml` configures it deliberately, and `hack/check-gpu-operator-variants.bats` covers it, but the `iaas` bundle guard only accepted `default` and `vgpu`. The exclusion made sense when it was written and stopped making sense a few minutes later. The variant and the guard came from parallel branches, neither an ancestor of the other, variant merged first. Before the guard the bundle hardcoded `default`, so `container` was selected by hand-writing a Package CR, and leaving it out of a bundle-side list cost nothing. The guard made the bundle the variant owner and closed that route without noticing it was one. Its comment calls itself typo protection and names `passthrough` as the example typo. Allowing the value is not the whole fix. The same file puts `extraFeatureGates: [HostDevices]` on the KubeVirt CR whenever gpu-operator is enabled, and builds a `permittedHostDevices` table under it. Both are VM-facing. The container variant keeps the host driver bound and pins `vfioManager` off, so no GPU reaches a VM: the gate would advertise something the node cannot serve, and the table would name resources no sandbox device plugin registers there. Both now render for `default` and `vgpu` only, and `container` gets the gpu-operator Package with the kubevirt component untouched. The helm-unittest case that pinned the rejection is gone. Two replace it: the Package renders at the container variant with no component overrides, and the KubeVirt wiring stays absent even with `gpu.permittedHostDevices` and `gpu.mediatedDevicesConfiguration` set. Comments and docs that called the mirroring unconditional on variant, or the container variant unmerged, are fixed in the same pass. No issue for this one, so the history is in the body. ### Screenshots No UI change. ### Downstream repositories Walked the trigger map against the diff. The only file it touches that appears there is `packages/core/platform/values.yaml`, and the website hit is not the hand-written key table (that page has no GPU section): it is the documented workaround this change makes obsolete. `content/en/docs/next/operations/gpu-container-workloads.md` currently tells readers the bundle rejects `container` and routes them to a hand-crafted Package CR, and `virtualization/gpu.md` / `vgpu.md` describe the KubeVirt mirroring as unconditional on variant. Filed as an issue rather than a PR because the docs must not land ahead of this change, and the `v1.6` copies describe released behaviour and stay correct as they are. Nothing else matches: no package added, renamed or removed, no `values.schema.json` or version enum touched, no `ApplicationDefinition` semantics, no installer values key, no `hack/` layout change, no platform variant or bundle recomposed. `gpuOperatorVariant` appears in none of `ccp`, `ansible-cozystack` or `terraform-provider-cozystack`. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/issues/685 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note fix(platform): accept `container` as a value for `bundles.iaas.gpuOperatorVariant`, so the gpu-operator variant the platform already ships can be selected from platform values instead of a hand-crafted Package CR. The KubeVirt `HostDevices` feature gate and `permittedHostDevices` table are now emitted for the `default` and `vgpu` variants only; `container` serves GPUs to pods and gets neither. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added the `container` GPU operator variant for workloads that do not require VM passthrough. * Container workloads no longer receive KubeVirt host-device or mediated-device GPU wiring. * Clarified GPU configuration behavior for passthrough, vGPU, SR-IOV, and container workloads. * **Documentation** * Updated passthrough guidance, configuration references, examples, and KubeVirt wiring instructions for all supported variants. * Added details for manually configured package resources and platform-specific container setup. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
a3ddba441f |
fix(opensearch-operator): set vm.max_map_count from a node DaemonSet
OpenSearch refuses to start when vm.max_map_count is below 262144 and
the store may use mmap. The kernel default is 65530 and nothing in the
platform's node configuration raises it: the Talos profiles declare no
sysctls, and neither does any documented node-preparation path.
The value cannot come from the OpenSearch pod. The kernel does not
namespace vm.*, so kubelet refuses it in a pod's securityContext.sysctls
and in --allowed-unsafe-sysctls alike, both of which admit namespaced
sysctls only. The operator's own answer, spec.general.setVMMaxMapCount,
appends a privileged init container to every OpenSearch pod. Tenant
namespaces carry no PodSecurity label of their own and take the cluster
default, which Talos ships as baseline through the apiserver's
PodSecurityConfiguration; that default rejects the container, so the
StatefulSet is created and never produces a pod.
Set the value once per node instead, from a DaemonSet in the operator's
own namespace. The init container raises the limit to a floor rather
than assigning it: the tunable is node-global, so a node already tuned
higher for another mmap-heavy workload keeps its own value. The pod runs
at system-node-critical: it is a precondition for every OpenSearch pod on
the node, so it must not lose a scheduling race to the workload it
enables, though priority is not start ordering: a pod that starts first
just fails its own bootstrap check and retries. The idle container runs
the sleep under a shell with a TERM handler, because PID 1 of a
namespace receives only the signals it installed a handler for and a
bare sleep would be SIGKILLed after the full grace period on every
rollout and drain. The write
goes to /proc/sys only, with no hostPath and no edit to the node's
on-disk configuration; the pod restarts with the node and reapplies the
value. The component is declared privileged so the platform labels that
namespace enforce=privileged, which is what this DaemonSet needs and
what tenant namespaces deliberately do not get.
The file returns to a path
|
||
|
Andrei Kvapil
|
54621bc046 |
feat(migration): tenant self-service VM import from VMware vSphere
Hidora's customers are leaving VMware now, and a tenant has to be able to import their own machines without a platform administrator. The chart approach could not express that: a tenant can neither create the Secret holding vCenter credentials nor name the VDDK image, so both had to come from somewhere they cannot reach. Two CRDs in a new `forklift.cozystack.io` group, split by lifecycle. A VMImportSource is a long-lived connection -- type, endpoint, credentials, readiness -- and a VMImportTask is the one-shot operation that names machines and produces VMDisks and VMInstances. The outputs carry no owner reference back to the task, so deleting a finished import leaves the machines alone. Konveyor Forklift does the transfer underneath; the controller renders its objects, mirrors its verdicts, and hands each finished volume into a VMDisk without copying it. What a live vCenter migration forced out, and no amount of source reading would have: - A guest copied verbatim keeps the drivers it had under VMware and no virtio, so on a virtio disk it does not fail the import -- it succeeds and then does not boot. Imported disks go on SATA. - VDDK connects straight to the ESXi host at the address vCenter advertises, which is routinely unreachable, and on the first cluster collided with the Service CIDR. `spec.hosts` redirects per host. - Firmware must be carried across or a UEFI guest imports as BIOS and never boots (stacked on #3002). - Deleting a task mid-transfer orphaned its DataVolume; a finalizer now removes the engine's volumes and leaves the outputs. The console gains a Migration section: connections are editable, and the VM field is a picker built from a machine list the controller publishes, since the aggregated API cannot reach the inventory and should not hold a source's credentials to try. Two general mechanisms fall out of that -- an option source may take an argument, and one named in a CRD annotation may reference a sibling field. Design: cozystack/community#62. Docs: cozystack/website#673. Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Aleksei Sviridkin
|
6e8566a94c |
fix(platform): accept the container gpu-operator variant in iaas
The cozystack.gpu-operator PackageSource ships three variants: default, vgpu and container. The iaas bundle's variant guard accepted only the first two, so selecting container failed the render on a value the package ships, values-container.yaml configures deliberately, and the variant bats suite already covers. The exclusion was coherent when it was written and stopped being so immediately. The container variant and the guard landed within minutes of each other from parallel branches, neither an ancestor of the other, variant first. Until the guard, the bundle hardcoded the default variant and container was selected by hand-crafting a Package CR, so leaving it out of a bundle-side list cost nothing. The guard then made the bundle the variant owner and closed that route without noticing it was one; its comment describes itself as typo protection and names passthrough as the example typo. The KubeVirt side is not a bare allowlist edit. The container variant serves GPUs to pods through the device plugin and runs no VM-facing GPU stack: sandboxWorkloads and vfioManager are both off and the host driver stays bound. Both KubeVirt-side pieces the bundle wires are VM-facing — the HostDevices feature gate advertises a capability such a node cannot serve, and permittedHostDevices lists resources no sandbox device plugin registers there. Both are now emitted only for default and vgpu, and container yields the gpu-operator Package with the kubevirt component untouched. Comments and docs that described the KubeVirt mirroring as unconditional on variant, or the container variant as unmerged and unavailable, are corrected to match. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
cd6a9b3a8f |
fix(platform): correct what disabledPackages does to ouroboros
Both the publishing.proxyProtocol comment in values.yaml and the disable-hazard guard comment in the system bundle told operators that adding a package name to bundles.disabledPackages triggers the vendored ouroboros chart's pre-delete Corefile cleanup hook, alongside an explicit `kubectl delete package.cozystack.io cozystack.ouroboros`. It does not. A name in disabledPackages only makes the rendering helper in _helpers.tpl skip emitting the Package document, which is the same suppression that leaving proxyProtocol false already performs. Every emitted Package carries helm.sh/resource-policy: keep, so Helm does not prune the Package it has stopped rendering — that annotation was added precisely to stop disabling a package from deleting it. The Package CR survives, the HelmRelease it owns via ownerReference survives with it, ouroboros keeps running and keeps re-applying its rewrite block to kube-system/coredns, and the pre-delete hook never runs because nothing uninstalls the release. No platform values setting reaches that hook; the operator has to delete the Package CR. Anyone who followed the old advice was left with ouroboros still installed and, on the proxyProtocol false transition, still facing the render guard with no idea why. Pins the behaviour with a case asserting the guard still fails while cozystack.ouroboros sits in disabledPackages and the Package CR is live. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
6935df8c7f |
chore(kubevirt): update KubeVirt to v1.9.0 (#4015)
## What this PR does KubeVirt operator manifest goes from v1.8.4 to v1.9.0. The 1.9 release turns every Beta feature gate on by default and adds `disabledFeatureGates` to the CR for opting out, so the CR side needed a few decisions. `Template` is disabled. With it on, virt-operator deploys virt-template: two Deployments, two APIServices, a webhook and a ValidatingAdmissionPolicy that nothing here uses. One more aggregated API server is one more way to break discovery. This one is not a knob. `- Template` is written into the CR unconditionally and there is no value that takes it back out, because nothing on the platform uses VirtualMachineTemplate; if something starts to, the static entry becomes a value then. Asking for it through `extraFeatureGates` is refused at render time with an error that says so, rather than producing a CR KubeVirt would reject. `ImageVolume` is the host-dependent one: off below Kubernetes 1.35, left on from 1.35, with a platform value for runtimes that cannot serve image volumes even there (#3190 has the k3s case). Nothing the platform ships uses containerDisk. The value is `kubevirt.disabledFeatureGates` on the platform values, which the iaas bundle forwards into the emitted `cozystack.kubevirt` Package as `components.kubevirt.values.disabledFeatureGates`: Closes #3190. ```yaml kubevirt: disabledFeatureGates: - ImageVolume ``` The chart refuses to render a CR that names one gate in both lists. KubeVirt's `kubevirt-update-validator.kubevirt.io` webhook rejects that combination and runs with `failurePolicy: Fail`, so on a cluster that already has the CR the write is refused and the `kubevirt` HelmRelease stops converging with everything else the CR carries. The create admitter runs no validation at all (its own comment says the update validator is where that happens, and it only gets installed after the CR exists), and `IsEnabled` in virt-config checks `FeatureGates` before `DisabledFeatureGates`, so on a fresh install the same CR is admitted and the gate silently stays on. Render-time is the only place left where an operator gets told. A name repeated inside one list is not an error: upstream compares the two lists against each other, and both fields are `x-kubernetes-list-type: atomic` in the CRD, so that CR is accepted and the guard stays quiet. The gates the chart pins off can arrive a second time from a supported values file, since the chart writes them itself and an operator listing the gates their cluster declines has no reason to know that, so the range skips them and the CR states each opt-out once. The guard walks its own copy of the seven gates the chart enables. That copy exists because the rendered list has to stay a literal list for `make update` to find its `- VMExport` anchor, and a gate quietly dropped from the copy would keep rendering under `featureGates` while leaving the guard. The suite carries one case per gate, so removing any single name turns exactly one of the seven red. `HotplugVolumes` stays in the list even though 1.9 deprecates it. A deprecated gate listed in the CR is still enabled, and while it is enabled virt-controller skips the declarative hotplug path. Dropping it would switch on declarative hotplug, but then virt-api rejects the VMI-level `removevolume` subresource on VM-owned VMIs, and kubevirt-csi uses that as a fallback during teardown. The CSI driver has to move off that subresource first. Upstream names no removal release for it: the `Deprecated` state's own doc comment says "discontinued in the following release", but `MultiArchitecture` and `DisableMDEVConfiguration` have been deprecated since 1.8.0 and are still registered at 1.9.0, so that is a direction rather than a date. What makes that opt-out list matter is a semantics change: on 1.8 `isFeatureGateEnabled` ended in `return false`, so a Beta gate nobody listed was off; on 1.9 `IsEnabled` ends in `return fg.State == Beta`. Sixteen gates are Beta at v1.9.0 and none of them is in this chart's `featureGates`, so all sixteen arrive on with the bump. Two are switched back off here, one is switched off below Kubernetes 1.35, and I went through the rest: - `Template`: off. Reason above. - `ExternalNetResourceInjection`: off. With it on, virt-controller stops calling `multus.NetworkToResource`, so a NAD's `k8s.v1.cni.cncf.io/resourceName` no longer becomes a resource request on the launcher pod. That is correct for clusters running a network-resources-injector; this platform ships none, so a VM on an SR-IOV NAD would schedule without its device. - `ImageVolume`: off below Kubernetes 1.35, left alone from 1.35 on. KubeVirt renders containerDisk images as `VolumeSource.Image` volumes when the gate is on, and it does not check the host: `dropImageVolumes` in the apiserver strips every image volume and the mounts referencing it while Kubernetes' own `ImageVolume` gate is off, which is the default until 1.35 (`kube_features.go` has it `Default: false` at 1.31 and 1.33). The pod is admitted without the disk and virt-launcher fails looking for it. The management cluster is supported from 1.33, so on 1.33 and 1.34 the chart keeps the KubeVirt gate off and the containerDisk sidecar path in use; from 1.35 the gate stands and `extraFeatureGates` can name it. The apiserver version is a proxy for a runtime capability, not the capability itself, so a cluster at 1.35 or newer whose container runtime cannot serve image volumes (the k3s case in #3190) switches the gate off through `kubevirt.disabledFeatureGates`; that is what the value is for. - `Snapshot`: on. `VirtualMachineSnapshot` needed the gate listed on 1.8; snapshot and export were exercised on the test cluster. - `DeclarativeHotplugVolumes`: on and inert. `handleDeclarativeVolumeHotplug` returns early while `HotplugVolumes` is enabled, which this chart still does. - `GPUsWithDRA`, `HostDevicesWithDRA`: on; each only lets a VMI request a DRA-provisioned device, and nothing here creates one. - `WorkloadEncryptionSEV`: on; needs SEV hardware and a VM that asks for it. - `PasstBinding`: on; needs a VM using the passt binding. - `RebootPolicy`: on; adds a field on the VMI domain spec, unset means the old behaviour. - `VmiMemoryOverheadReport`: on; fills `status.memory.memoryOverhead`, reporting only. - `OptOutRoleAggregation`: on; it enables the `RoleAggregationStrategy` field, and aggregation stops only for a CR that sets it to `Manual`. This one does not. - `KubevirtSeccompProfile`: on; installs the profile, and a workload uses it only where `seccompConfiguration` says so. - `NodeRestriction`: on; virt-handler node authorization, needs the Kubernetes `ServiceAccountTokenPodNodeInfo` gate, GA since 1.32. - `LibvirtHooksServerAndClient`: on; pre-migration hooks on the target launcher, and no hooks ship here. - `PodSecondaryInterfaceNamingUpgrade`: flips on; it adds `--upgrade-ordinal-ifaces` to virt-launcher and a pre-migration hook that renames the ordinal secondary interfaces of VMs still carrying them. The test cluster's migrations exercised that path. Five of the seven gates the chart enables (`ExpandDisks`, `LiveMigration`, `AutoResourceLimitsGate`, `GPU`, `VMExport`) reached GA by 1.9 and are inert: `IsEnabled` returns true for a GA gate whether or not the CR names it. I left the list as it is; dropping the five is a follow-up, because it changes what the CR says without changing what the cluster does and it belongs in its own commit. QEMU is 10.1.0 on both sides of this jump; the el9 build revision moves from -10 to -20 and libvirt goes 11.9 to 11.10. A moving build revision is the thing to check rather than a major: [kubevirt/kubevirt#16386](https://github.com/kubevirt/kubevirt/issues/16386), where every existing VM lost live migration, was 1.6.2 to 1.7.0 on qemu-kvm 9.1.0-19 to 9.1.0-20, one build apart within the same QEMU version. So I tested it: VMs running on the 1.8.4 launcher live-migrated onto the 1.9.0 one on a three-node cluster. `make update` in `kubevirt-operator` used to `rm -rf templates` before the fetch and took the local PrometheusRule template with it. Now it only overwrites the file it downloads. `make update` in `kubevirt` now greps for every line of the new block before it passes: the range directive, the `disabledFeatureGates:` key, the three static entries, and the guard header that decides which static entries render. Every insert in that target is guarded by one directive it also inserts, so a hand-merge that keeps the directive and drops a sibling line leaves the guard satisfied and the insert skipped. That shape covers the directive and nothing else. What makes it worth a grep per line here is the static entries: they are the only lines those inserts add that are CR content in their own right rather than a wrapper around a value, so they render on every install and their absence cannot surface as a value that stopped taking effect. The CR just goes back to deploying virt-template and to letting KubeVirt skip NAD resource injection. The target cannot rebuild a partial block, so it names the missing line and stops. The payload lines of the other four blocks are exposed the same way; I checked, and both dropping `- {{ . }}` from the `extraFeatureGates` range and dropping the `permittedHostDevices:` key from its `with` render green and lose the operator's values. That is a wider change than this PR and I left it out. Two sharp edges I know about and left. `kubevirt.disabledFeatureGates: [HostDevices]` on a cluster that also has `cozystack.gpu-operator` in `enabledPackages` makes the bundle forward the same gate into both lists, and the platform renders fine while the child kubevirt HelmRelease then stops on the guard: fail-closed, with the gate named, but diagnosed one level below where it was configured. And the version-conditional opt-out is evaluated at render time, so a cluster upgraded from 1.34 to 1.35 keeps `ImageVolume` disabled until something re-renders the chart; the direction is the safe one, since the gate stays off and containerDisk keeps the path it had on 1.8.4. Two things I decided against doing here. The guard's copy of the gate list is pinned against a name being dropped, not against a name being added to the rendered list only; the copy exists because `make update` needs the literal `- VMExport` anchor, so closing the other direction means changing the self-heal contract. And `hack/run-kubernetes-block-io_test.bats` still names a `virt-launcher:v1.8.4` image in a fixture: inert, the test matches on the container name, and the file is outside this change. Tested on a three-node Talos v1.13.6 / Kubernetes v1.36.2 cluster with Flux suspended. The upgrade took about ten minutes. virt-operator had already deployed virt-template before the CR change landed and removed it afterwards, CRDs included. Two VMs started on the 1.8.4 launcher, one DataVolume-backed and one containerDisk-backed, live-migrated to the 1.9.0 launcher. VMDisk plus VMInstance through the apps API boots. `addvolume` on the VM subresource reaches Ready, `removevolume` on the VMI subresource is accepted. VirtualMachineSnapshot and VirtualMachineExport from it reach Ready. Restore from snapshot and PVC clone failed on that cluster inside the LINSTOR API layer, unrelated to this change. CDI v1.66.0 from #3920 on top of it: HTTP imports to Block and Filesystem, blank population and hotplug all work. `helm unittest` is 26/26 for the kubevirt chart and 142/142 for the platform chart, and the update idempotency script passes its eleven cases. ### Screenshots Not a UI change. ### Downstream repositories Walked the trigger map against the diff. No app added or renamed, no variant, release asset, `ApplicationDefinition` or tooling contract touched, and none of the `packages/core/platform/values.yaml` keys the ansible role sets are in the diff. One repository is reached. `packages/core/platform/values.yaml` gains a top-level key, `kubevirt.disabledFeatureGates`, defaulting to the empty list, and `operations/configuration/platform-package.md` on the website is a hand-written table of `spec.components.platform.values.*` that nothing generates. The row is in cozystack/website#678, which lands after this PR merges. - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/678 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note chore(kubevirt): update KubeVirt to v1.9.0. Beta feature gates are on by default upstream now. The platform keeps `Template` off, which would deploy virt-template, and `ExternalNetResourceInjection`, which would hand `resourceName` injection for NetworkAttachmentDefinitions to an injector the platform does not run. `ImageVolume` is disabled on management clusters below Kubernetes 1.35: containerDisk-backed VMs are rendered with native image volumes when it is on, and an apiserver whose own ImageVolume gate is off, the default before 1.35, drops those volumes silently, leaving the VM without its disk. On 1.35 and newer the gate stands, and that is the first release where containerDisk-backed VMs are rendered with image volumes instead of sidecars; the bundled Talos ships Kubernetes 1.36, so this is the default there. A `kubevirt.disabledFeatureGates` list in the platform values opts out of any other gate a cluster cannot carry. ``` |
||
|
Alexey Artamonov
|
daf733ad9c |
fix(keda): narrow autoscaler RBAC and scope the operator-down alert
Restrict the KEDA operator ClusterRole: set permissions.operator.restrict.secret so it drops cluster-wide secret list/watch, and pin rbac.scaledRefKinds to postgresql.cnpg.io/clusters so it no longer holds a wildcard */scale across every group. Gate DatabaseAutoscalerKedaOperatorDown on the presence of keda-hpa-* HPA series so a tenant running its own Monitoring app (which never scrapes keda_build_info, root-only) does not fire it forever about a healthy platform component. Make the two negative containsDocument matchers in the bundles-default test non-vacuous (any: true) so "KEDA is not installed by default" is actually asserted. Drop the unused pollingInterval/cooldownPeriod params from the cozy-lib keda helper. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
42a533fc50 |
fix(platform): inject networking.clusterDomain into the KEDA package
KEDA's clusterDomain is a vendored-subchart value baked into the metrics-apiserver gRPC target (keda-operator.<ns>.svc.<clusterDomain>:9666) and the operator's serving-cert SANs at render time. A chart template cannot derive a subchart value from _cluster, so the keda package hardcoded cozy.local — which silently breaks (NXDOMAIN, every DB HPA FailedGetExternalMetric) on any cluster whose domain is not the default. Inject the authoritative networking.clusterDomain into the shared cozystack-values Secret under the keda subchart key so the value tracks the platform config; the keda package keeps cozy.local as a standalone fallback. The key is ignored by packages without a keda subchart (no app/extra schema sets root additionalProperties:false). Add platform unittests asserting the Secret carries keda.clusterDomain and that it tracks a non-default networking.clusterDomain. Addresses the hardcoded-clusterDomain portability gap found in branch review. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Alexey Artamonov
|
7f8f2c1d86 |
feat(keda): add KEDA as an optional platform component
Vendor the upstream KEDA chart (kedacore/keda 2.20.2) as a new optional system package and wire it into the platform, so database read-replica autoscaling (which renders KEDA ScaledObjects) has a backend. - packages/system/keda: umbrella cozy-keda + vendored chart (Makefile pins KEDA_CHART_VERSION with a lockstep assert), shim values enabling the operator / metrics-server / webhook ServiceMonitors under the platform Prometheus. - core/platform: PackageSource cozystack.keda (namespace cozy-keda, dependsOn networking + prometheus-operator-crds; cert-manager is not needed, KEDA self-generates its webhook TLS) plus an optional bundle entry, opt-in via bundles.enabledPackages. It is not a dependsOn of any default package, so the platform HelmRelease never blocks on it. - KEDA claims the cluster-wide external.metrics.k8s.io APIService (distinct from metrics-server's metrics.k8s.io — no conflict). Covered by helm unittest: platform bundle enable/disable/opt-out, the PackageSource dependsOn, and the vendored chart's external-metrics APIService + shim-enabled ServiceMonitor. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
3a369fed99 |
feat(redis): add opt-in TLS for Redis and Sentinel (#2729)
## What this PR does `tls.enabled` puts Redis and Sentinel on a TLS-only listener with a per-release cert-manager chain, and hands the tenant the CA certificate as `redis-<name>.tenant-ca`. Off unless set. redis-operator v3.3.5 has no TLS support. The work is ported upstream in freshworks-oss/redis-operator#85, which is open and not in a release, so this carries it as three patches over the v3.3.5 tarball. `images/redis-operator/patches/tls.diff` is the source half, cut from that PR's commit `2ad2ed8d1`, which the patch header pins; it is applied by the glob the image Dockerfile already runs, after the Dockerfile strips the tarball's `manifests`, `charts`, `example` and `.github`, so the image builds from source alone and the upstream tests that read the CRD YAML skip instead of failing against the stale v3.3.5 copy. `patches/crd-tls.diff` adds `spec.tls` to the vendored CRD, and with it the CEL rule that fixes `tls.enabled` at creation; without the schema block the apiserver prunes the field on admission and the patched binary waits for something it never sees, and the rule cannot exist without the block it references. `patches/rbac-tls.diff` grants Create, Get and Update on Secrets, so the operator can publish the CA-only Secret it addresses by name. That is the whole delta, and it is narrower than the ported operator's own chart asks for: those three are the only Secret calls in the patched source, and cert-manager Certificates get no grant at all, because the operator only touches one under `spec.tls.certManager` and this chart pins `certificateSecret` mode instead. One change in `tls.diff` is not TLS and takes effect with TLS off, named in the patch header: `MakeSlaveOfWithPort` connects to the replica on its own port rather than on the master's. It changes a dial, not a pod template, so shipping the rebuilt image rolls nothing; the plaintext pod templates stay byte-identical to v3.3.5. The image build registration this branch used to carry (`make -C packages/system/redis-operator image` in the root `build:` target) landed on main with the Valkey service (#3380), so the rebuilt operator image is produced by CI from main as well; the digest in `values.yaml` is still refreshed by release preparation, not here. The chart patches re-apply at the end of `update:`, the same way reloader, grafana-operator, ingress-nginx, seaweedfs and kamaji carry theirs. Otherwise `rm -rf charts` takes TLS back out on the next `make update` and nothing fails. `make update` reproduces the committed tree byte for byte. TLS is chosen when the instance is created. The config gets `port 0` and the listener moves to `tls-port`, so TLS replaces plaintext rather than running beside it, and a running instance cannot change mode: a replica restarted onto the TLS listener cannot replicate from a master still on plaintext, and Sentinels already on TLS would fail over to a stale replica. The RedisFailover schema in the carried CRD patch rejects a change of `tls.enabled`, which on an existing instance surfaces as a failed HelmRelease upgrade while the instance keeps serving what it did; the README points at creating a new instance and migrating. That is also why TLS is not inferred from `external`: an externally reachable instance is the one most likely to have clients an upgrade must not drop. Existing releases keep their plaintext listener and are not restarted by the rebuilt operator image. The tenant gets `ca.crt` and nothing else, through `tenantsecrets`. The chart's own Secrets take a dot between the release and the suffix (`<release>.ca-tls`, `<release>.tls`, `<release>.ca-cert`): the suffixes are dot-free, so the split is unique and release `foo`'s CA Secret can no longer be release `foo-ca`'s leaf, the collision the hyphenated form had. The chart renders a `TenantProjection` pointing at `<release>.ca-tls`, and the CA-extraction controller re-encodes the one named key from the parsed certificate, so nothing that is not a certificate gets through. Same path as postgres, mongodb, nats and qdrant. No Secret of this release is granted to the tenant by name beyond the generated password, and that includes `<release>.ca-cert`, the CA-only copy the operator publishes for itself. It holds no key today, but the operator has Create and Update on Secrets cluster-wide, so what sits at that name is the operator's call rather than the platform's, and RBAC cannot filter by key. `hack/check-redis-rd-secrets.bats` pins the whole rendered Secret surface instead of a list of known-bad names, because an include with no `resourceNames` and no `matchLabels` restricts nothing. The chainsaw suite for redis gains two cases. A TLS fixture asserts the cert-manager chain is issued, the TLS-probed Redis replicas and Sentinel come up, the operator publishes its CA-only Secret under the widened ClusterRole, the tenant receives exactly `ca.crt` through `tenantsecrets` while the key-bearing Secrets stay off that path, and a `redis-cli --tls` PING through the service name completes with the same CA. A second case applies `tls.enabled` to a running plaintext release and asserts the refusal: the HelmRelease upgrade fails with the schema's message, the StatefulSet keeps its revision with both replicas Ready, plaintext still answers and no TLS listener appears. Certificate renewal rolls the pods from inside the operator. Redis and Sentinel read the certificate once at startup, so the patched operator stamps a hash of the TLS Secret's `tls.crt` and `ca.crt` into the pod template of both the Redis StatefulSet and the Sentinel Deployment on every reconcile, as `redis-failover.freshworks.com/tls-secret-hash`. A renewal changes the hash; the operator's own roller replaces the Redis pods one at a time and the Deployment controller rolls Sentinel. README names the fallback: if the roll cannot run, the deadline is the old certificate's expiry, past which the operator's own TLS connections fail along with the clients. `tls.authClients` maps to `tls-auth-clients` and defaults to `no`. The platform mints no client certificate for the tenant and the CA key is out of reach, so `yes` is only usable when the client certificate comes from whoever also controls the CA. Changing the value on a running release restarts the Redis and Sentinel pods: Redis reads the directive from its config file only at startup, so the operator folds the effective value into the certificate hash that drives the rollout, and the change lands when it is made rather than at the next unrelated restart. Follow-ups in the carried operator patch go upstream first, to freshworks-oss/redis-operator#85 or a successor there, and are picked up here afterwards; the carried patch does not grow Go that upstream has not seen. `tlsConfigFor` fetches the Secret on every call, tens of GETs per reconcile. The renewal roll is static analysis so far: the live check, `cmctl renew` on a running release while watching the `rfr-*` and `rfs-*` pods roll, is still owed and belongs with the e2e follow-up. Two facts about the shipped image rather than the patch. `packages/system/redis-operator/values.yaml` keeps the digest of the image built before the patch until release preparation rebuilds it, so on `main` between this merge and the next release-prep, `tls.enabled` renders the chain and `spec.tls` while the running operator ignores them and serves plaintext. And the pinned v3.3.5 tarball carries GO-2026-5026 in `golang.org/x/net` v0.53.0, reached through `PodService.UpdatePodLabels`; the port neither introduces nor fixes it, upstream `master` is already on v0.55.0, so it lands with the next tag bump rather than here. Outside the patch: `cozyvalues-gen` renders `{}` in the README Value column for `tls.authClients`: a child property without a default inherits the parent object's `{}`, and `make generate` reproduces it byte for byte, so it is a generator bug rather than something to hand-edit here. ### Screenshots No UI changes. ### Downstream repositories - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [x] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: cozystack/terraform-provider-cozystack#37 - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: I walked the trigger map against the diff file by file. terraform-provider-cozystack is reached, tracked as cozystack/terraform-provider-cozystack#37 rather than a speculative PR: `values.schema.json` gains a `tls` object, and the provider hand-writes a schema, model and expand/flatten pair per Kind. It already models `tls` for postgres, kafka, nats and qdrant, not for redis. Redis needs different wording from those four, whose descriptions say to omit the block to follow `external`, which is the one thing redis does not do. Nothing else is reached. The website regenerates this package's reference page from `README.md` on a stable tag, so the README changes need no PR there. No package added, renamed or removed. `packages/core/platform/values.yaml`, variant and bundle composition and release asset names are untouched, and so are `hack/package.mk`, `hack/common-envs.mk` and `hack/update-crd.sh`, which is what ccp and external-apps-example read. The two new files under `hack/` are bats unit tests. No node prerequisite, installer value, telemetry metric or cozy-proxy label changes. ### Release note ```release-note feat(redis): add opt-in TLS for Redis and Sentinel. Set `tls.enabled: true` to serve both on a TLS-only listener backed by a per-release cert-manager chain; the tenant reads the CA certificate as the `<release>.tenant-ca` tenant secret. TLS is off unless set and is not inferred from `external`, so existing releases keep serving plaintext and are not restarted by the rebuilt operator image. TLS is chosen at creation: a running instance cannot switch modes, the change is rejected and surfaces as a failed HelmRelease upgrade, so add TLS by creating a new instance and migrating to it. Requires cert-manager, which the platform's `system` bundle installs on every variant. A renewed certificate rolls the Redis and Sentinel pods through the operator. TLS support is carried as a downstream patch over redis-operator v3.3.5 until freshworks-oss/redis-operator#85 lands in a release. ``` |
||
|
Aleksei Sviridkin
|
8adf3a0863 |
feat(platform): forward kubevirt.disabledFeatureGates to the kubevirt CR
The kubevirt chart reads disabledFeatureGates from its own values, and the iaas bundle is the only writer of the cozystack.kubevirt Package, so nothing could set it: a Package patched by hand is reverted the next time the platform re-renders it. Platform values gain a documented kubevirt.disabledFeatureGates list, forwarded into the emitted Package as components.kubevirt.values.disabledFeatureGates alongside the entries derived from resources.cpuAllocationRatio and the gpu block. The parenthesized value walk is nil-safe, matching the gpu block, so a cluster that nulls out the whole kubevirt key still renders. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Aleksei Sviridkin
|
5976ddf02f |
fix(gateway): normalise the GatewayClass name and refuse a blank one
The class lists arrive over the values channel comma-joined, so every reader trims each entry as it splits them. The scalar those entries are compared against was not trimmed anywhere. A class name carrying a stray space therefore failed `has` against its own padded self, and the two readers of that scalar disagreed about the same class: the gateway chart put the tenant on an edge-terminated class, while the cluster-wide ACME issuers still pinned their solver to a listener an edge Gateway does not render. That stops every HTTP-01 certificate in the cluster from issuing, with nothing in any status naming the cause. Normalise once at the single writer so both readers see a clean value, and keep a defensive trim at each reader. The tenant's own choice does not travel that channel — it comes from tenant.spec.gatewayClass — so it is trimmed where the gateway chart reads it. Unset and whitespace-only are different events and no longer share an outcome. Unset means the choice was not made, and falling back to the bundled class stays its documented behaviour. Whitespace-only means a choice was made and is garbage, in the field that decides where TLS terminates: resolving it quietly would take a tenant the operator meant to put behind an edge terminator and publish it directly, with a real certificate on a real address. Both the platform default and the tenant's own choice now refuse it by name. The kind check beside that refusal is part of the same fix, not a tidy up. Through Helm's values path an unquoted numeric scalar arrives as a float64, and the whitespace check compares it against a string, so `className: 123` dies at that comparison — one line before the trim — with an error naming neither the key nor the file. Refusing whitespace without checking the kind first would therefore have made the numeric case worse than it was. `kindIs "string"` is the right test because it never has to name the type that actually arrived. The order of the guard's steps is part of the guard. The value is read before anything normalises it, so a default that fires on any falsy value cannot launder 0 or false into the empty string and resolve them to the bundled class ahead of the kind check. Unset is the one case that still falls back, and each of those four inputs is pinned, so a normalisation drifting back ahead of a check fails rather than quietly covering only the loudest bad input. Also correct a cleanup log line: the wildcard Certificate is deleted for every mode that is not DNS-01, so naming HTTP-01 in that message is wrong for existingSecret and for edge alike. And state the listener name invariant plainly: the cap is 253, not the 63 of a DNS label, and the longest name the builders produce is 84. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
487cafac3e |
feat(platform): let a tenant pick its Gateway class from an allowlist
Tenant Gateways were pinned to the bundled Cilium class. Give the platform a default in gateway.className, the set a tenant may choose from in gateway.tenantSelectableClasses, and the set whose provider terminates TLS upstream of the Gateway in gateway.edgeTerminatedClasses. A tenant picks its own with tenant.spec.gatewayClass, read only when it owns a Gateway; an inheriting tenant runs on its ancestor's. The selectable set is an allowlist rather than free choice because Tenant is a tenant-writable object while a GatewayClass is cluster-scoped: without it a tenant could attach its Gateway to a class an admin keeps for internal-only traffic. Naming anything outside the set fails that tenant's own gateway release and reaches no other tenant. Where TLS ends is a property of the class, so the gateway chart derives certMode from whichever class the tenant landed on instead of taking it as a separate input. The two therefore cannot be configured out of step. The publishing tenant cannot cross that line by itself. The cluster-wide ACME issuers point their HTTP-01 solver at its Gateway and are rendered from the platform default alone, so a mismatch would pin the solver to a listener that Gateway does not have. Those issuers now drop the sectionName pin when the default class is edge-terminated, letting the challenge attach by hostname instead. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
f5206a6e28 |
docs(redis): declare the TLS-on-upgrade break and fix the parameter sections
TLS replaces plaintext in this operator rather than running beside it — with TLS on, Redis and Sentinel are configured with port 0 and serve only on the TLS port. Since tls.enabled falls back to external, an already deployed instance with external: true switches to TLS-only on its first reconcile after the upgrade and every plaintext client breaks. That was documented as a feature and nowhere as a change of behaviour for existing instances. State it as a breaking change and name tls.enabled: false as the opt-out. Also spell out which hostname the certificate actually covers. The SAN list carries the four service names and the external tenant hostname, and its only IP entries are the loopback addresses used by the in-pod probes and the metrics sidecar, so connecting to the external LoadBalancer by IP fails for clients that verify the hostname. The TLS section marker sat above the version parameter, which put version in the TLS table in the generated README instead of Common parameters. Move the marker below it. Regenerating also reorders version ahead of tls in keysOrder, matching how postgres orders the same two fields. Pin the PackageSource CRD replacement policy the way the mongodb-operator one is pinned. The RedisFailover CRD gained spec.tls with the fork, and Helm never upgrades CRDs shipped in crds/, so losing CreateReplace would leave the live CRD behind and the apiserver would strip spec.tls from every RedisFailover the chart renders — TLS silently off, no error. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Timofei Larkin
|
f5b957ee27 |
feat(kubernetes)!: manage worker pools as KubernetesNodes, adopt on upgrade (Phase 2, breaking) (#3315)
## What this PR does Phase 2 of the Kubernetes-app split ([cozystack/community#8](https://github.com/cozystack/community/pull/8)) — the **breaking** half. Stacked on #3314 (review that first). Removes worker node pools from the `kubernetes` chart (they moved to `kubernetes-nodes` in #3314) and adopts existing pools on upgrade: - `kubernetes` chart is now **control-plane only**: drops the pool `range` block, the `talos-reconcile` Job + RBAC (deleted; moved to `kubernetes-nodes`), the preserve-old-KMT block, the `kubernetes.nodeGroups` helper, and the per-group dashboard RBAC. - **`spec.nodeGroups` is removed from the `Kubernetes` CR** (BREAKING). Worker pools are now separate `KubernetesNodes` resources. - **Migration 54** (bumps `targetVersion` to 55): pre-upgrade hook that, for every `kubernetes-<cluster>` release, creates one `kubernetes-nodes-<cluster>-<pool>` HelmRelease and re-annotates the existing worker objects onto it with `helm.sh/resource-policy=keep`, so the now control-plane-only parent upgrade does not prune running workers. The default `md0` (implicit when `nodeGroups` was empty) is materialised explicitly. Idempotent. Genuine infrastructure/read failures fail closed (`exit 1`, retried on the next upgrade); hostile stored shapes (a pool name over Helm's 53-char release-name cap, an RFC-1123-invalid or whitespace/slash map key, a non-object `nodeGroups`) are warned, pinned, and skipped rather than deadlocking the entire fleet's upgrade. - The `ingressNginx` addon no longer auto-provisions a node (the implicit `md0` default is gone). If enabled without a pool carrying `roles: [ingress-nginx]`, the controller DaemonSet simply schedules nothing (degraded, not failed) — the requirement is documented in the chart NOTES.txt rather than enforced at render time. Because the child chart renders the pool objects **byte-identically** to the old parent render (golden parity in #3314; the KubevirtMachineTemplate content-hash name is preserved), adoption does not churn live worker VMs. ### Migration number `54` is contiguous with main (latest is `53`) and `targetVersion` is `55`. The `52`/#3201 collision noted in earlier revisions has been resolved: this branch was rebased and merged past it, so the numbering is contiguous and `run-migrations.sh` has no gap to hard-fail on. ### Not yet done (blocking draft → ready) - [ ] **dev e2e upgrade validation** (`cozystack-pr-test`): upgrade a custom-`nodeGroups` cluster and a default-`md0` cluster; assert zero worker-VM churn (KMT hash, Machine uid/creationTimestamp stable) and that the adopted child HRs reconcile. - [ ] downstream trigger-map walk (website docs for the `nodeGroups` → `KubernetesNodes` change; terraform-provider for the removed API field). ### Downstream repositories Deliberately unticked (draft) — the breaking API change affects docs and the terraform provider; follow-ups to be filed before ready. - [ ] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: ```release-note Worker node pools of a managed Kubernetes cluster are now managed as separate `KubernetesNodes` resources. `spec.nodeGroups` on the `Kubernetes` CR is removed; existing pools are adopted automatically on upgrade. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Worker pools are now separate `KubernetesNodes` resources with their own lifecycle. * Added a render-time compatibility guard to prevent worker pools from running ahead of the parent control-plane minor version. * Improved upgrade/uninstall behavior to better preserve already-adopted worker objects. * **Breaking Changes** * Removed per-pool `nodeGroups` and worker node health settings from the `Kubernetes` chart/CR; pool configuration now lives in `KubernetesNodes`. * Ingress requires at least one `KubernetesNodes` pool with `roles: [ingress-nginx]`. * `cluster` and `storageClass` are immutable after creation; pool `version` may lag but must not be ahead of the parent minor. * **Documentation** * Updated READMEs, notes, and schemas to reflect the `KubernetesNodes` split and the immutability/version rules. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
IvanHunters
|
d93e136e23 |
fix(computeplane): provision worker pools via kubernetes-nodes after Phase 2 split
Phase 2 moved worker nodeGroups out of apps/kubernetes into apps/kubernetes-nodes, but the ComputePlane module still passed nodeGroups to the wrapped kubernetes chart, which now ignores it: a fresh ComputePlane cluster came up with zero workers (and the pinned ingress-nginx addon stuck Pending), while existing ones were unreachable by migration 56 (it only adopts kubernetes-* releases, not computeplane-cluster). The module now emits one kubernetes-nodes child HelmRelease per nodeGroup, shaped like a natively created pool, and materialises the pre-split default md0 (roles: [ingress-nginx]) when nodeGroups is empty. The pool chart's clusterName gains an internal clusterReleaseName override so pools attach to the module's fixed computeplane-cluster release, which is off the aggregated API's kubernetes-<cluster> convention; the default keeps every API-created pool byte-identical. Signed-off-by: IvanHunters <xorokhotnikov@gmail.com> |
||
|
Aleksei Sviridkin
|
9ccb133b66 |
fix(ci): de-escalate select-e2e and name every escalation (#3817)
## Why
I replayed the 150 most recently merged pull requests through
`hack/select-e2e.sh` with every escalation branch instrumented. 118 of
them (78.7%) ran the full 21-suite Chainsaw run, and the causes, counted
once per escalating PR with the count where that cause was the only one
in brackets, were `FULL_PATTERN` 88 [68], `NO_SUITE_FOR_GROUP` 24 [24],
`CHAINSAW_SHARED` 18 [1] and `UNCLASSIFIED` 7 [4]. `YQ_BROKEN`,
`NO_GRAPH_OWNER` and `BACKSTOP_EMPTY` never fired.
Two problems come out of that. Most of the escalation is real but
unnecessary. Whole classes of path that provably cannot affect a
Chainsaw suite were escalating because no rule claimed them. And the
commonest cause of a full run was absent from the log, so "why did this
pull request run everything" could only be answered by re-deriving the
selection by hand.
## What changes
**Every escalation now names its cause on stderr.** Seven branches could
reach the full suite and four did it in silence, `full_suite_pattern`
among them, the commonest cause by a wide margin, so the usual answer
was the one the log never carried. The lines go to stderr and must stay
there: stdout is the suite list and both e2e lanes parse it, so a reason
line on stdout would be read as a suite name. A unit test asserts every
message, verified by muting each of the eleven in turn and confirming
the file goes red for all of them. A reason that regresses to silence
changes no selection and is otherwise invisible.
**`cozystack.etcd-application` gains its missing `dependsOn:
cozystack.etcd-operator`.** `packages/extra/etcd` renders `kind:
EtcdCluster` from `etcd-operator.cozystack.io/v1alpha2`, and this was
the only operator-backed application source in the tree with no edge to
its operator. It is a latent install bug as much as a selection gap:
without the edge `etcd-rd` registers the ApplicationDefinition as soon
as the engine is up, so a tenant can create an `Etcd` before the
operator exists and its HelmRelease fails on `no matches for kind
"EtcdCluster"`, and `hack/select-install.sh`'s forward closure for the
`etcd` suite omitted the operator for the same reason. On the selection
side `cozystack.etcd-operator` reached no runnable suite, so every
change to the operator ran all 21.
**`vm-disk-application` maps to the `vminstance` suite.** It resolved to
a `vm-disk` suite that does not exist, `intersect_suites()` dropped the
name, and the source reached nothing runnable. Neither did
`cozystack.kubevirt-cdi` above it, whose only other dependents are
`vm-default-images` and `vm-disk-application` itself, so every CDI
change ran all 21. The coverage was there and the table did not know it:
the `vminstance` suite creates a `VMDisk` and asserts the `DataVolume`
behind it (`hack/e2e-chainsaw/vminstance/vmdisk.yaml`,
`vmdisk-vmi.yaml`). Same defect class as the etcd edge above, on the
mapping side rather than the graph side, and with no production blast
radius of its own, since `src_to_suites` is read by the selector and by
nothing else.
**`cozystack.cozystack-basics` joins `cozystack.cozystack-engine` as a
propagation hub excluded from the reverse-dependency walk.** Every edge
into it exists so a namespace or a platform-wide policy is in place
before the dependent installs, which `kubevirt-cdi` states in its own
source ("Depend on cozystack-basics so the target namespace exists
first"), and that is install ordering rather than behaviour. The damage
runs opposite to the engine's: the engine fans one change out to every
app, basics narrows instead. It reaches no suite today so it escalates,
but it sits upstream of `kubevirt-cdi`, so the first suite to land under
CDI silently converts the platform's namespace-and-policy package from
the full run to that one suite. Not hypothetical — on #3426 enabling the
site-router suite takes `cozystack-basics` from 22 suites to
`site-router` alone, and nothing in the output says coverage was lost.
Dropping the reverse edges keeps the package reachable, stops it
propagating, and leaves a change to it running everything through the
per-path escalation.
**`hack/e2e-apps/<name>.bats` maps to the `<name>` suite**, off the
basename, exactly as the per-suite rule takes the name off a
`hack/e2e-chainsaw/<app>/` directory. Deliberately mapped rather than
marked inert: what remains there is wired to nothing after the Chainsaw
migration, and inert would bake that orphan status into the rule and go
quietly wrong the day a lane runs those files again. The basename is
membership-tested against the suite list on the spot, so an unmatched
one escalates there and then rather than having the verdict deferred to
the final intersection, where the rest of the diff would decide it.
**`packages/tests/` and three `.gitattributes` join
`inert_config_pattern`,** which is what the script's own header says to
do with a genuinely inert path instead of widening the fall-through.
`packages/tests/` is a helm-unittest fixture chart, and changing a test
*of* `cozy-lib` does not change `cozy-lib`, no PackageSource lists those
paths as a component, and nothing installs them, while a change to the
library itself still escalates through `packages/library/`. The
`.gitattributes` entries are enumerated rather than matched by filename,
because the justification is what those files contain (only
`linguist-generated` markers) and the name does not carry it:
`.gitattributes` can also set `filter`, `eol`, `working-tree-encoding`
and `export-subst`, each of which changes what lands in the working tree
and therefore what gets built.
**`full_suite_pattern` escalates only `hack/e2e-*.bats`, not every
`hack/*.bats`.** The root `Makefile` is the authority on the split and
draws it at exactly that prefix, `BATS_UNIT_FILES := $(filter-out
hack/e2e-%.bats,$(wildcard hack/*.bats))`, so the 60 files it keeps are
the unit lane and the e2e sandbox runs none of them, while the three it
filters out are what `packages/core/testing`'s recipes execute and those
still escalate. Being inert here does not leave them untested: `make
unit-tests` is gated on the `plan` job's `code` output, which
`pull-requests.yaml` computes as "any changed path outside `docs/`" and
never from `select-e2e.sh`, so a bats-only pull request still runs the
whole unit lane, plus install and the OpenAPI tests, since the `e2e` job
reads the same output rather than the selection.
## Measured effect
The before-and-after comparison below is a second measurement over a
slightly different population, stated separately because the two are not
interchangeable: it replays the last 150 first-parent merge commits on
`main` (`git diff --name-only $sha^1 $sha`) through the base selector
with base sources versus this branch with its own, where the histogram
above walks the 150 merged pull requests as GitHub lists them. The
populations overlap heavily and the direction is the same, and the
totals differ by two commits.
| | full suite | scoped | nothing |
|---|---|---|---|
| before | 120 | 20 | 10 |
| after | 106 | 25 | 19 |
Full-suite rate 80.0% → 70.7%, verdict changed on 15 commits. Fourteen
de-escalated and I read every one of their file lists: they are
unit-lane bats files, non-e2e workflows, `docs/`, `CODEOWNERS`,
helm-unittest fixtures and linguist markers. The fifteenth gets one
suite *wider*: a `cert-manager` change now also selects `etcd`, which is
correct, since `cert-manager` is a dependency of `etcd-operator` and the
etcd app now genuinely reaches it through the new edge.
That replay predates the two reachability rules, and replaying the same
150 first-parent merges across those two on their own — the selector at
`73bc4c0ae` against the selector at `adf231513`, same sources on both
sides — changes no verdict at all. Two of the 150 touch
`cozystack-basics` and both ran the full suite before and after, which
is the hub exclusion keeping something true rather than failing to do
anything, and none of the 150 touch `kubevirt-cdi` or `vm-disk`, so the
mapping shows up only in a direct selection.
A few individual selections, before → after:
| changed files | before | after |
|---|---|---|
| `hack/select-e2e_test.bats` | 21 suites | *nothing* |
| `hack/select-e2e_test.bats` +
`packages/apps/redis/templates/redis.yaml` | 21 suites | `redis` |
| `hack/e2e-install-cozystack.bats` | 21 suites | 21 suites |
| `packages/tests/cozy-lib-tests/**` | 21 suites | *nothing* |
| `packages/system/.gitattributes` | 21 suites | *nothing* |
| `hack/e2e-apps/postgres.bats` | 21 suites | `postgres` |
| `packages/system/etcd-operator/values.yaml` | 21 suites | `etcd` |
| `packages/system/kubevirt-cdi/values.yaml` | 21 suites | `vminstance`
|
| `packages/apps/vm-disk/values.yaml` | 21 suites | `vminstance` |
| `packages/system/cozystack-basics/values.yaml` | 21 suites | 21 suites
|
## Blast radius of the `dependsOn` edge
A `dependsOn` on a PackageSource is an install-ordering edge in
production, not only a test-selection hint, so this is the part worth
reviewing hardest. Both Packages are emitted by
`packages/core/platform/templates/bundles/system.yaml` under the single
`bundles.system.enabled` guard with no intervening conditional between
them, in every variant, so no supported configuration has the app
without the operator. `hack/select-install.sh --validate` reports the
graph still has no cycle and nothing dangling. The one new exposure is
an admin who lists `cozystack.etcd-operator` in
`bundles.disabledPackages` while keeping the app, which now leaves it
`DependenciesNotReady`. That is the same exposure postgres, mariadb,
kafka and redis already carry, with `Package.spec.ignoreDependencies` as
the escape hatch. On upgrade `etcd-rd` waits for the operator's two
HelmReleases, and an unhealthy `etcd-operator` already means broken etcd
apps.
## Testing
Every rule added or changed has a test in `hack/select-e2e_test.bats`,
the install-side change has one in `hack/select-install_test.bats`, and
the new graph edge has a helm-unittest guard at
`packages/core/platform/tests/sources_etcd_application_dependson_test.yaml`.
The round-trip test that walks every suite through both mapping tables
stays green, `vminstance` included now that two sources map onto it.
The two reachability rules were each checked by removing the rule and
confirming its own test reddens: without the mapping the CDI test falls
back to the full-suite escalation, and without the hub exclusion
`cozystack-basics` selects `redis vminstance` instead of everything. The
hub test has to seed its own hazard, since no suite sits downstream of
basics in this tree yet, so it gives `redis-application` an edge onto
basics in a copied sources directory and pins that a redis change still
selects redis, or the assertion would pass against a graph that resolved
nothing.
`make bats-unit-tests` aborts on the first failing file (`for f in …; do
… || exit 1; done`), and `hack/ghcr-mirror_test.bats` fails on `main`
today and sorts before `select-e2e_test.bats`, so the aggregate target
never reaches these tests. I ran the 60 unit files individually instead:
59 pass, and the one failure is that pre-existing one, byte-identical to
the base commit and referencing nothing this branch touches. Also green:
`make helm-unit-tests`, `make rd-presets-check migrations-target-check
test-check-readiness`, `hack/select-install.sh --validate`, and
`pre-commit run --all-files` leaving a clean tree.
## Two things a reviewer should know, neither addressed here
`assert_selection` in `hack/select-e2e_test.bats` aborts the whole file
with `exit 1` rather than failing a single test. That is the only thing
that works under `hack/cozytest.sh`, which provides no real `run`,
`$status` or `skip`, and it is the idiom the existing tests in that file
already use. It becomes wrong when #3497 and #3498 flatten these onto
real `bats(1)`, where a failing assert should end one test and let the
rest report. Worth converting with that work rather than ahead of it.
`cozystack.mongodb-application` carries the identical missing operator
edge: `packages/apps/mongodb` renders `psmdb.percona.com/v1` while the
source depends only on `cozystack.networking` and
`cozystack.cozystack-engine`, so `cozystack.mongodb-operator` still
reaches no runnable suite and every change to it escalates to all 21.
Same defect class and same fix shape as the etcd edge, but it is a
second production install-ordering change and belongs in its own pull
request.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **New Features**
- The default etcd application now includes its operator and required
platform dependencies, ensuring the EtcdCluster provider is available.
- **Bug Fixes**
- Improved end-to-end test selection for application-specific changes,
shared components, inert paths, and unresolved or unclassified files.
- Full-suite escalations now provide clear reasons through standard
error.
- **Tests**
- Expanded coverage for test selection, dependency reachability, and
etcd application dependency declarations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
|
||
|
Aleksei Sviridkin
|
4385990869 |
feat(redis-operator): migrate from archived spotahome to freshworks-oss fork (#3406)
## What this PR does `spotahome/redis-operator` was archived on 2026-06-11 (read-only; last release v1.2.4, 2022). This migrates the operator to the actively-maintained **freshworks-oss/redis-operator** fork (v3.3.5, Apache-2.0). ### Why freshworks-oss It keeps the same `RedisFailover` CRD and API group (`databases.spotahome.com`) — so existing managed Redis is unaffected — and adds a native `spec.engine: Redis|Valkey` switch, letting managed Valkey drop its Redis-compatibility shim image and run the official `valkey/valkey` image directly. ### Changes - Re-vendor the chart from `oci://ghcr.io/freshworks-oss/charts/redis-operator` (v3.3.5), which carries the `engine`-aware CRD. - Build the operator image from freshworks-oss v3.3.5 source. - Re-target the label/annotation-preservation patch onto the freshworks Go module (`github.com/freshworks/redis-operator`). ### Verification - freshworks-oss source + our `labels.diff` patch compiles natively with Go 1.26.3 (`service` and `cmd` packages). - `git apply --check` of the re-targeted patch against v3.3.5 succeeds. - The operator chart renders with the cozystack-registry image override. - The image is built (freshworks-oss v3.3.5 source + our patch, multi-arch amd64/arm64), pushed to `ghcr.io/cozystack/cozystack/redis-operator`, and pinned by digest in `values.yaml`. - `hack/promote-retag_test.bats` and `hack/nightly-mirror_test.bats` pass with the pinned digest. ### Note for maintainers `packages/system/redis-operator` is not part of the automated `make build`; its image is built out-of-band and the digest is pinned in `values.yaml`. That image is already built and pinned in this PR; re-run `make -C packages/system/redis-operator image` only if you bump the operator version. ### Follow-up The cozystack/redis-operator fork carries cert-manager TLS work (cozystack/cozystack#2729) that freshworks-oss does not have yet; the plan is to upstream it to freshworks-oss so it lands in the shared maintained fork rather than a cozystack-only one. ### Release note ```release-note feat(redis-operator): migrate from the archived spotahome/redis-operator to the actively-maintained freshworks-oss/redis-operator fork (same RedisFailover CRD, adds native spec.engine: Redis|Valkey) ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for injecting additional environment variables into the Redis Operator (including a default operator group id). * Added default configurable labels for the Redis Operator. * **Bug Fixes** * Preserves and merges existing Service labels and annotations during Redis Operator updates. * **Chores** * Updated the Redis Operator and Helm chart to the Freshworks-maintained 3.3.5 release, aligned chart/image sources, and simplified Helm chart retrieval. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
c9ee4f6de8 |
test(platform): pin redis-operator upgradeCRDs=CreateReplace
Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Myasnikov Daniil
|
42bd45a87c |
fix(platform): make etcd-application depend on etcd-operator
extra/etcd renders kind: EtcdCluster from etcd-operator.cozystack.io/v1alpha2, but cozystack.etcd-application declared no dependsOn edge to cozystack.etcd-operator -- the only operator-backed application source in the tree that did not. The edge has been absent since the sources were first added. Two consequences, one per side of the graph. On install this is a latent ordering bug. Without the edge the etcd-rd HelmRelease registers the ApplicationDefinition as soon as cozystack-engine is up, so a tenant can create an Etcd before the operator exists and its HelmRelease then fails with `no matches for kind "EtcdCluster"`. hack/select-install.sh's forward closure for the etcd suite omitted the operator for the same reason, which would install the suite against a cluster with no CRD to apply to. On test selection cozystack.etcd-operator reached no runnable suite, so every change to the operator escalated to all 21 suites -- 5 of the last 150 merged pull requests. The etcd suite exists; only the edge that makes it reachable from the operator was missing. Scope of the ordering change: both Packages are emitted unconditionally by the same bundle (templates/bundles/system.yaml, under the single bundles.system.enabled guard, in every variant), so no supported configuration has one without the other. An admin who lists cozystack.etcd-operator in bundles.disabledPackages while keeping the app now leaves it DependenciesNotReady -- the same exposure postgres, mariadb, kafka and redis already carry, with Package.spec.ignoreDependencies as the escape hatch. select-install.sh --validate confirms the edge adds no cycle and nothing dangling. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
c3cead8f64 |
fix(telemetry): make the documented opt-out cover both collectors
Telemetry is reported by two components, cozystack-operator (cluster facts) and cozystack-controller (application counts), but only the operator had a reachable switch. The documented opt-out sets cozystackOperator.disableTelemetry on the cozy-installer chart, which cannot reach a package the platform deploys, so cozystack-controller kept reporting after an operator had followed the procedure — silently, with nothing to indicate the opt-out was partial. The platform now exposes telemetry.disabled and threads it into the cozystack-controller component of the cozystack-engine Package, the same way the bundle already threads cilium, multus and linstor values. The installer key is also declared in values.yaml: it was only ever referenced from the template, so the documented --set target did not appear in the chart's own values. Both keys stay false by default; this changes no behaviour for anyone who has not opted out. Assisted-By: Claude Signed-off-by: Andrei Kvapil <kvapss@gmail.com> |
||
|
Aleksei Sviridkin
|
f4d812cfb9 |
fix(velero): apply CRD updates on upgrade via CreateReplace
velero evolves its CRD schema between versions; v1.18 adds the Queued and ReadyToStart backup phases. Helm never upgrades CRDs shipped in crds/, the generated HelmRelease left spec.upgrade.crds at the Skip default, and the package disables the chart's upgrade-crds Job, so live CRDs stayed frozen at first-install state: the server upgrades, the apiserver rejects the new phases, and backups sit in New while the HelmRelease stays green. Opt the velero PackageSource into CreateReplace, consistent with mongodb-operator, add a platform test pinning the policy, and correct the stale Job-disable rationale in the package. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <3811295@gmail.com> |
||
|
Aleksei Sviridkin
|
7739d1e294 |
feat(multus)!: stage reference CNI plugins from the multus image
Multus-backed NetworkAttachmentDefinitions delegate to the upstream containernetworking reference plugins (bridge, macvlan, ipvlan, ...), and nothing guarantees a node has them in the directory Multus searches. On k3s they live under /var/lib/rancher/k3s/data/cni while Multus and Cilium install into /opt/cni/bin, so a bridge-type NAD fails there with `failed to find plugin "bridge" in path [/opt/cni/bin]`. Bundle 14 of the release's plugins into the multus image and add an install-cni-plugins init container that installs them into the host /opt/cni/bin, mirroring the existing install-multus-binary container. loopback, dhcp, tap and dummy are left out; the Dockerfile records why. Gate it on a new chart value, stageCniPlugins, resolved per bundle and on for both. Talos bundles only bridge, firewall, flannel, host-local, loopback and portmap, so the other ten are missing there exactly as on generic Linux; the four that overlap are rewritten with the same upstream version, v1.9.1 on both sides, rather than an older one. A marker in the image Dockerfile records the Talos release that pin was compared against, and a test holds it to the release the platform ships. multus therefore moves out of system.common-packages into the two variant branches, the way linstor already does for talos.enabled. networking.stageCniPlugins overrides the variant default in either direction. Where staging is on the install is unconditional: a plugin is replaced on every pod recreation, not only when the destination is missing. A destination that is already a directory is reported and left alone, since replacing it is not something this can do safely. Turning the value off afterwards does not undo it; nothing here removes or restores a plugin. The container is not privileged and takes no mount propagation. It runs in the spc_t SELinux domain, without which an enforcing node refuses the write. Each plugin is installed under a temporary name and swapped in with a rename: /opt/cni/bin is shared with cilium and kube-ovn. The plugin tarball is checksummed before extraction. A plugin that cannot be installed is reported on stderr and to /dev/termination-log, and the container still exits 0: Multus stays the node's primary CNI and nothing removes its config, so an init container that never completes would leave the shim with no daemon behind it. The manifest change lives in customize-deployment.patch so it survives make update. BREAKING CHANGE: enabling stageCniPlugins replaces reference CNI plugins in /opt/cni/bin on existing nodes, Talos included -- four of the names it writes are ones the Talos node image also carries. Opt out before upgrading: declining afterwards does not restore what was replaced. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
myasnikovdaniil
|
3f967ec7e0 |
fix(platform): restore the isp-hosted dependency closure (#3391)
## Problem Fixes #3376. The `isp-hosted` preset shipped `bundles.system.enabled: false` while `paas` and `naas` were enabled, so it enabled an app catalog whose dependency closure it simultaneously disabled. Every emitted Package sat in `DependenciesNotReady`, the `*-rd` HelmReleases failed with `no matches for kind "ApplicationDefinition" in version "cozystack.io/v1alpha1"`, and the `cozystack-platform` HelmRelease never went Ready. Rendering the preset at `origin/main` shows the scale: **24 emitted Packages, 24 of them blocked**, by 6 missing dependencies — two more than the issue lists (`cert-manager` and `prometheus-operator-crds` are also absent). `isp-full` renders 75 Packages with a fully satisfied closure, which is a clean control. ## Root cause `templates/bundles/system.yaml` is gated end to end on `bundles.system.enabled`, but it carries two separable things: a variant-selected **data-plane head** (kubeovn/cilium/linstor on `isp-full`, a noop networking Package on `isp-hosted`) and a variant-agnostic **control-plane body** — `cozystack-engine`, the sole source of the `application-definition-crd` component, plus `cert-manager`, `prometheus-operator-crds`, `postgres-operator` and `objectstorage-controller`. "Hosted" means the host already provides CNI, storage and virtualisation, so only the data-plane head should be dropped. Disabling the whole bundle also deleted the control-plane body that every variant needs. Three things show the disable was the mistake rather than the intent: the preset still sets `variant: "isp-hosted"`; `system.yaml` contains an `isp-hosted` branch (noop networking plus default scheduler) that could never render; and `iaas.yaml` already refuses to render on this variant, encoding the same "system bundle minus iaas" contract. The preset has never worked — it was born this way in `2d022e38e` (January 2026), the refactor that replaced `bundles/paas-hosted.yaml`, whose contents confirm hosted was always meant to ship the engine, `cert-manager` and `prometheus-operator-crds`. ## Changes 1. **`values-isp-hosted.yaml`** — enable the system bundle. Hosted now emits 53 Packages with every `dependsOn` satisfied, installing 59 components against the 60 of the pre-refactor `paas-hosted` bundle, with no LINSTOR, KubeVirt, CAPI or Kamaji leakage. 2. **Move `cozystack.bucket-application`** from the iaas bundle into the system bundle. Its own `dependsOn` closure is `networking` + `objectstorage-controller` + `cozystack-engine`, none of them iaas, while `cozystack.backupstrategy-controller` — a *default* package of the system bundle — hard-depends on it. Any system-on/iaas-off configuration therefore wedged `backupstrategy-controller`; both `isp-full` presets masked this by enabling iaas. This is a second, independent bug not reported in the issue. 3. **Guards** — `paas`, `naas` and `iaas` now also require `bundles.system.enabled`. The pre-existing checks looked only at `bundles.system.variant`, which keeps its value when the bundle is disabled, so they passed on exactly this configuration. `iaas` had the identical hole (12 Packages, 6 unsatisfied deps). This turns a silent multi-hour `DependenciesNotReady` hang into a render-time failure that names the remedy. 4. **Two helm-unittest suites** — one pinning the dependency closure the catalog needs, one pinning the guards. ## Verification - All four operator-registered variants render dependency-closed: `default` 0 Packages, `isp-hosted` 53, `isp-full` 75, `isp-full-generic` 75. The two `isp-full` variants are byte-identical before and after, so the bucket move is a no-op for them. - Hosted is a strict subset of full with zero data-plane leakage, and its exclusions match the pre-refactor hosted bundle. - 23 suites / 97 tests pass in `packages/core/platform`; repo-wide `hack/helm-unit-tests.sh` is green. - Every change was **mutation-tested** — reverting each one in turn makes the suite fail, so the tests are not vacuous. - The workaround documented in #3376 (`paas`/`naas` disabled) still renders cleanly; the new guards do not fire on it. - Hosted runtime spot-checks: `cozystack-api` is a Deployment with only a *preferred* control-plane affinity plus blanket tolerations, and `cozystack-scheduler` uses its `default` variant (plain Deployment, no LINSTOR extender) — both schedule fine on a cluster with no control-plane nodes. ## What this does not fix This restores the dependency closure. It is **not** by itself sufficient for a fully converged hosted install. Review surfaced further hosted-unaware defaults that the dead bundle had been masking, filed separately: - #3387 — `apps/tenant` renders 14 `cilium.io/v2` policies with no `.Capabilities` guard, so `tenant-root` fails on a non-Cilium cluster. This is the blocker for actual hosted convergence, and it needs a deliberate decision rather than a reflexive capability guard, because those policies implement tenant network isolation. - #3388 — `extra/monitoring` defaults `logsStorages` to the LINSTOR-only `replicated` StorageClass. - #3389 — `bundles.disabledPackages` can still break the closure with no render-time error; the guards here cover the bundle flags only. - #3390 — nothing in CI exercises `isp-hosted`, which is why all of the above accumulated undetected. ## Upgrade note Existing `*-application` Package CRs carry `helm.sh/resource-policy: keep`. An operator who applied the #3376 workaround but did not also delete the retained Package CRs (as that issue instructs) will find them install once this change makes their dependencies Ready, since the reconciler re-enqueues dependents on readiness. Deleting the unwanted Packages before upgrading avoids that. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added fail-fast validation that requires the **System bundle** when enabling **IaaS**, **NaaS**, or **PaaS** to prevent incomplete catalogs. * Updated bundle composition so the **bucket application** is provided by the **System bundle**. * Enabled the **System bundle** by default for the `isp-hosted` variant. * **Bug Fixes** * Improved error messaging to clearly indicate the required bundle/variant configuration. * **Tests** * Added Helm unittest coverage for rendering-gate failures, dependency/package expectations, and hosted data-plane exclusions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Andrei Kvapil
|
4c3262a4ee |
feat(computeplane): deliver ComputePlane as a tenant module wrapping the kubernetes app
Add the operator-owned computeplane tenant module per the revised design (cozystack/community design-proposals/compute-plane): an extra/computeplane chart that provisions a single-tenant, Cozystack-managed cluster for untrusted-code apps by deploying the ordinary apps/kubernetes chart with operator-fixed values (fluxcd, cert-manager, ingress-nginx Proxied). The package re-declares apps/kubernetes as its own source-only component, so the module is self-contained; the cluster release is named computeplane so Kamaji writes the admin kubeconfig to computeplane-admin-kubeconfig — the contract consumed by placement: ComputePlane applications. The tenant holds no admin kubeconfig and can set cluster shape (nodeGroups) only. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io> |
||
|
Aleksei Sviridkin
|
8bffa34b54 |
feat(platform): add opt-in in-cluster east-west encryption toggle (#2984)
## What this PR does Adds `networking.encryption.enabled` to the platform chart — a single operator-facing opt-in, off by default, that turns on in-cluster (east-west) transport encryption for the active datapath. When enabled, pod-to-pod traffic across nodes is encrypted at the network layer regardless of whether the workload implements TLS: defense-in-depth for the whole mesh without per-application changes. The default `isp-full` bundle selects the `kubeovn-cilium` datapath (and `isp-full-generic` selects `kubeovn-cilium-generic`), where KubeOVN owns the datapath and the overlay is a Geneve tunnel. The toggle therefore maps to KubeOVN's overlay IPsec (`kube-ovn` `ENABLE_OVN_IPSEC`). KubeOVN's own controller provisions the IPsec CA and per-node certificates through the Kubernetes CSR API, so no cert-manager wiring is needed. This complements, and does not replace, application-level / end-to-end TLS: in-cluster encryption never reaches an external client and gives a tenant no verifiable endpoint certificate. The values comments document the per-datapath scope — including that Kilo's WireGuard mesh (`meshGranularity: location`) covers only cross-location traffic and leaves the intra-location gap this toggle targets, and that the Cilium-native WireGuard/IPsec path on the `cilium` / `cilium-generic` variants is not reached because the system bundle does not select those variants — plus the MTU/throughput overhead. Scope: only the default KubeOVN datapath is wired through the platform toggle, which is what the system bundle actually selects. Wiring the Cilium-native path would require the bundle to expose cilium-only variant selection; that is out of scope here and called out in the values comments. Validation: `helm unittest` on the platform chart passes (66 tests, 7 new) covering on / off / absent / null and verifying the IPsec flag deep-merges without dropping the existing `kube-ovn` overrides, for both `isp-full` and `isp-full-generic`. Closes #2977 Part of #2811 ### Screenshots N/A — no UI changes. ### Release note ```release-note feat(platform): add opt-in `networking.encryption.enabled` (off by default) that enables in-cluster east-west transport encryption for the default KubeOVN datapath via overlay IPsec; complements, and does not replace, application-level TLS. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced opt-in in-cluster network encryption using KubeOVN IPsec for east-west traffic protection. * Added `networking.encryption` configuration toggle (disabled by default) with full backward compatibility. * **Tests** * Added comprehensive test coverage for network encryption configurations across system variants and edge cases. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timofei Larkin
|
2823b76802 |
feat(gateway): advertise HTTP/2 via ALPN on Gateway API listeners (#3398)
## What this PR does Browsers negotiate HTTP/2 exclusively through TLS ALPN: if the terminating listener does not advertise `h2`, every client silently falls back to HTTP/1.1, with no error surfaced anywhere. Cozystack's Gateway API path is served by Cilium's embedded Envoy, and Cilium ships with `enable-gateway-api-alpn=false` — so every Gateway listener advertised no ALPN protocols and all browser traffic through a Gateway ran over HTTP/1.1 (observable in browser dev tools as `http/1.1` on every request). The ingress-nginx path this replaces advertised `h2` out of the box, so the migration to Gateway API silently downgraded clients: head-of-line blocking and the ~6-connections-per-origin limit are back for anything served through a Gateway. This PR enables ALPN (`h2`, then `http/1.1`) on Gateway API listeners, default-on, with an opt-out at each management layer: - `packages/system/cilium/values.yaml` now defaults `gatewayAPI.enableAlpn: true`, which renders `enable-gateway-api-alpn: "true"` into the `cilium-config` ConfigMap. - The platform chart gains `gateway.http2` (default `true`) next to the existing `gateway.enabled`/`gateway.attachedNamespaces`. The `isp-full` and `isp-full-generic` bundles thread it as inline component values into the `cozystack.networking` Package CR (`components.cilium.values.cilium.gatewayAPI.enableAlpn`); inline HelmRelease values override the package `valuesFiles`, so an explicit `gateway.http2: false` actually reaches the dataplane rather than being shadowed by the new chart default. On `isp-hosted` the hosting-side CNI owns this setting and the flag is a no-op. - Tenant Kubernetes clusters (`packages/apps/kubernetes`) get the same fix: when the `gatewayAPI` addon is enabled, the tenant cilium HelmRelease now sets `enableAlpn: true`; individual clusters opt out via `addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false`. ### Why default-on rather than opt-in HTTP/2 for browsers was the platform's behavior before the Gateway API migration, so keeping ALPN off would preserve a regression, not a status quo. Defaulting on is safe because ALPN is a negotiation: clients that cannot speak HTTP/2 simply keep negotiating `http/1.1`, exactly as before. Operators who need the old wire behavior can set `gateway.http2: false` platform-wide. ### Frontend vs backend hop This change affects the client↔gateway hop only. Gateway↔backend connections stay HTTP/1.1 unless a Service explicitly opts in per GEP-1911 by declaring `appProtocol: kubernetes.io/h2c` on its port. Cilium couples that backend-protocol support to the ALPN flag (`enable-gateway-api-app-protocol` flips to true together with it), but it only acts on Services that declare a recognized `appProtocol` value — and no cozystack-shipped Service does today (only plain `http`/`https` values exist in-tree, which Cilium ignores for protocol selection). So upstreams see no change from this PR alone; h2c/gRPC backends become possible to configure, not configured. The setting is cluster-wide Cilium agent config, so it covers the root Gateway and all tenant Gateways at once — Cilium offers no per-Gateway granularity here. Flipping it re-rolls the cilium DaemonSet on upgrade (`rollOutCiliumPods: true`), the same disruption profile as any other cilium config change. ### How it was verified - New helm-unittest suites: `packages/core/platform/tests/bundles_gateway_http2_wiring_test.yaml` asserts the Package CR carries `enableAlpn` true by default and false with `gateway.http2: false` for both cilium bundles; `packages/apps/kubernetes/tests/cilium_gateway_alpn_test.yaml` asserts the tenant HelmRelease default, the addon-off case, and the `valuesOverride` opt-out. Full suites for both charts pass (93 and 184 tests). - `helm template` of `packages/system/cilium` confirms the value is consumed, not just rendered: default produces `enable-gateway-api-alpn: "true"` (and `enable-gateway-api-app-protocol: "true"`) in `cilium-config`; the threaded `false` produces `"false"` for both. ### Screenshots Not a UI change. ### Downstream repositories The website's hand-written platform values table (`content/en/docs/next/operations/configuration/platform-package.md`) needs a row for `gateway.http2` — filed as cozystack/website#625. No other downstream repository restates anything in this diff (no app `values.schema.json`, ansible-consumed platform key, or hack/ tooling changed). - [ ] No downstream repository is affected by this change - [x] [cozystack/website](https://github.com/cozystack/website) - follow-up: https://github.com/cozystack/website/pull/625 - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: ### Release note ```release-note feat(gateway): Gateway API listeners now advertise HTTP/2 via TLS ALPN (h2, then http/1.1) by default, restoring the browser HTTP/2 support lost in the migration from ingress-nginx. Opt out platform-wide with gateway.http2=false, or per tenant Kubernetes cluster via addons.cilium.valuesOverride.cilium.gatewayAPI.enableAlpn=false. Backend (gateway-to-upstream) connections are unchanged and stay HTTP/1.1 unless a Service declares appProtocol kubernetes.io/h2c. ``` |
||
|
Timofei Larkin
|
ebe111c439 |
feat(gateway): advertise HTTP/2 via ALPN on Gateway API listeners
Browsers negotiate HTTP/2 exclusively through TLS ALPN "h2". Cilium's Gateway API implementation ships with enable-gateway-api-alpn=false, so every Gateway listener advertised no ALPN protocols and every browser silently fell back to HTTP/1.1 — a regression against the ingress-nginx path, which advertised h2 out of the box. Enable ALPN (h2, then http/1.1) by default on the client-facing hop: - packages/system/cilium: default gatewayAPI.enableAlpn=true. - platform chart: new gateway.http2 value (default true), threaded as inline component values into the cozystack.networking Package for the isp-full and isp-full-generic bundles, so an explicit false overrides the system/cilium valuesFiles default either way. - apps/kubernetes: tenant-cluster cilium gets enableAlpn=true whenever the gatewayAPI addon is enabled; opt out per cluster via addons.cilium.valuesOverride. The backend hop is unchanged: gateway-to-upstream stays HTTP/1.1 unless a Service opts in per GEP-1911 via appProtocol kubernetes.io/h2c (that support is switched on together with ALPN; no cozystack-shipped Service declares such a value today). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Andrey KolkovandClaude Fable 5
|
7d117679d4 |
docs(backups): document bucketName's two modes, cover isp-hosted in the wiring test
Branch-review on this PR: for provisionBucket: false the backupstrategy-controller.bucketName helper returns .Values.backupStorage.bucketName verbatim as the real S3 bucket name and every strategy CR plus the Velero BSL bakes it in, so the external-S3 section must require updating bucketName (the Secret's bucketName key alone is not read by the strategies), and the knob table must state the two-mode semantics instead of the COSI-only one. Also route the external-S3 instruction through the platform component override instead of naming raw chart values, and add the isp-hosted variant case to the forwarding test (92 tests green). Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Andrey KolkovandClaude Fable 5
|
1422f8b78a |
fix(platform): forward backupStorage overrides to the backupstrategy-controller Package (#3245)
The documented admin override for the cozy-default backup S3 coordinates (spec.components.backupstrategy-controller.values on the cozystack.cozystack-platform Package) had no effect: that PackageSource exposes only the platform component, so the operator silently ignored the block, and nothing in the platform chart forwarded backupStorage to the child Package. Wire it the same way bundles/iaas.yaml forwards kubevirt / gpu-operator values: a new top-level backupStorage value on the platform chart is rendered into the emitted cozystack.backupstrategy-controller Package CR as components.backupstrategy-controller.values.backupStorage, from where the cozystack operator merges it into the backupstrategy-controller HelmRelease over the chart defaults. When backupStorage is unset the emitted Package is unchanged. Update docs/operations/backup-classes.md to the working override path (spec.components.platform.values.backupStorage) and call out the two plausible-looking paths that do not work. Add a helm-unittest suite covering the no-op default and the forwarding on isp-full and isp-full-generic. Fixes #3245 Assisted-By: Claude <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com> |
||
|
Myasnikov Daniil
|
0bfbff26ff |
fix(platform): restore the isp-hosted dependency closure
The isp-hosted preset shipped bundles.system.enabled: false with paas and
naas enabled. templates/bundles/system.yaml is gated end to end on that
flag, but it carries two separable things: a variant-selected data-plane
head (kubeovn/cilium/linstor on isp-full, a noop networking Package on
isp-hosted) and a variant-agnostic control-plane body — cozystack-engine,
the sole source of the application-definition-crd component, plus
cert-manager, prometheus-operator-crds, postgres-operator and
objectstorage-controller.
"Hosted" means the host already provides CNI, storage and virtualisation,
so only the data-plane head should be dropped. Disabling the whole bundle
also deleted the control-plane body, leaving an app catalog whose entire
dependency closure was absent: all 24 emitted Packages sat in
DependenciesNotReady, the *-rd HelmReleases failed with `no matches for
kind "ApplicationDefinition" in version "cozystack.io/v1alpha1"`, and the
cozystack-platform HelmRelease never went Ready. That the variant field
was still set to isp-hosted — and that system.yaml has an isp-hosted
branch which could never render — shows the disable was the mistake, not
the intent; iaas.yaml already refuses to render on this variant, encoding
the same "system bundle minus iaas" contract. The preset has never worked
since it was introduced in
|
||
|
Myasnikov Daniil
|
efa85a4f61 |
feat(platform): make the etcd-adoption skip-backup hatch reachable
Migration 50 has honoured ETCD_ADOPT_SKIP_BACKUP since it was written, and its failure paths tell operators to "re-run with ETCD_ADOPT_SKIP_BACKUP=1". But migration-hook.yaml passes only NAMESPACE/CURRENT_VERSION/ TARGET_VERSION, and the Job is a Helm hook the chart re-renders, so there has never been a way to set it. The script advertised an escape the platform made impossible. The endpoint-classification work landing alongside this makes that worse: it adds refuse-rather-than-guess paths, so more clusters can now legitimately stop here. Plumb it through migrations.etcdAdoptSkipBackup, default false. Skipping the pre-adoption snapshot rewires ownership of LIVE etcd storage with no way back, so it stays opt-in and deliberate — the flag is the last resort for a cluster that has nothing to fix, not the first response to a refusal. Render "1"/"0" rather than the bare bool. The script matches the literal string "1", so a bool would render "true", sail through the Job spec looking correct, and be ignored — a safety valve that is set but silently disregarded is worse than one that was never offered, because the operator believes they opted in. The value is normalised through a string, so a bool, a quoted "true" out of the Package CR's Values JSON, or a numeric 1 all land on the same answer, while nil or garbage resolves to "0": taking this hatch must require an affirmative spelling, never a typo. Resolved with `dig` so a Platform Package predating the key renders "0" instead of failing the render of the entire platform chart and bricking every upgrade. Widen the script's own check to accept 1/true/yes as well. The template guarantees the GitOps path, but an operator running this image by hand will reasonably type "true", and that is exactly the moment they are already stuck. The two layers cover independent failure modes. Emit the var unconditionally, including the "0" default, so the rendered Job states the cluster's safety posture outright rather than leaving an operator to diff values to find out whether a snapshot was skipped. Point the failure messages at the values path that actually exists, and describe it honestly: editing the Platform Package and letting Flux re-render is a real operation, not a one-liner, and the flag has to be reverted afterwards or the next etcd migration skips its snapshot too. Fixing the condition the migration names remains the recommended path — the migration is idempotent, so re-running after a fix is safe. templates/migration-hook.yaml had no coverage at all, because the Job only renders when a lookup of the cozystack-version ConfigMap reports a version below targetVersion, and lookup is nil under helm template. The new suite mocks that ConfigMap and asserts the RENDERED VALUE — enabled, disabled, key absent, stringified true, garbage, and the pre-existing three env vars — rather than merely that the variable exists, which is what let the trap above go unnoticed in the first place. Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
fbbe4af7df |
refactor(network): drop ExposureClass/ServiceExposure for native loadBalancerClass (#3218)
## What Removes the `network.cozystack.io` API group — `ExposureClass` + `ServiceExposure` (#3081) — and its reconciler, in favor of native `Service` `type: LoadBalancer` + `loadBalancerClass`. ## Why Full rationale and discussion in #3164. For **managed applications** the chart already owns the Service, so: - `type: LoadBalancer` + `loadBalancerClass` + an admin-provisioned pool covers vendor-neutral external exposure without a dedicated API group and controller; - the `externalIPs` (node-IP) default never went through `ServiceExposure` (its backend is a no-op) and is left unchanged; - the only current consumer, the host ingress, keeps rendering `type: LoadBalancer` and just drops the `ServiceExposure`; the admin provisions the address pool / announcer out of band, as the install docs already assume. `ServiceExposure`'s one unique feature — dynamic per-namespace scoping of a shared VIP pool — is a convenience the managed-app path does not need, and it does not justify a second tenant-networking API group next to `sdn.cozystack.io`. ## Changes - Delete `api/network/v1alpha1` and `internal/controller/serviceexposure` (+ CRD definitions, e2e bats, controller registration, RBAC). - Host ingress: drop the `ServiceExposure` template; add an optional `publishing.loadBalancerClass` that sets `spec.loadBalancerClass` on the host-ingress LoadBalancer Service. - The `externalIPs` path and default behavior are unchanged. ## Verification - `go build ./...` — green. - `helm unittest` — `core/platform` 72/72, `extra/ingress` 25/25 (incl. host-vs-tenant `loadBalancerClass` scoping and the immutable-field migration guard). ## Migration None required. `network.cozystack.io` (#3081) has never shipped in a release — it is on `main` only (not backported to `release-1.5`; absent from v1.4.x / v1.5.x), so no upgrading cluster has these CRDs and this removes it before it is ever released. Clusters that track `main` and explicitly set `publishing.exposureClass` (opt-in, only since 2026-06-30) should provision their MetalLB / Cilium address pool themselves and set `publishing.loadBalancerClass`; any orphaned `cozystack-<class>` pool can be deleted by hand. Default clusters (externalIPs path) are unaffected. ## Status **Draft / RFC for the #3164 discussion** — this removes an API group that landed only days ago, so nothing here should merge before the design call. cc @lexfrei (author of #3081) and @lllamnyp (#3164). Deferred: when Kubernetes actually removes `Service.spec.externalIPs`, the node-IP default migrates via a MetalLB `IPAddressPool` with no `L2Advertisement` (allocate-but-don't-announce) — a separate change. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for configuring the ingress host Service with a native `loadBalancerClass` value. * Expanded chart tests to verify class wiring and immutability behavior across install and upgrade scenarios. * **Bug Fixes** * Removed the legacy exposure-class/service-exposure flow and related controller behavior. * Updated templates and CRD packaging so the platform no longer renders or depends on those deprecated resources. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Ivan Okhotnikov
|
b46543e5f2 |
fix(monitoring): defer platform GrafanaAdmin lever to a future phase
The MAJOR finding from the cozy-review agent: the platform-side `oidc.grafanaAdmin: true` documented in the PR body and in docs/oidc-grafana.md never actually reached the platform Grafana's HelmRelease. `Package.spec.components["monitoring-system"].values.oidc.grafanaAdmin` was set in packages/core/platform/templates/bundles/system.yaml, but `monitoring-system` (and `monitoring`) have no `install:` block in packages/core/platform/sources/monitoring-application.yaml — only `monitoring-rd` does. PackageReconciler skips components without an `install:` block before ever reaching the assignment that copies `pkg.Spec.Components[].Values` onto `hr.Spec.Values` (internal/operator/package_reconciler.go around lines 197-201 and 292-293). The values-override dictionary was silently dropped, no HR carried `oidc.grafanaAdmin: true`, and the platform Grafana rendered with the chart default (`false`). The `bundles_monitoring_grafana_admin_test.yaml` unit test only asserted against the emitted Package CR spec — exactly the layer where the value still existed but was about to be discarded — so the broken propagation slipped past. Trim the broken promise from Phase 1. The chart-side helper (`monitoring.oidc.allowAssignGrafanaAdmin`) stays as-is so a future phase that routes the override through the tenant-root chart's monitoring HR values (or adds an `install:` block on the `monitoring-system` component) lands on a green chart-side contract. The helm-unittest that verifies chart-side emission stays too, but with a docstring calling out that end-to-end wiring is deferred. Everything else that promised the lever works — bundle line, docs, platform bundle test — is either removed or rewritten to match reality. Also picks up two MINOR findings from the same review: * Drop the misleading `secret-hash` annotation on the OIDC client-secret Secret. The comment claimed it "rotates the KeycloakClient CR and therefore the Grafana Deployment env when the client-secret changes", but nothing downstream watches an annotation on the Secret itself — no rotation is triggered. The `lookup`-based fallback already keeps the value stable across reconciles, so no rotation is even needed. Removing the dead annotation avoids planting a broken assumption for future readers. * Add `release.cozystack.io/helm-install-timeout: "15m"` on the monitoring cozyrds ApplicationDefinition. Under `mode: System` the chart now emits, on top of the existing Grafana + Alerta + VMCluster + CNPG stack, four additional Keycloak CRs whose reconciliation is asynchronous against an operator in `cozy-keycloak` (not the tenant namespace). Flux's default 5m install window was already tight for the pre-OIDC stack; 15m matches the precedent set by tenant-rd/kubernetes-rd cozyrds. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Ivan Okhotnikov
|
be52e80289 |
fix(monitoring): split monitoring-application PackageSource into default/oidc variants
Follow-up to the previous self-review-fix commit: the unconditional `- cozystack.keycloak-operator` in dependsOn was itself a CRITICAL cold-install regression on any cluster where `authentication.oidc.enabled: false` (the default). The keycloak-operator Package is only emitted by `packages/core/platform/templates/bundles/system.yaml:112` when that flag is on — a hard dep against it from the default variant would leave `cozystack.monitoring-application` stuck on DependenciesNotReady forever, cascading a `cozystack-platform` HR timeout that hits every OIDC-off customer. Fix by mirroring the cozystack-engine variant split: - `default` variant keeps the baseline reconciliation edges (no keycloak-operator dep). - `oidc` variant additionally waits for `cozystack.keycloak-operator` so the v1.edp.epam.com CRDs are registered before the monitoring chart reconciles and the `.Capabilities.APIVersions.Has` guard in templates/grafana/oidc-keycloak.yaml can safely emit KeycloakClient/Scope/Groups. The bundle template swings between variants on `.Values.authentication.oidc.enabled`, mirroring the cozystack-engine invocation two lines above. The `monitoring-system` component override (oidc.grafanaAdmin: true) is preserved across both branches — the values field is a no-op when mode: None (no auth.generic_oauth block is emitted at all), so keeping it identical avoids two divergent code paths. Also widened the platform helm-unittest coverage to lock: - default variant does NOT depend on keycloak-operator (permanent DependenciesNotReady guard). - oidc variant DOES. - Bundle emits the right variant based on authentication.oidc.enabled AND keeps the GrafanaAdmin uplift on the monitoring-system component in either branch. 83 platform tests + 23 monitoring tests still green. Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com> |
||
|
Ivan Okhotnikov
|
ffeb8e5e4f |
fix(monitoring): close self-review blockers for the OIDC PR
Address the two blocking-class findings from the cozy-review pass on
this PR:
(1) Missing PackageSource dependency on cozystack.keycloak-operator
(MAJOR cold-install race).
Without this edge, the monitoring chart can reconcile before the
v1.edp.epam.com CRDs are registered on a fresh cluster — the
`.Capabilities.APIVersions.Has` guard in
templates/grafana/oidc-keycloak.yaml then silently drops all
Keycloak-side objects for mode: System, and Grafana's
auth.generic_oauth block points at a client Keycloak never
provisions. Add the ordering edge, plus a
sources_monitoring_application_dependson_test.yaml regression
guard so a future edit cannot drop it silently.
(2) allow_assign_grafana_admin driven by `.Release.Name ==
monitoring-system` (MINOR fragile coupling).
Replace the release-name sniff with an explicit
`oidc.grafanaAdmin: false` values field, and flip it on for the
platform Grafana release via
Package.spec.components["monitoring-system"].values.oidc.grafanaAdmin
in packages/core/platform/templates/bundles/system.yaml —
mirroring the pattern monitoring-agents already uses for
per-component overrides. Tenant Grafana releases inherit the
chart default false and stay at org-level Admin.
Two new helm-unittest cases in packages/system/monitoring/tests/
lock the values-driven behaviour on both sides
(grafanaAdmin=true → Admin uplift, default → tenant stays at org
Admin), plus bundles_monitoring_grafana_admin_test.yaml in
packages/core/platform/tests/ regression-guards the Package's
component override so the platform release keeps its uplift on
future edits.
Also document the mode-toggle group-membership loss in
docs/oidc-grafana.md — mode: System → CustomConfig/None deletes the
three chart-owned KeycloakRealmGroups; membership is not restored on
toggle-back. Not a bug, but the operator UX cost of the toggle
deserves a bullet in Failure modes.
Regenerated artifacts (cozyvalues-gen + hack/update-crd.sh):
packages/extra/monitoring/values.schema.json
packages/extra/monitoring/README.md
packages/system/monitoring-rd/cozyrds/monitoring.yaml
Test suites still green: packages/system/monitoring 23/23,
packages/core/platform 81/81.
Signed-off-by: Ivan Okhotnikov <xorokhotnikov@gmail.com>
|
||
|
Aleksei Sviridkin
|
f24b56327b |
fix(platform): route OIDC backend calls through the in-cluster Keycloak by default
authentication.oidc.keycloakInternalUrl defaulted to an empty string, which left oauth2-proxy performing OIDC discovery against the external Keycloak hostname. That request resolves through public DNS and lands on the root ingress controller, which does not serve yet on a fresh install: the root tenant's IngressClass only exists once tenant-root is ready, and tenant-root is still reconciling while the dashboard starts. Discovery has nothing to talk to, the gatekeeper fails, and the dashboard never comes up. The bundled Keycloak is the only supported provider — the external hostname is templated as keycloak.<host> and the realm is fixed — so backend-to-backend calls have no reason to leave the cluster. Default the value to http://keycloak-http.cozy-keycloak.svc:8080/realms/cozy, the same Service address keycloak-configure already uses. The dashboard and linstor-gui gatekeepers then run with --skip-oidc-discovery and send token, jwks, userinfo and logout requests to the Service, while browser redirects stay on the external host. Those calls travel as plaintext HTTP between pods, as keycloak-configure's admin calls to the same address already do. The value remains overridable for a Keycloak that does not live at the standard in-cluster address, and setting it back to an empty string restores external discovery. Existing OIDC clusters that never set it get a gatekeeper rollout onto the internal path on upgrade. Enabling OIDC swaps the dashboard's token-proxy container for oauth2-proxy, so the E2E install now drives that swap to completion: it waits for the auth-proxy container to appear, then gates on the gatekeeper rollout. Readiness alone proves nothing there — the HelmRelease is still Ready from the token-proxy install, and the Deployment keeps the old ReplicaSet Available while a new pod crashloops, so only the rollout catches a regression back to external discovery. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Alexey Artamonov
|
f9afc73096 |
fix(monitoring): surface missing VictoriaLogs and gate dead-end vlinsert ExternalName (#3181)
Platform log ingestion could fail silently: fluent-bit shipped to vlinsert-generic while the monitoring HelmRelease reported Ready, and a vlinsert-generic ExternalName in cozy-monitoring dead-ended at a tenant-root VictoriaLogs that did not exist when the root tenant had monitoring disabled. Logs were dropped with no failing signal. Three complementary changes: - Readiness (do not stay silently Ready): the monitoring-system HelmRelease now uses a `poller` waitStrategy and a healthCheckExpr on its VLCluster (status.updateStatus), so a missing/unhealthy log store makes the release not-ready instead of flipping Ready as soon as the CR is applied. Mirrors the seaweedfs CNPG pattern. - No empty-log-store hole: system/monitoring rejects an empty logsStorages at render time (helm fail). With no entry no VLCluster is created, the readiness gate has nothing to match, and fluent-bit would silently drop logs — so this is made a loud misconfiguration rather than a documented footgun. - No silent fallback: the cozy-monitoring -> tenant-root vlinsert/vminsert ExternalName redirects in cozystack-basics are now created only when the root tenant hosts monitoring, gated on _cluster.monitoring-enabled — fed from a new platform value monitoring.rootEnabled (default true, preserving the historical always-create behaviour). monitoring.rootEnabled is the single render-time source of truth: changing it re-renders cozystack-basics (helm-controller watches the cozystack-values Secret it flows into). A live `lookup` of Tenant/root spec.monitoring is deliberately NOT used: cozystack-basics is an operator-generated HelmRelease with drift detection off, so it re-renders only on install/upgrade/values-change — never on a reconcile tick or when a lookup result changes — and a lookup would go stale in both directions. Operators must keep monitoring.rootEnabled in sync with Tenant/root spec.monitoring. Covered by helm-unittest across all four charts: readiness gate fields, empty-vs-populated logsStorages contract, ExternalName present/absent and null-guard contract, and the platform wiring into the cozystack-values Secret. The CEL health predicates are exercised live, not in helm-unittest (which cannot evaluate CEL). Adds the missing `test:` target to packages/extra/monitoring/Makefile so its suite runs in CI. Known limitation (follow-up): the legacy cozystack.monitoring PackageSource that installs system/monitoring directly into cozy-monitoring cannot carry the readiness gate, because the engine's ComponentInstall/HelmRelease builder and the vendored helm-controller API v1.4.3 lack waitStrategy/healthCheckExprs. That source is not referenced by any active bundle; full coverage needs a helm-controller bump plus engine support. Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io> |
||
|
Aleksei Sviridkin
|
a1748009ac |
fix(ingress): scope host loadBalancerClass to the external-facing ingress
publishing.loadBalancerClass is a host-publishing setting, but its _cluster.load-balancer-class block was emitted on any ingress taking the LoadBalancer branch — including tenant ingresses — unlike expose-external-ips and proxy-protocol, which gate on the host namespace. A tenant ingress would then be claimed by an LB controller the tenant never selected. Gate it the same way, and add helm-unittest coverage for the host/tenant split and the platform to _cluster wiring. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |