mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-09 05:25:14 +09:00
7157158c937e331dee48dfd9ee5edd61ff6c4c3a
3884
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Myasnikov Daniil
|
7157158c93 |
fix(seaweedfs-db): make db resources configurable
values.yaml declared an unused Resources typedef while templates/database.yaml hardcoded cpu/memory limits and requests. The wrapper chart at packages/extra/seaweedfs/templates/seaweedfs-db.yaml already passes db.resources through cozy-lib.resources.defaultingSanitize into the sub-chart values, so the user-facing db.resources and db.resourcesPreset on the outer chart silently dropped on the way in. Drop the unused typedef, accept .Values.db.resources as a free-form object (the wrapper already sanitizes it into the K8s nested limits/requests shape that CNPG Cluster expects), and wire it through to the Cluster manifest. Defaults are preserved verbatim so helm template output is byte-identical for a no-override install. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
dccdeb5229 |
fix(tenant): bump parent HR timeout to 15m to cover seaweedfs split
PR #2619 split seaweedfs into a serialized chain
(seaweedfs-db -> seaweedfs-system -> seaweedfs) that needs ~5 min of
wall-clock to install: CNPG bootstrap, master quorum, then the wrapper.
With the prior 5m wait budget on the parent tenant HR, helm-controller
would time out before the children reach Ready, rollback would delete
the child HRs, and the upgrade would retry in a loop. The e2e bats step
that patches the Tenant CR and waits 60s for child HRs to exist landed
in the rollback gap and timed out.
Two complementary changes:
- tenant-root is created statically by cozystack-basics, so its
HelmRelease spec.timeout is bumped 5m -> 15m directly.
- Sub-tenants (Tenant CRs) get their HR built by cozystack-api from
the tenant ApplicationDefinition. Add the
release.cozystack.io/helm-install-timeout=15m annotation that
kubernetes-rd already uses to opt the kind into the longer budget.
Keeps both values in lockstep and documents the coupling inline.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov DaniilandClaude Opus 4.7
|
29c6afc84d |
refactor(seaweedfs): split seaweedfs-system into seaweedfs-db + seaweedfs-system
Splits the seaweedfs-system HelmRelease in two so the CNPG Cluster/seaweedfs-db lives in its own HR (seaweedfs-db), and the application HR (seaweedfs-system) dependsOn it. The new HR uses Flux v2 HelmRelease.spec.healthCheckExprs with a CEL expression on Cluster.status.conditions[type=Ready] plus waitStrategy.name: poller, so its Ready=True only flips after the postgres primary is actually serving connections — not just after helm install applied the Cluster CR. This eliminates the seaweedfs-filer CrashLoopBackOff race on a fresh tenant install. With Cilium kubeProxyReplacement: true, socket-LB returns EPERM from connect(2) to ClusterIPs with no Ready endpoints. Pre-split, the filer StatefulSet scheduled concurrently with the CNPG bootstrap (~55–70s of unavailable postgres), each connect() failed EPERM, kubelet exponential restart backoff pushed past the e2e bats 'kubectl wait hr/seaweedfs-system --timeout=2m' window, and the 'Configure Tenant and wait for applications' test failed. healthCheckExprs uses Flux's three-predicate form: route ClusterIsNotReady (the CNPG bootstrap condition) to `inProgress` so the HR keeps polling, keep `failed` for other Ready=False reasons, and rely on the HR `timeout: 10m` as the real backstop for a genuinely stuck cluster. A plain Ready=False predicate would flag the HR failed within seconds of creating the Cluster CR. Migration 42 adopts existing Cluster/seaweedfs-db resources into the new release on upgrade by rewriting meta.helm.sh/release-name and stamping helm.sh/resource-policy: keep so the seaweedfs-system upgrade (which no longer renders the Cluster) does not delete it during the transition. The application HR's dependsOn collapses the db dependency (unconditional) with the existing ingress dependency (guarded by `if eq $ingress .Release.Namespace` so sub-tenants that inherit ingress from a parent namespace don't deadlock on a non-existent local HR) into one list. Requires Flux v2.8.x (helm-controller v1.5.0+) for healthCheckExprs. Provided by the parent PR #2602 in this chain. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
401993ac62 |
test(e2e): wait for parent HR Ready before downstream asserts
Two timing bugs unmasked by Flux v2.8 kstatus, with the same root shape:
the bats steps assumed a side-effect ordering ("by the time X is true,
the parent HR's helm install is done") that held under v2.7's faster
dispatch but breaks under v2.8 where helm-controller walks kstatus on
every applied resource before flipping the HR Ready.
run-kubernetes.sh:
After every addon HR (cilium, coredns, csi, vsnap-crd, ingress-nginx)
reaches Ready, the remediation guard reads `.status.history` on the
parent kubernetes-${test_name} HR. The parent's helm install can still
be Reconciling at that point because kstatus is finishing its full
sweep; `.status.history` is empty until install completes, so the
guard exits 1 with "empty .status.history on a Ready HR". kubernetes-
latest masked this because the ouroboros assertions add ~10m of
buffer; kubernetes-previous (no ouroboros) hit the guard with the
parent still installing. Add an explicit
kubectl wait hr/kubernetes-${test_name} --for=condition=ready
before the guard.
vminstance.bats:
The test waited 120s for `kubectl get vmi vm-instance-$name` to
return a VMI with an assigned IP, then waited for the parent HR Ready
separately afterwards. Under v2.7 the parent HR's helm install
applied the VirtualMachine CR fast enough that kubevirt had created
the VMI before the 120s budget expired. Under v2.8 the dispatch is
slower and the VMI poll returned NotFound for its entire budget.
Reorder: wait for HR Ready first (which implies the VirtualMachine
CR is applied + kstatus-ready), then poll for the VMI with IP.
Loosen the VMI poll budget from 120s to 5m for nested-virt slow paths.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
7b9f286faa |
test(e2e): bump app HR-Ready waits for Flux v2.8 kstatus
Flux helm-controller v1.5.0 (shipped in Flux v2.8) replaced the legacy
Helm readiness logic with kstatus-based health checking, which polls
Deployment/StatefulSet/Service status and waits for kstatus to assess
rollout-ready. This typically adds 10-30s of latency between resource
apply and HR condition Ready=True compared to v2.7.
Our per-app bats tests had `kubectl wait hr/... --for=condition=ready
--timeout=` budgets of 20s, 30s, 60s, 100s — sized for v2.7's faster
readiness flip and previously masked by the 3x retry on Run E2E tests
(dropped in #2558). Under v2.8 with no retry, every app that runs after
the first few hit its HR wait timeout exactly, surfacing as 13 of 16
test failures on PR #2619.
Standardise on 5m for HR Ready across all app tests. Tight enough to
catch genuine bring-up failures within the per-app job budget, loose
enough to absorb kstatus dispatch jitter on a management cluster that
is concurrently reconciling multiple tenant-test HRs.
Post-HR-Ready checks for app-specific resources (StatefulSet/Service/
PVC/operator CRs) remain — for CR-based apps kstatus only observes
that the chart applied, not that the operator finished. Audit of
which of those are actually redundant under v2.8 kstatus is queued
as a follow-up.
Also bumps the 10s "kamajicontrolplane appears" wait in
run-kubernetes.sh to 2m, and the kubernetes-addon HR Ready loop from
1m to 5m for the same reason.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
f6c7379835 |
fix(kubernetes): drop lookup-guarded parent-HR dependsOn on tenant addon child HRs
Each of the 19 child HR templates in packages/apps/kubernetes/templates/
helmreleases/ added the parent kubernetes-application HR as a dependsOn
entry, guarded by a `lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease"`
on the parent so it only stamped after the parent already existed.
Under Flux 2.7 (helm-controller v1.4.x), the parent HR's Ready=True flips
after helm install returns, regardless of any child HRs it just applied,
so the guard was a no-op for first install and a soft ordering hint for
subsequent reconciles.
Under Flux 2.8 (helm-controller v1.5.0), the default waitStrategy is
`poller` -- the parent HR now waits via kstatus for every applied
resource, including the 19 child HRs, to be Ready=True before flipping
its own Ready. The child HRs in turn cannot become Ready until they have
a working `super-admin.svc` kubeConfig and the lookup-stamped dependsOn
back-edge resolves -- which only happens after the parent HR is Ready.
Deadlock: parent waits on child, child waits on parent.
Removing the parent dependsOn from each child breaks the cycle. The
children start reconciling immediately; helm-controller's existing retry
loop covers the brief window before the parent's helm install lands the
admin-kubeconfig secret. The cross-child ordering (cilium -> cert-manager,
prometheus-operator-crds -> metrics-server, etc.) stays intact via the
unconditional sibling entries.
Five files (the *-crds and vsnap-crd HRs) had only the parent reference
under dependsOn; remove \`dependsOn:\` entirely in those. Cilium had one
conditional gateway-api-crds entry under it; pull the \`dependsOn:\` key
inside that conditional so it disappears when gateway-API is off rather
than rendering as a null array.
This is option B from issue #2642 task 1.
Verified:
- \`helm template\` clean with default values-ci.yaml and with every addon
enabled (cert-manager, ingress-nginx, monitoring-agents, velero, fluxcd,
gateway-api, gpu-operator, hami, ouroboros).
- No remaining \`- name: {{ .Release.Name }}\` dependsOn entries in
packages/apps/kubernetes/templates/helmreleases/.
- \`helm unittest\` passes (123/123).
Refs: #2642
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
435b3e6275 |
fix(vpa): break circular wait between parent install and nested vpa-for-vpa HR
The packages/system/vertical-pod-autoscaler chart's vpa-for-vpa.yaml template
emits an HR in cozy-vpa-for-vpa that the parent helm install applies and waits
on. That nested HR had `dependsOn: monitoring-agents`, which never resolves:
parent VPA helm install
waits for nested vpa-for-vpa HR Ready
waits for monitoring-agents HR Ready
waits for cozystack.vertical-pod-autoscaler PackageSource Ready
waits for parent VPA HR Ready <-- back to start
Symptom observed on PR #2619 e2e (cozyreport):
Helm upgrade failed for release cozy-vertical-pod-autoscaler/
vertical-pod-autoscaler with chart cozy-vertical-pod-autoscaler@...:
timeout waiting for: [HelmRelease/cozy-vpa-for-vpa/vpa-for-vpa
status: 'InProgress']
The recommender's /health-check readiness probe is served by the binary
itself and doesn't require Prometheus. Removing the dependsOn lets the
nested HR become Ready quickly; the recommender will log connection
errors against vmselect until monitoring-agents installs later in the
boot sequence, then start producing recommendations normally.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
2e3e7bdb39 |
fix(vm-instance): emit empty disk object so KubeVirt accepts disks without bus
When a disk entry has no bus set (the schema default), the previous
template rendered `disk:` with no value — YAML parses that as
`disk: null` and KubeVirt's virtualmachines-mutator webhook denies the
request with: "spec.template.spec.domain.devices.disks.disk in body
must be of type object: null".
Restructure the disk loop so the disk/cdrom value is always a valid
object: `disk: {}` when bus is empty, `disk: {bus: <x>}` when set.
Mirrors how `cloudinitdisk` already does it in the same template.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
7f94182a87 |
fix(platform): drop deprecated upgrade.force from HelmReleases; fix kafka WorkloadMonitor replicas paths
Flux v2.8 / helm-controller v1.5.0 embeds Helm v4, which renamed --force
to --force-replace and rejects combining it with SSA's --force-conflicts.
Existing HelmReleases with spec.upgrade.force: true fail every upgrade with
"cannot use force conflicts and force replace together", looping
upgrade -> rollback -> upgrade and blocking dependants. Observed on PR #2619
CI for tenant-root/{ingress-nginx-system,seaweedfs-db}, tenant-test/info,
and others.
Per Flux docs, upgrade.force has been a no-op since v2.8 anyway, so removing
it is safe on green clusters and unblocks broken ones.
Also fix packages/apps/kafka/templates/workloadmonitor.yaml: it referenced
.Values.replicas (no such field; values.yaml has kafka.replicas and
zookeeper.replicas). Flux v2.7 silently dropped the null; v2.8 SSA forwards
it, and the WorkloadMonitor CRD rejects "spec.replicas: null" as not
type:integer, which blocks every kafka-test install.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
7cb7be8124 |
fix(kafka): place enableServiceLinks under template.pod, not a phantom .spec
Same class of bug as the foundationdb misplaced-field fixes. The kafka
chart had:
entityOperator:
template:
pod:
metadata: { labels: { ... } }
spec: ← child of template, not of pod
enableServiceLinks: false
The Strimzi Kafka CRD's `entityOperator.template` has no `spec` key.
`enableServiceLinks` is a direct field of `template.pod` (alongside
metadata, affinity, tolerations, etc.). Flux v2.7's permissive
client-side patch silently dropped the misplaced field; v2.8's strict
Server-Side Apply rejects:
failed to create typed patch object (Kind=Kafka):
.spec.entityOperator.template.spec: field not declared in schema
Move `enableServiceLinks` under `template.pod` where the operator
actually reads it, and fix the inconsistent metadata.labels indentation
(was 9/11 spaces instead of 10/12) while we're here.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov DaniilandClaude Opus 4.7
|
5121c7b91b |
fix(foundationdb): relocate misplaced fields from automationOptions to spec
The chart nested `faultDomain`, `imageType`, `labels`, and `minimumUptimeSecondsForBounce` inside `automationOptions`, but the FoundationDBCluster CRD schema defines all four as direct children of `spec`. Flux v2.7 (helm-controller v1.4.3) used permissive client-side patch and silently dropped the unknown fields; v2.8 (helm-controller v1.5.0) does strict Server-Side Apply and rejects: failed to create typed patch object: .spec.automationOptions.<field>: field not declared in schema This was a latent bug from day one — none of those fields ever actually configured anything. Moving them one level up applies the values to where the operator looks for them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
4cc60e3ab5 |
fix(kubevirt-instancetypes): drop persistent strip that produced null TPM
`make update` stripped `persistent: true` from upstream's
`preferredTPM: { persistent: true }`, leaving the parent key with a null value.
KubeVirt v1.6's CRD tolerated it; v1.8 (the operator version we bumped to in
|
||
|
Myasnikov Daniil
|
d1ad59509d |
feat(flux): regenerate embedded management-cluster manifests for Flux 2.8.0
Re-render internal/fluxinstall/manifests/fluxcd.yaml via `make update` in packages/core/flux-aio (timoni bundle build against oci://ghcr.io/stefanprodan/modules/flux-aio at the latest tag) so the cozystack-operator binary embeds Flux 2.8.0 instead of 2.7.3. Auto-syncs the helm-controller image into fluxcd-tenants.yaml. Embedded versions now match the chart bump in the prior commit: flux: v2.7.3 -> v2.8.0 helm-controller: v1.4.3 -> v1.5.0 (enables healthCheckExprs) source-controller: v1.7.3 -> v1.8.0 kustomize-controller: v1.7.2 -> v1.8.0 notification-controller:v1.7.4 -> v1.8.0 source-watcher: v2.0.2 -> v2.1.0 Verified by rebuilding cmd/cozystack-operator and running `strings` on the binary -- the expected v1.5.0 / v1.8.0 / v2.1.0 image strings are now embedded in place of the v1.4.x / v1.7.x ones. Structure preserved: - single AIO Deployment named "flux" in namespace cozy-fluxcd - all 5 controllers + source-watcher as containers in one pod - custom tolerations from flux-aio.cue intact - Namespace pod-security label still "privileged" via the Makefile yq post-processing step Net diff is -3664 lines (mostly CRD schema simplification in 2.8). Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
9aa8710ef2 |
feat(fluxcd): bump flux-operator/instance to v0.48.0, web UI opt-in
Bump flux-operator and flux-instance vendored charts from v0.33.0 to v0.48.0, moving the Flux distribution from 2.7.x to 2.8.x. This brings helm-controller v1.5.x, which exposes spec.install.healthCheckExprs on HelmRelease — a prerequisite for adding real readiness gating to dependency-target packages (postgres, mongodb, kafka, etc.). The new flux-operator chart defaults web.enabled=true; override it to false in packages/system/fluxcd-operator/values.yaml so the Flux Status Web UI stays opt-in. Operators wanting it can set web.enabled=true and configure web.config / web.configSecretName / web.ingress per the upstream chart values. Patches reapplied: - kubernetesEnvs.diff: matched cleanly (offset shift only) - networkPolicy.diff: extended to also restrict port 9080 (web UI port); hunk count updated 20 -> 22 Pre-flight: no references to the v0.39-removed --disable-wait-interruption flag anywhere in packages/ or internal/. Verified locally: - helm template renders both packages without errors - web.enabled=false (default): no web Deployment/Service/NetworkPolicy - web.enabled=true: renders web NetworkPolicy on port 9080 - CiliumClusterwideNetworkPolicy lists 8080/8081/9080 - FluxInstance distribution.version: "2.8.x", all 7 components present - Cozystack kustomize patches (concurrent, requeue, storage-adv-addr, events-addr) intact Notable upstream changes between v0.33 and v0.48: - Flux 2.8.x with HelmRelease inventory format change (auto-migrated) - Reliability fixes for stuck FluxInstance/ResourceSet on health-check cancellation (v0.39, v0.46) - New CLI: flux-operator migrate, diff, patch instance, distro mirror - ResourceSetInputProvider: GitLab Environments, GitHub App, Gitea/Forgejo - CVE-2026-23990 (Web UI OIDC impersonation bypass) — N/A while web is off Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
8afb99b282 |
docs(release): add changelog for v1.4.2 (#2759)
This PR adds the changelog for release `v1.4.2`.
✅ Changelog has been automatically generated in
`docs/changelogs/v1.4.2.md`.
|
||
|
cozystack-ci[bot]
|
6270c728eb |
docs: add changelog for v1.4.2
Signed-off-by: cozystack-ci[bot] <274107086+cozystack-ci[bot]@users.noreply.github.com> |
||
|
myasnikovdaniil
|
74de393e1a |
test(e2e): ouroboros fold + CSI/NFS ordering + OIDC keycloakInternalUrl (#2728)
## What this PR does Bundles four independent e2e improvements lifted from #2619: 1. **`test(e2e): fold ouroboros assertions into kubernetes-latest`** — moves the standalone `hack/e2e-apps/ouroboros.bats` (which provisioned its own full Kamaji control plane, ~25m) into the existing `kubernetes-latest` run as an optional addon. The assertions cover ouroboros HR Ready, controller pod Running, `Ingress → coredns-custom` rewrite-line injection, and end-to-end DNS resolution from inside the tenant. Reuses the existing tenant cluster instead of building a second one. 2. **`test(e2e): wait for tenant CSI HR Ready before NFS test`** — explicit `kubectl wait hr/kubernetes-${test_name}-csi --for=condition=ready` ahead of the NFS pod. Otherwise the NFS pod schedules while the `kubevirt-csi-node` DaemonSet is still rolling out, eats ~1m on `FailedAttachVolume` retries, and trips the pod-Succeeded wait. 3. **`test(e2e): bump NFS pod wait to 10m to fit slow-path chain`** — was 5m. With dependencies properly serialized (per #2), 5m was tight on slow runners; 10m gives the chain genuine room without masking real bugs. 4. **`test(e2e): set keycloakInternalUrl when enabling OIDC`** — the dashboard gatekeeper (oauth2-proxy) does OIDC discovery against the keycloak Service URL; without an explicit `keycloakInternalUrl`, it falls back to the public ingress URL which isn't routable from inside the cluster yet during install. ## Origin All four commits lifted unchanged from #2619. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Integrated Ouroboros validations into the main Kubernetes end-to-end flow (removed separate standalone Ouroboros test) * Added hairpin-NAT and in-tenant DNS resolution checks for Ouroboros scenarios * Improved Keycloak OIDC test by configuring an in-cluster internal URL * Increased various timeouts and readiness waits and added tenant CSI readiness gating to improve test reliability <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2728?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
6b76db6929 |
fix(platform): add cert-manager dependency to webhook-cert consumers (#2726)
## What this PR does Adds `cert-manager` as a `dependsOn` entry to the five PackageSources whose charts apply `cert-manager.io/v1` `Certificate` CRs and need the webhook ready before their first reconcile. Without this, the helm install renders Certificate manifests, the cert-manager webhook isn't admitting requests yet (cold cluster), the apply fails, and helm-controller retries until the webhook eventually comes up — burning ~2-3 minutes per affected package on cold installs. The five PackageSources: `capi-operator`, `cozystack-engine`, `ingress-nginx`, `kamaji`, and `rabbitmq-operator` (the platform `cozy-system` set with webhook-cert consumers; not the apps layer). In `cozystack-engine`, the dependency is added to both the `default` and `oidc` variants — the `oidc` variant installs the same `cozystack-api` and `lineage-controller-webhook` components that render Certificate resources, so it had the same race. ## Origin First commit lifted unchanged from #2619. Second commit (`oidc` variant) added in response to review feedback. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Chores** * Updated package source dependencies across multiple components to ensure proper installation sequencing. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2726?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
78b9740094 |
feat(dashboard): add cluster-usage RBAC for the new admin page (#2743)
## What this PR does Adds a new `ClusterRole` `cozystack-dashboard-cluster-usage` and its `ClusterRoleBinding` under `packages/system/dashboard/templates/`, plus the matching helm-unittest fixture. The role grants the `cozystack-cluster-admin` group cluster-wide read on `nodes`, `pods` (get/list/watch) and on `metrics.k8s.io` `nodes` (get/list). Companion to the upcoming UI work in cozystack/cozystack-ui#15 — a new admin page `Console → Administration → Cluster Usage` that renders cluster-scoped resource utilisation and a per-node table including arbitrary extended resources (GPUs and other accelerators). The page-side sidebar item is gated by a `SelfSubjectAccessReview` on `nodes list`; without this binding the gate fails closed and the entry stays invisible, which is the desired fallback for clusters that have not adopted the new RBAC yet. The built-in `edit` `ClusterRole` already bound to `cozystack-cluster-admin` via `cozystack-dashboard-cluster-admin` does not grant cluster-wide pod read or any access to `nodes`, so an explicit grant is required for the page to render anything beyond an error. This chart intentionally does not add permissions for deeper sources (Prometheus, DCGM, HAMi). Those belong in their respective packages or in operator-provided RBAC; the page detects them via discovery and gates each section with its own per-source review. ### Screenshots No UI in this PR — RBAC only. ### Release note ```release-note feat(dashboard): grant cluster-admin read access to nodes, pods and metrics.k8s.io for the new Cluster Usage admin page ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Dashboard cluster usage monitoring now includes configured role-based access control with appropriate read permissions for nodes, pods, and metrics data. * **Tests** * Added comprehensive test suite to validate the dashboard's role-based access control configuration, including verification of proper permission bindings and authorization rules. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2743?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
ea7a8c8675 |
test(e2e): quote HelmRelease names in kubectl wait
Both \${test_name} and \${component} are literal-safe today (no
whitespace, no glob characters), so this is a style-only change — but
quoting variable expansions in kubectl arguments matches the rest of
the file and removes a future-foot-gun if a test name ever grows a
hyphen-y suffix that interacts oddly with completion or globbing.
Scoped to the two HelmRelease wait lines flagged by CodeRabbit and
echoed by the human reviewer; the machinedeployment wait on the
preceding line keeps its existing style.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
64c443caf1 |
test(e2e): split local k8s_version declaration from yq invocation
\`local k8s_version=\$(yq ...)\` masks the exit status of the command substitution — the assignment to a local variable always succeeds, so a failing yq is swallowed. Declare the local first, then assign, so non-zero yq exits propagate via shell errexit semantics if the script ever runs under \`set -e\`. Pure shell hygiene; no behavioral change on the success path. Surfaced by CodeRabbit and confirmed by the human reviewer as non-blocking but worth applying. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
8c6e149177 |
test(e2e): delete standalone ouroboros.bats now folded into kubernetes-latest
The previous commit (3c81261b) folded the ouroboros hairpin-NAT reconciliation assertions into the kubernetes-latest tenant under run-kubernetes.sh, but did not remove the standalone bats file. The e2e workflow auto-discovers every hack/e2e-apps/*.bats and dispatches test-apps-<name> via the wildcard rule in packages/core/testing/Makefile. Leaving ouroboros.bats in the tree meant the assertions ran twice: once on the standalone ~25m Kamaji-backed tenant and again as the folded addon on the kubernetes-latest tenant — the opposite of the consolidation the fold was intended to deliver. The folded assertions in hack/e2e-apps/run-kubernetes.sh cover the same surface (HR Ready, controller pod Running, coredns-custom rewrite-line injection, and end-to-end DNS resolution from inside the tenant), so the standalone file is dead weight after the fold. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
3ecc030a64 |
test(e2e): set keycloakInternalUrl when enabling OIDC
The dashboard gatekeeper (oauth2-proxy) does OIDC discovery against the
external host at container startup. The e2e sandbox uses
host=example.org which does not resolve, so the auth-proxy crashloops:
Get "https://keycloak.example.org/realms/cozy/.well-known/openid-configuration":
dial tcp: lookup keycloak.example.org: no such host
Under Flux v2.8 kstatus the gatekeeper Deployment then flips to
'Failed' (rather than the pre-v2.8 'InProgress') and stalls the
cozy-dashboard/dashboard HelmRelease, which in turn blocks
cozy-fluxcd/flux-plunger via dependsOn. Previous runs masked this with
an unrelated ImagePullBackOff on cozy-dashboard-console; with the
console image fixed the OIDC discovery failure now surfaces.
packages/core/platform/values.yaml:194-198 already declares
authentication.oidc.keycloakInternalUrl. When set, the dashboard
chart (packages/system/dashboard/templates/gatekeeper.yaml:59-66)
renders --skip-oidc-discovery plus explicit redeem/jwks/validate/
backend-logout URLs pointing at the in-cluster keycloak Service.
Browser-facing redirect-url and login-url stay on the external host
(they are not exercised by the e2e flow).
The patch sets the value alongside oidc.enabled in the same
kubectl-patch so the configmap and gatekeeper rollout happen in one
reconcile pass.
Observed on CI run 26142986969 (PR #2619).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
68a9a6e5ad |
test(e2e): bump NFS pod wait to 10m to fit slow-path chain
Previous run failed kubernetes-latest (and then kubernetes-previous via
cascade) on the 5m budget for nfs-test-pod to reach Succeeded. The
chain is host linstor prime PVC + CDI import + tenant CSI attach +
busybox image pull + containerd CreateContainer (which occasionally
stalls past kubelet's 2m CRI deadline and retries with name-reservation
conflict). ~40s on the happy path, 6m+ in observed worst case.
When kubernetes-latest times out, its tenant cluster (2x 20Gi worker
PVCs) leaks because set -e exits before any cleanup. kubernetes-
previous then runs, adds another 40Gi, and the host tenant-quota
(100Gi) is exhausted -- every downstream app test (mariadb, mongo,
postgres, qdrant, redis, openbao, vminstance) fails with
ErrCreatingPVCPrime / exceeded quota.
Bumping the wait to 10m gives the slow path room. Headroom costs
nothing on green runs since the pod is busybox echo.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
04a54b01cf |
test(e2e): wait for tenant CSI HR Ready before NFS test
kubernetes-previous fails on the NFS pod step. From cozyreport + job log:
- Pod scheduled to tenant worker
- 7x FailedAttachVolume over 35s: CSINode does not contain driver
csi.kubevirt.io (kubevirt-csi-node DaemonSet still rolling out)
- Attach succeeds at T+1m08s after the driver registers
- busybox pull queued ~1m25s, completes at T+2m48s
- CreateContainer in containerd then stalls past kubelet's CRI
deadline; retry trips "failed to reserve container name ... is
reserved for <prev>" -- the 5m pod-Succeeded wait then expires.
Root cause: run-kubernetes.sh creates the NFS PVC+Pod before asserting
kubernetes-${test_name}-csi is Ready. That HR lives in the parent
cluster but installs cozy-kubevirt-csi-node into the tenant via
kubeConfig.secretRef; helm-controller walks kstatus on the tenant
DaemonSet using the tenant kubeconfig. HR Ready in parent therefore
implies kubevirt-csi-node Pods Ready on every tenant worker, which
implies csi.kubevirt.io is published in CSINode. The existing wait at
line 354 runs AFTER the NFS test, so it does not gate the race.
Move the wait earlier with a 10m budget. Post-NFS sibling waits stay
as is, so component coverage is unchanged.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
5035ac1414 |
test(e2e): fold ouroboros assertions into kubernetes-latest
The standalone hack/e2e-apps/ouroboros.bats provisioned a full Kamaji-
backed tenant Kubernetes cluster (control plane + 2 CAPI workers + base
addons) just to flip `addons.ouroboros.enabled=true` on top, then ran
~15m of hairpin-NAT reconciliation assertions. Cluster shape was
identical to what hack/e2e-apps/run-kubernetes.sh already provisions for
kubernetes-latest, so the second Kamaji bringup was pure duplication —
roughly 25-45m of CI wall-time per run.
Add an `enable_ouroboros` 4th parameter to `run_kubernetes_test()`. When
"true", inject the ouroboros addon block into the Kubernetes CR's
addons map and run the post-cluster assertions inline:
- Wait for the ouroboros HR Ready in the host namespace.
- Verify coredns-custom ConfigMap exists in tenant kube-system.
- Wait for the ouroboros controller pod Ready.
- Apply an Ingress with a TLS host in tenant default namespace.
- Poll coredns coredns-custom for the rewrite line (5m budget).
- Resolve the hairpin host from inside the tenant via a netshoot pod
and assert it points at the ouroboros-proxy ClusterIP.
- Clean up dnscheck pod + Ingress.
kubernetes-latest passes "true" (gets ouroboros coverage); kubernetes-
previous leaves it empty (control-plane-only smoke remains).
(The standalone ouroboros.bats was removed in the preceding commit
alongside an unrelated foundationdb chart fix — the git rm staging had
already happened when the foundationdb commit landed.)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
Myasnikov Daniil
|
535f881160 |
fix(platform): include cert-manager in cozystack-engine oidc variant
The oidc variant installs system/cozystack-api and system/lineage-controller-webhook, both of which render cert-manager.io/v1 Certificate resources. Add cozystack.cert-manager to the oidc variant's dependsOn so it matches the default variant and webhook-cert consumers do not race the cert-manager controller on cold installs. Address review feedback from gemini-code-assist and coderabbitai on packages/core/platform/sources/cozystack-engine.yaml. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
69cb7b5f18 |
fix(platform): add cert-manager dependency to webhook-cert consumers
Five PackageSources whose charts apply cert-manager Certificate CRs and
mount the resulting Secret on a webhook Deployment lacked an explicit
dependsOn: cozystack.cert-manager. Under Flux v2.8 helm-controller v1.5.0
kstatus, the dependent Deployment trips progressDeadlineSeconds (10m
default) when cert-manager controller isn't running yet, and the HR is
failed early. Repeatedly observed on PR #2619 with capi-operator,
kamaji, rabbitmq-operator, cozystack-api (cozystack-engine), and
ingress-nginx all cascading to InstallFailed at ~12-15m into a fresh
install.
The mariadb-operator, kubeovn-webhook, snapshot-controller, etcd-operator,
postgres-operator, mongodb-operator, victoria-metrics-operator, linstor,
and linstor-scheduler PackageSources already had the dependency, which
is why they weren't in the failure set.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
(cherry picked from commit
|
||
|
myasnikovdaniil
|
628ebce653 |
feat(platform): add deletion-protection guardrail via ValidatingAdmissionPolicy (#2650)
## What this PR does Adds a `ValidatingAdmissionPolicy` that blocks `DELETE` on critical platform objects labeled `platform.cozystack.io/no-delete=true`. Native, in-process CEL evaluation — no webhook DaemonSet, no Service, no TLS, no image to maintain. Requires Kubernetes 1.30+. This is an alternative to the closed #2402 (admission webhook approach). ### Protected objects (labeled in this PR) | Resource | Kind | Source | |---|---|---| | `cozy-system` | Namespace | `packages/core/installer/templates/cozy-system-{namespace,labels}.yaml` | | `tenant-root` | Namespace | `packages/system/cozystack-basics/templates/tenant-root.yaml` | | `tenant-root` | HelmRelease (ns: tenant-root) | same | | `cozystack-version` | ConfigMap (ns: cozy-system) | `packages/core/platform/templates/cozystack-version.yaml` | | `cozystack-packages` | OCIRepository (ns: cozy-system) | `packages/core/platform/templates/repository.yaml` | | `letsencrypt-prod`, `letsencrypt-stage`, `selfsigned-cluster-issuer` | ClusterIssuer | `packages/system/cert-manager-issuers/templates/cluster-issuers.yaml` | | `linstorcluster` | LinstorCluster | `packages/system/linstor/templates/cluster.yaml` | | `packages.cozystack.io`, `packagesources.cozystack.io` | CRD | label stamped in `internal/crdinstall/install.go` | ### Bypass ```bash kubectl label <kind> <name> platform.cozystack.io/no-delete- kubectl delete <kind> <name> ``` ### Why VAP instead of a webhook Same outcome as #2402 but with zero runtime infrastructure: no DaemonSet, Service, TLS certificate, or image. The kube-apiserver evaluates the CEL `expression: "false"` in-process and denies because `validationActions: [Deny]`. `objectSelector` on the binding scopes evaluation to labeled objects only — every other DELETE in the cluster is unaffected. ### Release note ```release-note feat(platform): add deletion-protection guardrail (ValidatingAdmissionPolicy) blocking DELETE on cozystack platform objects labeled `platform.cozystack.io/no-delete=true`. Bypass by removing the label first. Requires Kubernetes 1.30+. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Deletion-protection guardrail: objects labeled platform.cozystack.io/no-delete=true are denied DELETE requests; removal of the label allows deletion. * **Chores** * Platform migration to v43 backfills the no-delete label onto system resources and namespaces. * **Tools** * Added an "unprotect" teardown utility to remove the no-delete label for uninstall/disaster recovery. * **Documentation** * Helm NOTES now warns when the guardrail is inactive due to missing cluster API. * **Tests** * New tests and e2e checks validate deletion-protection behavior. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2650?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Timofei Larkin
|
72c4eb9c08 |
feat(backups): implement Job backup strategy (#1721)
## What Implements the `strategy.backups.cozystack.io/Job` backup strategy — a generic, application-agnostic driver that runs a user-supplied Kubernetes Job to create (and restore) a backup. ## Background This PR began as a draft written against an earlier shape of the backups API. The API has since moved to a `BackupClass`-based dispatch model, which broke the original draft on three counts: - `BackupJob.Spec.StrategyRef` and `BackupJob.Spec.StorageRef` were removed; the strategy and its parameters are now resolved from a `BackupClass` into a `ResolvedBackupConfig`. - Strategy handling moved from standalone per-strategy reconcilers to a dispatch model: `BackupJobReconciler.Reconcile` / `RestoreJobReconciler.Reconcile` resolve the strategy and switch into per-strategy methods (`reconcileVelero`, `reconcileCNPG`, `reconcileAltinity`, …). - The draft's template context (`.Values`, `.Storage`) diverged from the convention the other strategies adopted, and its `.Storage` half targeted a storage API that does not exist. ## What changed - **`reconcileJob` / `reconcileJobRestore`** (`internal/backupcontroller/jobstrategy_controller.go`): replaces the standalone `BackupJobStrategyReconciler` with the two dispatch methods the `BackupJob` / `RestoreJob` reconcilers call. The Job strategy is the generic counterpart of the Altinity driver — same lifecycle (render `PodTemplateSpec` → run a one-shot `batch/v1.Job` → poll its terminal condition → emit a `Backup`), minus the ClickHouse-specific `applicationRef` gate, so it runs for any application. Shared package helpers (`renderJobTemplate`, `jobConditionState`, `getApplicationUnstructured`, status helpers, deterministic Job naming, controller refs) are reused. - **Template context** follows the in-tree convention used by the other drivers: `.Application` (live application object), `.Release.Name` / `.Release.Namespace`, `.Mode` (`backup` / `restore`), `.Parameters` (resolved `BackupClass` parameters), and `.Backup` (restore runs only). Strategy parameters round-trip through the `Backup`'s `driverMetadata` so a later restore re-renders against the same values. - **Restore** re-renders the same template with `.Mode == "restore"`. Cross-namespace restore is unsupported (the RestoreJob target reference is namespace-local), matching the Altinity driver. - **`JobSpec.Template` doc + regenerated CRD**: the field doc previously advertised `.Values`; it now documents the real context, and `strategy.backups.cozystack.io_jobs.yaml` is regenerated to match (the change is limited to that field's description). - **Tests** (`jobstrategy_controller_test.go`): backup-Job creation / args / labels / status, completion → `Backup` artifact with parameter round-trip, Job-failure surfacing, missing-strategy terminal failure, restore into the target namespace, restore completion, and the parameter-prefix unit. A deliberately generic application kind pins that the driver never gates on `applicationRef.kind`. No RBAC changes are required: the controller already holds `strategy.backups.cozystack.io/*`, `batch/jobs`, `apps.cozystack.io/*`, and `backups` permissions. ## Notes for reviewers - Backup and restore share one template, so a strategy author's `PodTemplateSpec` must branch on `.Mode` to do anything restore-specific. This matches the existing convention; worth confirming it's the intended contract for the Job strategy's docs/examples. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Implemented full backup/restore job strategy with deterministic Job creation, idempotent execution, and status/condition updates; added demo example scripts and reusable shell helpers for a NATS JetStream backup/restore workflow. * **Documentation** * Clarified Job template rendering context and listed supported template variables; added README and step-by-step scenario docs for the NATS example. * **Tests** * Added comprehensive unit tests covering job creation, completion/failure handling, parameter extraction, and unmappable-target failure cases. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/1721?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
b630706df6 |
feat(apps): mark stateful-app storageClass fields as immutable (#2639)
## Summary Annotate `storageClass` as immutable across every stateful chart that exposes one: clickhouse, foundationdb, harbor, http-cache, kafka (Kafka + ZooKeeper), kubernetes (top-level + nodeGroups[]), mariadb, mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq, redis, vm-disk. The PVC-pinning argument is identical for every chart — Kubernetes fixes `storageClassName` on each PVC at creation, so editing this field on an existing resource never migrated data; the schema annotation makes that contract explicit and lets schema-aware consumers refuse the edit up-front. Schema-aware consumers today: the Cozystack UI (cozystack/cozystack-ui#6) reads the rule from the chart's openAPISchema and renders the field read-only on edit forms. This PR is **schema-side only**. The cozystack aggregated apiserver does not yet evaluate CEL rules in `openAPISchema` (`MariaDB` / `Postgres` / etc. are application-backed, not CRDs); tracking issue cozystack/cozystack#2657 covers wiring CEL into `pkg/registry/apps/application/rest.go::Update`. Until that lands, an apiserver-level `kubectl patch` that changes `storageClass` is still accepted — but the underlying StatefulSet PVCs do not move, so there's no data-corruption risk, only a confusing UX. The UI catches that case today; the apiserver gate closes it permanently. ## What changes For each app in the list above: - `packages/apps/<app>/values.yaml` — gain a `## @immutable` directive directly above (or below, for fields inside a `@typedef`) the `storageClass` line. - Regenerated via `make generate` in each package: - `api/apps/v1alpha1/<app>/types.go` — kubebuilder `XValidation` marker on the relevant field(s). - `packages/apps/<app>/values.schema.json` — `x-kubernetes-validations: [{rule: "self == oldSelf", message: …}]` on `storageClass`. - `packages/apps/<app>/README.md` — regenerated parameter table. - `packages/system/<app>-rd/cozyrds/<app>.yaml` — the embedded `openAPISchema` on the ApplicationDefinition reflects the change. Refactor: `pkg/registry/apps/application/rest.go` factors the inline schema-build path into a new `buildSpecSchema` helper so it can be unit-tested without standing up a fake apiserver. Three regression tests in `pkg/registry/apps/application/rest_spec_schema_test.go` pin the contract that `x-kubernetes-validations` survives the v1→internal→structural conversion (test loads the actual MariaDB ApplicationDefinition fixture on disk, asserts the rule reaches the structural schema). Empty-input and malformed-JSON paths are also pinned. No e2e admission test is added because the apiserver does not yet enforce the rule — see cozystack/cozystack#2657. No changelog entry: schema-side annotation is invisible at the apiserver layer today (UI-only consumer) and the release-cut PR will add the entry when admission enforcement lands. ## Dependencies Requires cozystack/cozyvalues-gen#24 to be merged and released, and the `cozyvalues-gen` binary pin in `.github/workflows/pre-commit.yml` line 50 bumped to the release that includes that PR. Pre-commit CI will fail until both happen — the stock `v1.3.0` binary silently ignores `## @immutable` and regenerates artefacts without the rule, producing a drift against what's committed. Once that ordering is satisfied, rebase this branch and the pre-commit job will be green. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * storageClass is now treated as immutable after creation for many stateful apps (enforced by chart/CRD schema validation). * **Documentation** * Added a storage immutability guide explaining the contract and enforcement. * Updated chart READMEs and values docs to mark storageClass as immutable and clarified nodeGroup storage remains intentionally mutable. * **Tests** * Added tests validating schema parsing and preservation of immutability rules. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2639?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
myasnikovdaniil
|
d3f282cee1 |
test(e2e): pre-pull platform images via staged-busybox DaemonSet (#2724)
## What this PR does Pre-pulls every image referenced by `packages/system/kubeovn`, `packages/system/linstor`, and `packages/system/cert-manager` onto every node before those HelmReleases install. Cluster-member workloads (OVN raft, LINSTOR, cert-manager webhook) fail when replicas start at different times due to per-node image-pull stagger; pre-pulling means all replicas start with images already cached. ## Mechanics Renders each chart with `helm template`, then runs a `yq` filter that walks every PodSpec-shaped object and emits the images of each container — scoping the output to images the kubelet actually pulls (skips configmap fields and CRD examples that happen to contain an `image:` key). Each render is staged through a tmp file so a helm-template failure trips `set -e` cleanly (cozytest.sh runs `@test` bodies under `/bin/sh`, which is dash on Ubuntu CI — no `pipefail`). The pre-pull DaemonSet itself is the hard part: distroless images (cert-manager and friends) ship no shell and no `/bin/sleep`, so the obvious `command: ["sleep", "infinity"]` fails with `exec: ENOENT`. The script stages a statically-linked `busybox:musl` into a shared `emptyDir` from an initContainer; every prepull container then execs `/shared/sleep` regardless of what the image itself ships. The glibc `busybox` tag is unusable — it needs `/lib64/ld-linux-x86-64.so.2` which distroless lacks. ## Origin Lifted from #2619 — squash of `test(e2e): pre-pull cert-manager images` → POSIX rewrite → cert-manager removal (failed on distroless) → reintroduction with the busybox helper that solves the distroless problem. Final state only — the four iterative commits aren't review-friendly individually. ## Verification - `sh -n hack/e2e-prepull-images.sh` clean - All chart-referenced images held Ready under the DaemonSet on local kind v1.33.1 single-node ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Improved end-to-end test reliability for platform image pre-pulling by rendering charts separately and consolidating extracted images. * Simplified and consolidated readiness waits during tenant configuration for more robust application readiness checks. * **Chores** * Enhanced image pre-pulling to reliably handle distroless images using a staged helper binary and shared volume approach. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2724?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
5af5962edf |
feat(apps): mark stateful-app storageClass fields as immutable
Annotate storageClass as immutable across every stateful app whose chart exposes one — clickhouse, foundationdb, harbor, http-cache, kafka (Kafka + ZooKeeper), kubernetes (top-level only), mariadb, mongodb, nats, openbao, opensearch, postgres, qdrant, rabbitmq, redis, vm-disk. Without coverage across the catalogue the UI would render storageClass read-only on some apps and editable on others — a worse mental model than uniform behaviour. The annotation lands at the schema layer (CEL XValidation rule 'self == oldSelf' on values.schema.json / types.go / embedded openAPISchema in cozyrds/<app>.yaml). The cozystack aggregated apiserver does not yet evaluate CEL on application-backed resources; that is tracked at cozystack/cozystack#2657. The UI (cozystack/cozystack-ui#6) consumes the marker today. The PVC-pinning argument is identical for every chart: Kubernetes fixes a PVC's storageClassName at creation time, so editing this field on an existing resource never migrated data. The schema annotation makes that contract explicit. kubernetes.nodeGroups[name].storageClass is intentionally NOT annotated: the field is optional and undefaulted, so a strict 'self == oldSelf' rule would block any future attempt to set it on an existing node group. A richer ratcheting form is needed there — see docs/storage-immutability.md. A refactor in pkg/registry/apps/application/rest.go extracts the inline schema-build path into buildSpecSchema so it can be unit-tested. Three regression tests in rest_spec_schema_test.go pin the contract that x-kubernetes-validations survives the v1->internal->structural conversion (loaded from the actual MariaDB ApplicationDefinition fixture on disk, not a hand-edited mock). Each per-app README gains a one-line blockquote pointing to a new docs/storage-immutability.md that explains the contract and which consumers enforce it today. Requires cozystack/cozyvalues-gen with @immutable support — bump the pre-commit pin in .github/workflows/pre-commit.yml once the release exists. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
12ac2adec9 |
chore: retrigger CI
Empty commit to fire the Pull Request workflow, which never ran for this commit because GitHub Actions was down during the original push. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov DaniilandClaude Opus 4.7
|
a58c0d193b |
chore: retrigger CI
Empty commit to fire the Pull Request workflow, which never ran for 3bac5cac because GitHub Actions was down during the original push. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Andrei Kvapil
|
7e92265823 |
chore(img): refresh Cozystack logo and add icon/stacked variants (#2746)
## Summary
- Replaces `img/cozystack-logo-black.svg` and
`img/cozystack-logo-white.svg` with the updated Cozystack brand mark
(icon + "COZYSTACK" wordmark).
- Adds new layouts so the logo can be used on more surfaces without
re-cropping:
- `cozystack-logo-stacked-{black,white}.svg` — icon stacked over the
wordmark
- `cozystack-icon-{black,white}.svg` — icon-only mark
- All SVGs are exported on a transparent background, so the same file
works on light and dark surfaces.
## Files
| Path | State | Notes |
|---|---|---|
| `img/cozystack-logo-black.svg` | replaced | horizontal wordmark, black
fill, transparent bg |
| `img/cozystack-logo-white.svg` | replaced | horizontal wordmark, white
fill, transparent bg |
| `img/cozystack-logo-stacked-black.svg` | new | icon over wordmark,
black fill, transparent bg |
| `img/cozystack-logo-stacked-white.svg` | new | icon over wordmark,
white fill, transparent bg |
| `img/cozystack-icon-black.svg` | new | icon only, black fill,
transparent bg |
| `img/cozystack-icon-white.svg` | new | icon only, white fill,
transparent bg |
## Test plan
- [ ] Verify rendering of `cozystack-logo-{black,white}.svg` in the
README on both light and dark GitHub themes.
- [ ] Confirm no other in-repo references to `img/cozystack-logo-*.svg`
need width/height adjustments (existing aspect ratio ~2.8:1 → new ~3:1,
very similar).
|
||
|
Viktoriia Kvapil
|
6451373251 |
chore(img): refresh Cozystack logo and add icon/stacked variants
Replace the old "COZYΞTACK" wordmark with the updated brand mark
(icon + "COZYSTACK" wordmark) and add additional layouts:
- cozystack-logo-{black,white}.svg horizontal wordmark (replaced)
- cozystack-logo-stacked-{black,white}.svg icon over wordmark (new)
- cozystack-icon-{black,white}.svg icon only (new)
All variants use a transparent background so they work on both light and
dark surfaces. SVGs are optimized with https://autocrop.cncf.io while
keeping the original padding.
Signed-off-by: Viktoriia Kvapil <bisunviktoria@gmail.com>
|
||
|
Timofei Larkin
|
727343d259 |
docs(backups): add NATS Job-strategy example
Add examples/backups/nats/, the first worked example of the generic Job
backup strategy. It backs up a NATS JetStream stream that has no dedicated
driver, using a stock natsio/nats-box image: 'nats stream backup' -> tar ->
curl --aws-sigv4 PUT to the COSI/seaweedfs S3 endpoint, and the reverse for
restore. One PodTemplateSpec serves both directions, branching at runtime on
{{ .Mode }} (the engine templates each string leaf independently and cannot
restructure the Pod per mode).
Mirrors the clickhouse example's structure: numbered scripts (bucket -> NATS
src + seeded stream -> strategy -> BackupClass -> BackupJob -> in-place and
to-copy RestoreJob), shared helpers, run-all.sh, README, and scenario docs.
Like every examples/backups/* dir it is a runnable demo, not CI-gated.
Signed-off-by: Timofei Larkin <lllamnyp@gmail.com>
|
||
|
Timofei Larkin
|
5029af6d83 |
test(backups): harden Job strategy tests
Address review feedback on the Job backup strategy: - Assert .Application is templated: the rendered backup Job now reads .Application.spec.replicas, the load-bearing context key that lets a generic backup tool read live app spec (previously only .Release, .Mode, .Parameters and .Backup had coverage). - Pin failure-path messages so a reordered error path can't pass on the wrong branch: the missing-strategy test asserts the strategy name, and the unmappable-kind tests assert "kind not registered". - Document why the full Status().Update in the success branches is safe (the object is freshly Get() each pass; the StartedAt write requeues). - Note the fixed batch/v1.Job backoffLimit=2 and link the follow-up to make it configurable (#2742). Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Aleksei Sviridkin
|
adddd4f6e1 |
feat(dashboard): add cluster-usage RBAC for the new admin page
The cluster-usage admin page in the dashboard reads cluster-scoped resources (nodes, pods, metrics.k8s.io) to render aggregate utilisation and a per-node table that includes arbitrary extended resources such as GPUs. The built-in 'edit' ClusterRole already bound to cozystack-cluster-admin via cozystack-dashboard-cluster-admin does not grant cluster-wide pod read or any access to 'nodes', so the dashboard needs an explicit grant for the page to render anything beyond an error. The page-side sidebar item is gated by SelfSubjectAccessReview on 'nodes list' — without this binding the gate fails closed and the entry stays invisible, which is the desired fallback for clusters that have not adopted the new RBAC yet. The chart intentionally does not add permissions for deeper sources (Prometheus, DCGM, HAMi). Those belong in their respective packages or operator-provided RBAC; the page detects them via discovery and gates each section with its own per-source review. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la> |
||
|
Timofei LarkinandClaude Opus 4.7
|
3bac5caac1 |
fix(platform): remove duplicate test target in Makefile
The deletion-protection commit added a second, identical 'test:' recipe to the platform Makefile, causing GNU Make to print 'overriding recipe' warnings on every 'make test' (and thus in helm-unit-tests CI output). Drop the duplicate; keep the original target. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: Timofei Larkin <lllamnyp@gmail.com> |
||
|
Myasnikov Daniil
|
2ace581117 |
test(e2e): stage prepull image list to avoid yq pipefail trap
Without pipefail (dash-CI), `yq | hack/e2e-prepull-images.sh` would let a yq failure produce empty input; prepull-images.sh exits 0 on no-input and the test passes with pre-pull silently skipped. Stage the yq output to a tmp file and feed it via stdin so set -e trips on a yq failure. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
myasnikovdaniil
|
2fa15ce044 |
fix: startupProbes for kubevirt-cdi-operator and grafana-operator (#2725)
## What this PR does Adds `startupProbe` to two operator deployments whose readiness probes fail repeatedly during a cold cluster install, generating noise and (for CDI) blocking dependants on slow leader-election: - **`kubevirt-cdi-operator`** — leader-election against a slow `kube-apiserver` (Kamaji control plane bringup) consistently takes >30s on a cold install. The default readinessProbe with 1s `periodSeconds` / 3 `failureThreshold` fires the first time election times out, the operator pod gets marked NotReady, and dependants block on it. - **`grafana-operator`** — bootstrap CRD discovery + Helm-managed Dashboard reconcile loop spikes startup duration past readinessProbe's default budget under load. `startupProbe` gives each operator a longer one-time window to reach steady state; the regular `readinessProbe` then takes over with its tight per-second cadence to detect genuine ongoing failures. ## Origin Both commits lifted unchanged from #2619. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Configuration Updates** * Added startup health checks to Grafana operator container * Added startup health checks to KubeVirt CDI operator container <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2725?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Aleksei Sviridkin
|
edb3eeb011 |
fix(info): use root-host for Keycloak OIDC issuer URL in tenant kubeconfig (#2704)
## What this PR does The kubeconfig generated by the `info` package for non-root tenants pointed `--oidc-issuer-url` at `keycloak.<TENANT>.<ROOT-DOMAIN>`, but Keycloak's ingress and cert live at `keycloak.<ROOT-DOMAIN>` (built from `_cluster.root-host`). `kubectl oidc-login` then failed TLS verification against the nginx-ingress default fake cert, so the auto-generated kubeconfig was unusable for any non-root tenant. `$host` was being sourced from `_namespace.host` (tenant-specific subdomain) with a lookup-based override that tried to read `tenant-root`'s `spec.values.host`. That override has been dead since #1787 migrated `tenant-root` to `valuesFrom` (Secret-backed) — `spec.values` is now always empty, so the override never fires. Source `$host` from `_cluster.root-host` directly, matching what `packages/system/dashboard/templates/gatekeeper.yaml` already does for the same Keycloak realm. Verified with `helm template tenant1 packages/extra/info -n tenant-tenant1 --set-string _namespace.host=tenant1.example.org --set-string _cluster.root-host=example.org ...` — issuer URL is now `https://keycloak.example.org/realms/cozy` regardless of the per-namespace host. ### Release note ```release-note fix(info): use root-host for the Keycloak OIDC issuer URL in tenant kubeconfigs so `kubectl oidc-login` succeeds against the real Keycloak cert on non-root tenants ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated kubeconfig generation to consolidate host configuration handling, now using a single authoritative source instead of multiple fallback mechanisms. <!-- review_stack_entry_start --> [](https://app.coderabbit.ai/change-stack/cozystack/cozystack/pull/2704?utm_source=github_walkthrough&utm_medium=github&utm_campaign=change_stack) <!-- review_stack_entry_end --> <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|
Myasnikov Daniil
|
76fd5d1791 |
fix(platform): address lllamnyp review on PR #2650
unprotect.sh: add `gitrepository` to the namespaced-kind list so the
platform stays unprotectable when an operator configures
`sourceRef.kind: GitRepository`. Currently only `ocirepository` is
swept, while repository.yaml templates `kind: {{ $sourceRef.kind }}`
unrestricted.
run-migrations.sh: replace the silent skip on a missing migration file
with a hard error. All migrations 1..targetVersion exist in the shipped
image — a gap can only mean a packaging mistake (digest advanced
without the file landing), which previously stalled the cluster at a
stale version while reporting success.
migrations/42: drop the misleading "no 3-way-merge subtlety" claim from
the header. The apply records the no-delete label under the default
field manager's last-applied annotation, so a future label-less apply
by the same field manager would strip it. Document the constraint
instead of denying it.
hack/e2e-install-cozystack.bats: restore `helm.sh/resource-policy: keep`
on the reconstructed cozystack-version ConfigMap in the bypass-path
teardown so cluster state matches the chart template shape
(packages/core/platform/templates/cozystack-version.yaml) on exit.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
||
|
Myasnikov Daniil
|
2bed6eb090 |
fix(platform): do not swallow kubectl errors in unprotect.sh
The blanket `2>/dev/null || true` suppression masked RBAC, wrong-context, and API-unreachable failures, letting the script print "Done." while leaving labels in place. Replace with a helper that only tolerates the "unknown resource type" error (the actual best-effort case — kinds that don't exist on every cluster, e.g. linstorcluster on a non-LINSTOR install) and propagates everything else. Also keep stdout (resource list) and stderr (kubectl warnings) separate so deprecation warnings can't sneak into the loop variable. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
c2f1bb2eac |
feat(platform): make unprotect strips no-delete label for teardown
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: with this PR every protected object — CRDs, ClusterIssuers, cozy-system and tenant-root Namespaces, cozystack-version ConfigMap, cozystack-packages Repository, tenant-root HelmRelease, LinstorCluster, plus the VAP and Binding themselves — carries the no-delete label. `helm uninstall cozy-platform` issues DELETE on each, and DELETE is denied for every one of them, so the chart effectively cannot be uninstalled without manual `kubectl label` against each object. Disaster recovery and dev-cluster reset both regress as a result. Add packages/core/platform/hack/unprotect.sh: enumerates the guarded set locally (per-kind, not via a single all-resources sweep — some kinds may not exist on every install, e.g. linstorcluster on a non-LINSTOR cluster, and kubectl errors on unknown kinds). For cluster-scoped kinds, lists objects with the label and labels them off one at a time. For namespaced kinds, lists (namespace, name) pairs via jsonpath and labels each in the right namespace. Idempotent: a label that is already absent is a no-op. Wire `make unprotect` in packages/core/platform/Makefile so the operator runs it before `helm uninstall`. The Makefile target carries a comment noting this is teardown / disaster-recovery only — after it runs, the cluster has no guardrail until the next `helm upgrade` re-stamps the label. Verified with a stub kubectl in PATH that the script issues the right get/label calls for both cluster-scoped (e.g. `kubectl label customresourcedefinition/packages.cozystack.io platform.cozystack.io/ no-delete-`) and namespaced kinds (e.g. `kubectl label --namespace tenant-root helmrelease tenant-root platform.cozystack.io/no-delete-`). The template-header reference at templates/deletion-protection.yaml that forward-links to `make unprotect` was added in the preceding L3 commit; the recipe it points at now exists. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
91aae36da3 |
feat(platform): NOTES.txt warns when no-delete VAP is silently skipped
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: on a pre-1.30 cluster the capability gate (`.Capabilities.APIVersions.Has "admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy"`) evaluates to false and the policy + Binding render to nothing. There was no signal in `helm install`/`helm upgrade` output that the guardrail had been silently skipped, so operators on older clusters lost the protection without any way to discover that from the install log. Compounded the integration-test gap @lllamnyp called out in blocker 3. Add packages/core/platform/templates/NOTES.txt that renders a multi-line WARNING whenever the capability is absent: states the guardrail is NOT active, explains why (1.30+ required), notes that the label is still stamped on every protected object so the guardrail becomes active automatically on the next upgrade after a cluster bump, no migration step required. Add packages/core/platform/tests/notes_capability_gate_test.yaml: positive case asserts the warning text + the 1.30 hint + the label string render on a chart-level `capabilities.apiVersions: []`. Negative case asserts the warning is absent when admissionregistration.k8s.io/v1/ ValidatingAdmissionPolicy is in the capability list. Verified the positive case fails when the warning text is removed from NOTES.txt. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
919a8a9d05 |
docs(platform): clarify sequential self-protection on no-delete VAP
Address review feedback from @lexfrei on packages/core/platform/templates/ deletion-protection.yaml: both the ValidatingAdmissionPolicy and the ValidatingAdmissionPolicyBinding carry the no-delete label so each individually requires the bypass dance, but the labeling is symmetric, not truly bilateral. Removing the label from the Binding and deleting it first leaves the VAP labeled but unenforced, so the VAP itself is then deletable without the bypass — and vice versa. That is by design: the guardrail is operational guidance, not adversarial defense against an operator with cluster-admin who is determined to remove it. Add a paragraph in the template comment header so a future reader does not waste time wondering why the symmetric self-protection isn't truly bilateral. Forward-references the whole-chart teardown recipe (make unprotect) so the comment links to the documented escape hatch. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com> |
||
|
Myasnikov Daniil
|
155c011dd5 |
test(platform): e2e check that no-delete VAP denies + bypass works
Address review feedback from @lllamnyp on packages/core/platform/templates/
deletion-protection.yaml: the VAP contract had no integration coverage.
A unit test that greps the rendered manifest is the wrong shape — the
contract is "the api-server denies a labeled DELETE with the documented
message" and that needs a real apiserver in the loop.
Add a post-install bats test to hack/e2e-install-cozystack.bats that runs
once cozystack is installed on the e2e cluster:
1. Skips on clusters that pre-date Kubernetes 1.30 (no VAP API).
2. Asserts the cozystack-version ConfigMap actually carries
platform.cozystack.io/no-delete=true — precondition, so the deny
assertion below would not misreport on a regressed binding.
3. Asserts `kubectl delete configmap cozystack-version -n cozy-system`
exits non-zero AND its error message contains the documented
"Deletion blocked: ... platform.cozystack.io/no-delete=true"
substring AND the `--namespace` bypass hint.
4. Confirms the ConfigMap survives the rejected delete.
5. Removes the label, deletes (must succeed), recreates the ConfigMap
with the original data.version and the label so the cluster ends
the test in the same state it started.
This single test catches every regression the PR is meant to prevent:
capability gate inverted, binding objectSelector mistyped,
validationActions flipped Deny→Warn, expression flipped false→true,
label-key drift between the binding and the manifests, and the bypass
docs missing --namespace.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
|
