TEMPORARY revert. Restoring persistent EFI/TPM state (4cc60e3) makes
windows.11/2k22/2k25 VMs get a backend-storage PVC that is ReadWriteOnce on the
default StorageClass, which pins the VM to its node and blocks live-migration and
node drains (with evictionStrategy: LiveMigrate this can stall cluster upgrades).
See #3005.
Strip persistent EFI/TPM from the preferences again, but with `yq del` so an
emptied preferredTPM stays `{}` (the v1.8 CRD rejects null) — the bug the original
sed strip caused. Secure Boot and the vTPM stay present, just non-persistent.
Stopgap until vm-state can request an RWX backend (RWX-Block via the upstream
KubeVirt change, or an RWX-Filesystem vmStateStorageClass).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <andrei.kvapil@aenix.io>
Re-vendors the coredns Helm chart from 1.43.2 to 1.46.0 (chart appVersion
1.12.3 -> 1.13.1), picking up the chart-tooling improvements (1.46 ServiceMonitor
options, 1.45 customAnnotations/dnsConfig/podLabels/dnsPolicy, 1.44 multicluster
RBAC).
This is a chart-only refresh: the running CoreDNS image is deliberately KEPT at
registry.k8s.io/coredns/coredns:v1.12.4. It is pinned to the 1.12 line that
matches the Kubernetes-bundled kube-dns replacement, and v1.12.4 is the latest
patch of that line (itself a security release). Advancing the image to the
chart's 1.13.1 appVersion (or newer) would break the deliberate alignment with
the bundled CoreDNS version and is a separate platform decision, not part of
this chart refresh. The kube-dns overrides (k8sAppLabelOverride, service.name,
serviceAccount.name) stay valid and the rendered Deployment still uses v1.12.4.
Pins the update target to --version 1.46.0 for reproducible re-vendoring.
Note: this does NOT close#2899 -- that ticket asks to advance the coredns
*image* for CVE remediation, which conflicts with the kube-dns version
alignment. Whether to move the bundled CoreDNS line forward (and thus the image)
is a platform-level call, left open on the issue.
Adaptations:
- charts/: re-vendored via make update (helm pull coredns --version 1.46.0; upstream tests dropped per the existing recipe)
- Makefile: pin update target to --version 1.46.0
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The vm-disk pvc-resize, cozystack-api, seaweedfs and vm-instance update
hooks (whose alpine/k8s image this PR refreshes) ran as root with no
securityContext. Add a pod-level runAsNonRoot + runAsUser 65534 +
seccompProfile RuntimeDefault and a container-level
allowPrivilegeEscalation: false + capabilities drop ALL, mirroring the
dashboard adopt hook. kubectl needs no root, so this is behavior-neutral.
Each hook is lookup-gated; pin the securityContext in a per-chart
helm-unittest that mocks the gating object (DaemonSet / PVC / Service) via
kubernetesProvider so the guard renders without a cluster. vm-disk and
vm-instance had no Makefile test target, so add 'test: helm unittest .' to
both so CI runs the new suites.
Scope note: other kubectl hooks on clastix/kubectl (mariadb, keycloak,
kubernetes, tenant) are untouched here — they are not part of this
alpine/k8s refresh and belong to a separate hardening pass.
Also fix the vm-disk resize hook's backoffLimit: it was nested under the
pod spec (silently ignored, leaving the Job at the default 6 retries)
instead of Job.spec. Move it to Job.spec and pin it in the test.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the nats Helm chart from 1.3.13 to 1.3.16 (app 2.11.8 -> 2.11.10),
the latest patch in the 1.3 line, picking up the published security fixes plus
upstream bug fixes (route-reconnect storms, stream interest loss when lowering
max-connections, JetStream meta performance). Pins the update target to
--version 1.3.16 so make update stays on the 1.3 line instead of jumping to the
2.x chart restructure. The cozystack overrides (PVC retention policy, cluster
routeURLs domain, prometheus exporter PodMonitor) stay valid.
Adds helm-unittest coverage (the package shipped none): pins the server image,
the PVC retention policy, and the PodMonitor invariant.
Adaptations:
- charts/: re-vendored via make update (helm pull nats --version 1.3.16)
- Makefile: pin update target to --version 1.3.16; add test target
- tests/nats_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When the platform signals shared wildcard issuance via
_cluster.wildcard-issue, the publishing controller (Release.Namespace ==
expose-ingress) renders one cert-manager Certificate for <root-host> +
*.<root-host>. The resulting Secret is served as the controller's default
SSL certificate and the existing wildcard-secret-name consumption path
makes the system services drop per-host ACME. The Certificate uses the
DNS-01 ClusterIssuer that cert-manager-issuers already renders in dns01
mode, so no per-tenant Issuer is minted. Only the publishing controller
renders it — the Secret must be same-namespace for ingress-nginx to read
it.
Part of #2811.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
On the default ingress-nginx path (gateway.enabled=false) a DNS-01 solver
still minted a per-host ACME certificate for every system service, hitting
Let's Encrypt rate limits once a deployment exceeds ~50 endpoints. Add an
opt-in publishing.certificates.wildcard toggle (default false) and compute,
in apps.yaml, an effective wildcard secret name plus a wildcard-issue
signal: when enabled with solver=dns01 and no operator-provided
wildcardSecretName, the platform targets one shared wildcard cert
(*.<root-host> + <root-host>) instead of per-host certs.
Issuance is off by default because a single-label wildcard does not cover
a custom service host outside *.<root-host> (e.g. a keycloak ingress.host,
harbor host, or grafana host pointed at another domain); enabling it would
otherwise switch such a host to the controller's default certificate and
serve the wrong certificate. The operator-provided (BYO) name always wins
and never triggers issuance; http01 cannot issue wildcards; and the
Gateway API path issues its own wildcard via the controller, so all three
are excluded. This reuses the existing wildcard-secret-name consumption
path, so no per-service ingress template changes are needed.
Part of #2811.
Signed-off-by: Aleksei Sviridkin <f@lex.la>
VPA 1.4 deprecated updateMode Auto as an alias for Recreate that will be
removed in a future API version. The vpa-for-vpa VerticalPodAutoscalers
still used Auto; switch them to the explicit Recreate (behavior-identical
today) and pin it in helm-unittest so the deprecated value cannot creep
back.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the cowboysysop vertical-pod-autoscaler chart from 10.0.0 to 11.1.1
(app 1.3.0 -> 1.5.0), moving the VPA components to registry.k8s.io/autoscaling
images at 1.5.1 and picking up the published security fixes. The values schema
keeps every cozystack override valid: the recommender's prometheus storage
config (storage, prometheus-address, container labels), the crds.enabled=false
gate, and the updater/admissionController resource blocks.
The upstream chart's 11.0 default switched the bundled CRD-install Job image to
bitnamilegacy/kubectl, but cozystack already overrides crds.image to
docker.io/alpine/k8s AND disables the Job (crds.enabled=false), so no Bitnami
image is ever pulled. VPA 1.5 deprecations (humanized memory flag, UpdateMode
Auto) are not used by cozystack.
Adds helm-unittest coverage (the package shipped none): pins the recommender's
prometheus-storage args, the non-Bitnami registry.k8s.io image, and the
crds.enabled=false -> no CRD-install Job invariant. Pins the update target's
helm pull to an explicit chart version.
Adaptations:
- charts/: re-vendored via make update (helm pull vpa --version 11.1.1)
- Makefile: pin update target to --version 11.1.1; add test target
- tests/vpa_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rewrite the SecurityGroup design doc for the targetRef projection:
selector derivation from lineage labels, the structural (not
selector-validated) RBAC boundary, and the additive-policy reality —
a SecurityGroup can only widen a managed application's allowed traffic,
never sever its platform-managed management plane, and equally cannot
tighten below the platform baseline. Records the deliberate caveats:
raw non-application pods are not targetable, and a targetRef to a
non-existent application matches zero pods rather than erroring.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update the REST storage fixtures and e2e bats to the targetRef shape, and
add cases for the new behaviour: the derived endpointSelector matchLabels,
the apiGroup default to apps.cozystack.io, an empty-targetRef rejection,
and invalid-label-value rejection (over-length or malformed kind/name/
apiGroup). The e2e asserts the dotted-key lineage labels serialise and are
accepted by the real cilium.io/v2 CRD, and that the view reconstructs the
targetRef.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Replace the tenant-authored endpointSelector on SecurityGroup with a
targetRef that references a managed application by {apiGroup, kind, name}.
The backing CiliumNetworkPolicy's endpointSelector is now derived by the
API server from the application's lineage labels
(apps.cozystack.io/application.{group,kind,name}) instead of copied from
tenant input, so the selector is machine-generated and can only ever
match the referenced application's own pods in the tenant's namespace.
The in-tree CiliumNetworkPolicy mirror gets its own spec type carrying a
concrete endpointSelector; the ingress/egress rule lists still project
1:1. Reads reconstruct the targetRef from the derived matchLabels, which
is lossless because validation caps each component at a valid label value
below the lineage webhook's truncation threshold.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the velero chart from 11.0.0 to 12.0.3 (app 1.17.0 -> 1.18.1) and
moves the backup-plugin initContainers to the matching releases:
velero-plugin-for-aws v1.12.1 -> v1.14.1 (the release that bumps to velero
1.18.1) and kubevirt-velero-plugin v0.8.0 -> v0.9.0. The chart-major bump's
values-schema changes are backward-compatible (two required fields relaxed,
one optional key added), so all cozystack overrides (upgradeCRDs,
initContainers, configuration.{features,defaultItemOperationTimeout},
deployNodeAgent) stay valid. The 1.18 "PVC selected-node" deprecation needs no
action (velero handles it transparently); the CRD changes are additive.
Adds helm-unittest coverage (the package shipped none): pins the server and
plugin image tags and the upgradeCRDs=false -> no upgrade-crds Job invariant,
which has silently regressed before. Pins the update target's helm pull to an
explicit chart version so re-vendoring is reproducible.
Note: Helm v3 does not update CRDs on `helm upgrade` (upgradeCRDs is disabled);
the 1.18 CRDs must be applied manually on existing clusters.
Adaptations:
- charts/: re-vendored via make update (helm pull velero --version 12.0.3)
- values.yaml: plugin images to velero-1.18-compatible releases
- Makefile: pin update target to --version 12.0.3; add test target
- tests/velero_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Six further references to the alpine/k8s tools image were still pinned at
1.33.4 across the installer, the cozystack-api post-upgrade hook, the vm-disk
and vm-instance resize hooks, the vertical-pod-autoscaler CRD-install override,
and the e2e cilium leak healer. Move them all to 1.36.2 so the whole repo runs
a single current tools-image version (the two digest-pinned refs get the
matching multi-arch digest sha256:44ef4942...). Same stale-stdlib remediation
as the seaweedfs bump in this PR.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bumps the alpine/k8s tools image used by the seaweedfs resize hook and the
kubectl hook job from 1.33.4 to 1.36.2 (current). This non-vendored override is
the effective runtime value; the vendored chart's stale 1.28.4 default is unused
and left untouched per the vendoring rules.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bumps the etcd metrics scrape proxy from the 2021-era v0.11.0 to v0.22.0 on the
same quay.io/brancz registry that etcd-operator already standardized on. The two
flags used here (--secure-listen-address, --upstream) are core and stable across
the version range.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Refreshes the matchbox PXE image base from the 2021-era v0.10.0 to v0.11.0,
clearing the long-fixed golang.org/x/crypto and x/net versions baked into the
old build. v0.11.0 only bumps butane (v0.17->v0.18) and adds darwin arm64
artifacts; no behavior change for Talos PXE boot.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the HAMi vGPU chart from 2.8.1 to 2.9.0, moving the device-plugin
image to projecthami/hami:v2.9.0 and picking up the published security fixes.
The vendoring approach changed for 2.9.0: upstream now gates the scheduler and
device-plugin templates on .Values.dra.enabled and .Values.podSecurityPolicy.enabled
across ~27 files, so the previous "strip the dra/PSP values and remove the
conditionals" approach would nil-pointer at render time. Instead the dra and
podSecurityPolicy blocks are KEPT at their upstream default (enabled: false),
which cleanly disables both paths; only the unused hami-dra subchart and its
values block are removed.
Replays the indent->nindent patch on the custom Service labels block, now
extended to BOTH Services that carry the upstream defect: the device-plugin
monitor Service and the scheduler Service (the latter was previously unpatched
and would emit invalid YAML if scheduler.service.labels were set).
Adds helm-unittest coverage (the package shipped none): pins the device-plugin
image and guards the nindent patch on both Services.
Adaptations:
- Makefile: pin update to --version 2.9.0; keep dra/podSecurityPolicy blocks (drop the del); add test target; refresh vendoring notes
- charts/: re-vendored HAMi 2.9.0 (hami-dra subchart removed, deps emptied)
- charts/.../device-plugin/monitorservice.yaml + scheduler/service.yaml: replay indent->nindent
- values.yaml: refresh the upstream-values reference URL to v2.9.0
- tests/hami_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
psmdb-operator evolves its CRD schema between minor versions; 1.22.0 adds
spec.storageScaling/storageAutoscaling, vault and other fields. Helm never
upgrades CRDs shipped in crds/, so without an explicit policy the operator
binary upgrades on an existing cluster while the live CRD stays behind and
the apiserver rejects the new fields. Opt the mongodb-operator PackageSource
into CreateReplace, consistent with etcd-operator, and add a platform test
pinning the policy.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Matches the managed MongoDB CR (`spec.crVersion`) to the psmdb-operator 1.22.0
bump so the platform tracks the deployed operator instead of running at the
N-1 skew. The 1.22.0 CR schema is additive (no new required fields), so
existing instances reconcile cleanly under SmartUpdate. PMM (disabled by
default) and the PBM 2.11.0 sidecar stay compatible with operator 1.22.0;
refreshing their image matrix is a separate apps/mongodb concern.
Adaptations:
- templates/mongodb.yaml: spec.crVersion 1.21.1 -> 1.22.0
- tests/mongodb_test.yaml: update crVersion assertion to 1.22.0
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the Percona Operator for MongoDB Helm chart from 1.21.2 to 1.22.0,
moving the operator image to 1.22.0 and picking up the published security
fixes. The CRD gains additive fields (native MinIO backup storage, storage
scaling) and new metrics.k8s.io RBAC. The only schema removal -- the
restore-spec `replset` string, replaced by `replsetRemapping` -- is unused by
cozystack's mongodb app (restore.yaml uses clusterName/pitr/backupSource only).
Deprecations (PSMDB 6.0, PMM2, enableVolumeExpansion) stay backward-compatible;
the watchAllNamespaces override remains valid.
Note: Helm v3 does not update CRDs on `helm upgrade`; the bumped CRD must be
applied manually on existing clusters per Percona's upgrade docs.
Adaptations:
- charts/: re-vendored via `make update` (helm pull psmdb-operator 1.22.0)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the upstream csi-driver-nfs Helm chart from 4.11.0 to 4.13.3,
picking up the published security fixes in the bundled Go toolchain that the
issue enumerates. Refreshes the CSI sidecars to current sig-storage releases
(csi-provisioner v6.1.0, csi-resizer v2.0.0, csi-snapshotter/snapshot-controller
v8.4.0, livenessprobe v2.17.0, node-driver-registrar v2.15.0) and nfsplugin to
v4.13.3.
The csi-provisioner v5->v6 and csi-resizer v1->v2 majors declare a Kubernetes
1.34 minimum for the VolumeAttributesClass API, but the chart pins
--feature-gates=VolumeAttributesClass=false on both sidecars, so they skip that
API and keep running on a sub-1.34 cluster. cozystack overrides are empty so
nothing is clobbered, and helm template renders cleanly.
Adaptations:
- charts/: re-vendored via `make update` (helm pull csi-driver-nfs 4.13.3)
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Keycloak 26.6 adds zero-downtime patch upgrades, startup/liveness probes
reporting UP during DB migrations (smoother rolling updates on Kubernetes),
and a Quarkus 3.33 LTS base. Targets 26.6.3 specifically: 26.6.0/26.6.1
shipped a migration-persistence bug (server exits code 1 after async realm
migration) that is fixed in later patches.
Adaptations:
- values.yaml: image quay.io/keycloak/keycloak 26.5.2 -> 26.6.3
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Merge the AI-generated v1.5.0 changelog with a fuller pass:
- Add an "Upgrade Notes and Required Actions" section (K8s 1.33+, removal
of upgrade.force, GPU permittedHostDevices ownership, MetalLB FRR-K8s /
HTTPS metrics, automatic TLS on externally-published services).
- Expand "Platform Components" with per-component upstream highlights and
CVE references (Flux v2.8 CVE-2026-23990, MetalLB v0.16 CVE-2025-22874),
plus the new frr-k8s subchart and kuberture package.
- Scope the storageClass-immutability entry (#2639) to UI-only enforcement,
with apiserver enforcement tracked in #2657.
- Broaden Development/Testing/CI coverage and add the v1.4.x website docs
backports (#565, #566, #567, #580).
- Correct the loop-device LVM-filter attribution to talm#215 (the prior
ansible-cozystack#51 reference was a dependency bump, not this change).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The "What TIA does and does not do" section described the old single
Build job and the `detect-changes` paths-filter. Update it for the new
`plan` job (docs gate + build matrix), note that image builds are now
diff-scoped by hack/build-matrix.sh (distinct from TIA's app-loop trim),
and point the "widen the skip" advice at the plan docs-gate.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
build-main.yaml now warms the mode=max cache on the ephemeral pool
(oracle-vm-24cpu) instead of the dedicated self-hosted runner, so no
build path depends on that host (#2937 decommission). Adds an idempotent
flux install for the installer's image-packages step and bumps the
timeout for the serial build on the ephemeral shape.
The self-hosted runner remains referenced only by the `debug`-label
breakpoint path in e2e; fully de-registering it is a separate infra step.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
build-parity.yaml (workflow_dispatch) builds both ways on the same
commit and asserts the matrix decomposition patches the same tracked
files as the legacy serial `make build` — the cutover gate for #2937.
It compares the patched file SET, not the patch bytes, because image
digests are not reproducible across two independent builds. Also probes
whether the registry accepts SBOM attestations, gating SBOM=1.
TEMPORARY — to be removed once parity is confirmed (header notes this).
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Replaces the single serial Build job (one ephemeral VM running `make
build`, ~26 images one after another, wedging on the shared buildkit
under concurrent load — #2937) with a fan-out:
plan changed files + hack/build-matrix.sh -> scoped matrix
checks unit + controller Go tests (decoupled from e2e)
build one package per job, own docker-container buildx,
uploads a per-package digest patch fragment
build-talos talos image + nocloud disk (its own leg; e2e needs it)
finalize apply all fragments, then build the installer LAST (its
flux-push bundles the whole patched tree) and emit the
unified pr.patch e2e consumes unchanged
build-matrix.sh scopes the matrix to the packages a PR touches (full set
on shared-dependency changes) and fans seaweedfs out to
objectstorage-controller. common-envs.mk gains an SBOM=1 flag wiring
buildx --sbom, kept OFF until the OCIR referrers probe confirms support.
.github/actionlint.yaml registers the custom ephemeral runner labels.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
hack/build-matrix.sh emits the CI build matrix (the list of package dirs
to build) from the root Makefile `build:` target — the single source of
truth — so the matrix can't drift from `make build`. It supports
diff-driven scoping: only units whose package dir changed are emitted,
unless a shared-dependency path (cozy-lib, the build macros, go.mod, the
build workflows) is touched, in which case the full matrix is emitted.
talos and installer are excluded from the parallel matrix: talos shares
heavy _out/assets between its image and the nocloud disk (built once in a
dedicated leg), and installer's `flux push artifact --path=packages`
bundles the whole digest-patched tree, so it must run in finalize after
all other fragments merge.
hack/build-matrix_test.bats covers full/scoped/empty selection, the
talos+installer exclusion, shared-dep escalation, and JSON/Makefile parity.
Refs #2937
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
validateCIDRs rejected a bare IP (10.0.0.1) because net.ParseCIDR
requires a prefix, but Cilium's CIDR.sanitize accepts a bare IP as a
single-host CIDR. That made the validator stricter than Cilium and
blocked a legitimate single-host fromCIDR/toCIDR rule, contrary to the
design goal of rejecting only specs Cilium would silently discard.
Mirror Cilium by falling back to a plain-address parse, and add a
regression test for bare-IP CIDR acceptance.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The resourceVersion de-dup filter dropped ADDED events whose RV was <=
the requested startingRV. During a sendInitialEvents replay the backing
WatchList intentionally re-emits existing objects as initial ADDED
events with RV <= startingRV, so the filter broke the initial-state
replay. Skip the de-dup while sendInitialEvents is set; it still applies
when resuming a plain watch from a resourceVersion. Add a regression
test covering sendInitialEvents with a non-zero starting RV.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Enforce in the REST storage that Create persists into the request
namespace and that Update/force-create targets the request name,
rejecting a conflicting metadata.namespace or metadata.name with a
BadRequest. The aggregated apiserver already rejects these before the
storage is reached, but binding them here keeps the storage
self-defending against a cross-namespace or off-target write regardless
of the calling path. Add regression tests that drive the storage
directly to lock the contract.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Describe the implemented model: SecurityGroup as a synchronous 1:1
projection of a marker-labelled CiliumNetworkPolicy served by the
aggregated API server, the marker invariant the storage enforces on every
write, the RBAC boundary it provides, the in-tree mirror that avoids the
Cilium module dependency, and the deferred attachment / cluster-scope /
validation extensions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Register the v1alpha1.sdn.cozystack.io aggregated APIService, grant the
cozystack-api ServiceAccount full access to CiliumNetworkPolicy (the
backing object for SecurityGroup), and let tenants self-service their
SecurityGroups: full access in the base tenant role and read access in
the view role, mirroring how apps.cozystack.io resources are exposed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Install the sdn.cozystack.io API group and serve the securitygroups
resource from the SecurityGroup REST storage, register the static
SecurityGroup kinds, add the group to the server codec, and register the
in-tree CiliumNetworkPolicy mirror with the client scheme.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
The aggregated API server translates each SecurityGroup into a single
CiliumNetworkPolicy in the same namespace and back. Policies owned by the
SecurityGroup API are marked with the sdn.cozystack.io/securitygroup label,
which the storage always re-asserts on every write and hides from the
SecurityGroup view, so it only ever surfaces or mutates its own policies,
leaves platform-managed policies untouched, and cannot be orphaned by a
tenant overwriting the marker through spec labels. Mutating verbs run the
genericapiserver admission hooks, and updates replace labels/annotations
rather than merging them, matching Kubernetes PUT semantics.
To avoid importing the full Cilium module — which pins a Kubernetes
version incompatible with this project's apimachinery fork — the backing
object is a minimal in-tree CiliumNetworkPolicy mirror that reuses
SecurityGroupSpec as its spec body, making the projection a near-identity
translation in both directions.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
Introduce the sdn.cozystack.io/v1alpha1 API group with the SecurityGroup
resource. SecurityGroup is a namespace-scoped, tenant-facing firewall
object whose spec mirrors the subset of the CiliumNetworkPolicy rule that
the abstraction exposes (endpointSelector, ingress, egress, ports, CIDR,
FQDN), so the resource can be projected 1:1 onto a CiliumNetworkPolicy
without granting tenants direct access to the cilium.io API group.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Co-authored-by: Timofei Larkin <lllamnyp@gmail.com>
State next to the hostFirewall default why the cozystack cilium image
deviates from upstream: with the IPv6 datapath disabled, upstream drops
all node IPv6 pre-policy, while the patched image hands IPv6 to the
kernel stack. Cilium host policies therefore apply to IPv4 only.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The image build already fails on patch drift, but only when the image
is built. Check the patch against the freshly vendored upstream tag
inside make update as well, so the author of a cilium version bump sees
the drift immediately instead of at the next image build.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Cozystack ships Cilium with the IPv6 datapath disabled
(ipv6.enabled=false) and the host firewall enabled. In this combination
upstream bpf_host.c drops all IPv6 on managed devices as unknown L3
("Unsupported L3 protocol") before any policy evaluation, in both
directions. This kills ICMPv6 Neighbor Discovery and BGP unnumbered over
link-local addresses on IPv6/L3 fabrics, and no
CiliumClusterwideNetworkPolicy can allow the traffic back because the
drop happens before policy enforcement.
Carry a BPF patch in the cozystack cilium image that adds an ETH_P_IPV6
passthrough case to the host-firewall protocol switches, mirroring the
behaviour without the host firewall: IPv6 is handed to the kernel stack
instead of dropped. IPv4 host policies remain fully enforced; node IPv6
is not filtered by Cilium.
The patch is applied with patch --fuzz=0 in a multi-stage build, so any
upstream drift in the patched context fails the image build loudly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The Release E2E workflow (`release-e2e.yaml`) ran the full bats suite on
every release tag as a non-blocking alarm — its own header notes "a
failure here is an alarm, not a release blocker." It duplicates coverage
the release-cut PR already provides: the "Prepare release" commit bakes
image digests into `packages/core/` (and other packages), which TIA
escalates to the full suite, so the release PR exercises the whole suite
against the commit the tag will point at.
It has also never been green since it was added (2026-05-02): every tag
run failed at the build step. Even after #2973 fixed the registry login
today, the next run still failed (Build Talos image). Removing a
perpetually-red, non-gating duplicate removes noise without losing
coverage.
Removes the workflow, drops `release-e2e` from `select-e2e.sh`'s
full-suite pattern, repoints its unit test to `pull-requests.yaml`, and
updates the e2e-testing doc.
Refs #2397
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
The `Release E2E` workflow's build job logged into `iad.ocir.io` using
`secrets.OCI_REGISTRY_USERNAME` / `secrets.OCI_REGISTRY_PASSWORD` —
secret names that don't exist in the repo. `docker/login-action`
therefore received empty credentials and failed at the very first step
with `Username and password required`, before any image was built or any
test ran.
The workflow has **never** succeeded since it was added (2026-05-02):
all three tag runs to date died at this step — `v1.5.0-rc.2`,
`v1.5.0-rc.3`, and `v1.5.0`.
This switches the login to the configured `OCIR_USER` / `OCIR_TOKEN`
secrets — the same ones `build-main.yaml` and `pull-requests.yaml` use
against the same registry — and adds the per-job Docker config isolation
those workflows already use (`Set up Docker config` step +
`DOCKER_CONFIG` env on the login and build steps), so the login doesn't
clobber the shared `~/.docker/config.json` on the self-hosted runner.
### Release note
```release-note
ci(tests): fix OCIR registry login in the Release E2E workflow so it can build and run the full suite on release tags
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Improved the release workflow with enhanced Docker credential
isolation and configuration handling during the build process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The Release E2E build job referenced secrets OCI_REGISTRY_USERNAME /
OCI_REGISTRY_PASSWORD, which do not exist in the repo, so
docker/login-action received empty credentials and failed every run
with "Username and password required". The workflow has never built
since it was added (2026-05-02): v1.5.0-rc.2, v1.5.0-rc.3 and v1.5.0
all died at this step before any image was built or test run.
Use the configured OCIR_USER / OCIR_TOKEN secrets, matching
build-main.yaml and pull-requests.yaml, and isolate the Docker config
per job (Set up Docker config step + DOCKER_CONFIG on the login and
build steps) so login does not clobber the shared ~/.docker/config.json
on the self-hosted runner.
Assisted-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The system chart was bumped to Harbor 2.15.1, but the user-facing app
chart still declared appVersion 2.14.2. Since the Application type serves
appVersion as a top-level field, the platform API and dashboard misreported
the deployed Harbor version. Align appVersion with the shipped 2.15.1.
Signed-off-by: Алексей Артамонов <alexeyartamonov1987@gmail.com>
## What this PR does
Implements the tenant Flux sharding design from #2817: a new
`flux-shard-operator` system package that spreads tenant HelmReleases
across multiple helm-controller shards, so a noisy tenant (e.g. a
HelmRelease stuck in infinite remediation) cannot degrade the others.
- **Placement controller** owns the tenant→shard assignment: the unit of
placement is the tenant (all of its HelmReleases share one shard label),
distribution is greedy least-loaded weighted by HelmRelease count, with
pinned tenants, threshold-gated rebalance, paced moves, and never moving
deleting tenants. Watches are metadata-only, so the controller does not
decode the helm-controller status-patch firehose. Assignments are
recorded on tenant namespaces; HelmRelease labels remain the source of
truth on restarts.
- **Shard runtime** is provisioned by the operator: `shardCount`
helm-controller Deployments cloned from the flux-aio `flux` Deployment
and sanitised, so the image and feature-gates stay version-synced
automatically. flux-aio itself is unchanged and keeps owning everything
without a shard key.
- **CREATE-only mutating webhook** (`failurePolicy: Ignore`) stamps the
tenant's shard onto every HelmRelease at admission, so each one is born
on the correct shard regardless of creation path; any miss degrades
gracefully to the placement controller.
- The hand-rolled `flux-tenants` Deployment is retired: the operator
drains it (backfill `tenants` → `shard<i>`) and deletes it once empty;
migration 44 removes a leftover only under the same guard.
Ships with `shardCount: auto` (the default): the operator sizes the
shard count from the tenant HelmRelease count — `K = clamp(ceil(H/100),
1, min(16, T))` with hysteresis (scale up past 120 HR/shard, down under
60) — so small clusters stay at one shard (≈ today's behaviour) and
large fleets shard out automatically. An integer value pins the count
explicitly.
One deviation from the design doc: the webhook/controller scope is the
`sharding.fluxcd.io/key` label rather than
`internal.cozystack.io/tenantmodule` — every tenant HelmRelease is born
with `key=tenants` (stamped by ApplicationDefinition `release.labels`
and chart templates), so the webhook rewrites it and the placement
controller continuously relabels stragglers; `tenantmodule` covers only
the 6 tenant chart modules.
### Screenshots
Not a UI change.
### Release note
```release-note
feat(fluxcd): tenant HelmReleases are now reconciled by operator-managed helm-controller shards; the flux-shard-operator balances tenants across `shardCount` instances and retires the legacy flux-tenants deployment
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced **flux-shard-operator** to shard helm-controller workloads
with deterministic tenant placement, optional rebalance controls, and
autosized shard count.
* Added a **CREATE-only mutating webhook** to stamp the correct shard
label on tenant HelmReleases.
* **Bug Fixes**
* Application updates now preserve existing **flux-shard** label
assignments to prevent unwanted shard re-stamping.
* **Chores**
* Added platform packaging for the new component, including an image
build step.
* Removed the legacy tenants deployment and added a migration to retire
it safely.
* **Tests**
* Expanded unit and chart tests for placement, provisioning, webhook
behavior, and configuration parsing.
* **Documentation**
* Added operator documentation describing configuration and metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
The `Build cache (main)` workflow (`build-main.yaml`, added in #2948)
failed on
its first run:
https://github.com/cozystack/cozystack/actions/runs/27929277423
The `Set up Buildx` step ran without `DOCKER_CONFIG`, so
`docker/setup-buildx-action` created the docker-container builder under
the
runner's default `~/.docker`. The build step runs with
`DOCKER_CONFIG=$RUNNER_TEMP/.docker` (seeded earlier, holding the OCIR
auth), so
buildx couldn't find the builder and failed immediately:
```
ERROR: no builder "builder-6fb46242-…" found
```
The post-job `docker buildx rm` still succeeded — because the action's
cleanup
runs under the default config where the builder actually lived — which
confirms
the builder existed but in the wrong config store.
This sets `DOCKER_CONFIG` on the `Set up Buildx` step so the builder
instance is
created in the same config the build step reads.
Impact: only the main cache-warmer was broken. PR builds `--cache-from`
this
ref, so a missing cache is a cold build, not a failure — but the
warm-cache
speedup from #2948 wasn't being delivered until this fix.
### Release note
```release-note
NONE
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Improved build pipeline stability by ensuring consistent Docker
configuration across workflow steps. This enhances the reliability of
the container build and deployment process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
During node reprovisioning in tenant Kubernetes clusters, the VMI (and
its hot-plug pod) can outlive the parent VirtualMachine. The upstream
kubevirt-csi-driver treats a missing VM as "detach succeeded", so the
hot-plug pod stays alive and keeps the infra storage device exclusively
attached to the source host — every subsequent attach of that volume
fails, on DRBD with `failed to set source device readwrite`. This was
hit repeatedly in production tenant clusters (see
kubevirt/csi-driver#182).
This PR:
- updates the pinned `kubevirt.io/csi-driver` module to the latest
upstream commit;
- backports kubevirt/csi-driver#184 into the controller wrapper: after
the upstream unpublish returns success, the wrapper re-checks
`VMI.status.volumeStatus` and, if the hot-plug is still reported there
(e.g. because the parent VM is already gone), detaches it from the VMI
directly via the `removevolume` subresource and waits until it
disappears.
The backport is self-contained in the wrapper and can be dropped once
kubevirt/csi-driver#184 merges upstream and the module pin is bumped
past it.
Credit to @mattia-eleuteri for the root-cause analysis and the upstream
fix.
### Screenshots
Not a UI change.
### Release note
```release-note
fix(csi): tenant CSI driver no longer leaves orphan hot-plug volumes exclusively attached to the source host when the VMI outlives its deleted VirtualMachine during node reprovisioning
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced hotplug volume unpublishing with automatic verification to
ensure volumes are fully detached before completion, including retry
logic with up to 2-minute timeout.
* **Dependencies**
* Updated KubeVirt API and CSI driver dependencies to latest versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
