The Pull Request Build job authenticates only to the push registry (OCIR),
so every base image (golang, ubuntu, alpine, nginx, ...) is pulled from
Docker Hub anonymously. Those pulls intermittently fail at
auth.docker.io/token under Docker Hub's anonymous-pull rate limit, leaving
the base image unresolved and failing make build.
Build on a docker-container buildx builder whose buildkitd routes docker.io
through mirror.gcr.io, Google's public pull-through cache for Docker Hub: no
auth, far higher limits, transparent fall-through to Docker Hub on a miss.
This reuses the builder pattern already proven in build-main.yaml (same
make build, same DOCKER_CONFIG auth forwarding); only the committed
hack/buildkitd.toml mirror config is new.
Fail-safe: if mirror.gcr.io is unreachable, BuildKit falls back to
registry-1.docker.io and the build still succeeds, so this strictly reduces
build failures without adding a new hard dependency.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Under node CPU contention the linstor-controller JVM cannot answer its GET /health liveness probe within the piraeus-operator defaults (timeoutSeconds 1, failureThreshold 3). The kubelet then liveness-kills the container; the resulting crash-loop leaves the linstor-controller Service with no ready endpoints, linstor-csi gets EPERM, and RWX-Filesystem DataVolumes never bind, breaking tenant provisioning.
The operator-generated controller pod sets no resources (no CPU request, hence no scheduling/CPU guarantee) and ships the tight default probe. Add CPU/memory requests and relax the liveness timing via the controller podTemplate, which piraeus-operator merges into the Deployment as a strategic merge patch. The probe override carries only the timing fields; the httpGet target (GET /health on the api port) is left to the operator to avoid pinning a wrong path or scheme. Requests-only (no limits) keeps the JVM from being throttled or OOM-killed. The knobs are exposed under controller.* for tuning to observed RSS.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The operator Deployment ships a single replica but always passes
--leader-elect to the manager. On one replica leader election adds no HA
value: a transient apiserver/etcd lease-renewal blip makes
controller-runtime self-terminate (os.Exit on "leader election lost"),
which crashloops the manager and surfaces at install time as a
Deployment stuck in Failed.
Gate --leader-elect on the replica count via a manager.replicaCount
value (default 1). The single-replica install now rides out
control-plane latency instead of crashlooping, and scaling to more than
one replica re-enables leader election automatically for genuine HA.
Also set the Deployment strategy to Recreate on a single replica: with
leader election off, a default RollingUpdate would briefly run two
managers (maxSurge) during an operator upgrade, both reconciling without
lease coordination. Recreate closes that window at zero availability
cost, while installs scaled past one replica keep RollingUpdate.
The change to the vendored opensearch-operator 2.8.0 chart is carried in
patches/leaderElection.diff and re-applied by 'make update' so it
survives a chart refresh.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Every *-application PackageSource that ships a system/*-rd component
renders an ApplicationDefinition (cozystack.io/v1alpha1). That CRD is
installed by the application-definition-crd component of
cozystack-engine. Without an explicit dependsOn edge, a fresh install
where Flux reconciles a *-application source before cozystack-engine
fails to render with `no matches for kind "ApplicationDefinition" in
version "cozystack.io/v1alpha1"`, which cascades into a broad install
timeout unrelated to whatever change triggered the reconcile.
Add `- cozystack.cozystack-engine` to the dependsOn of every source that
ships a system/*-rd component; gateway-application already carried the
edge for the TenantGateway CRD. Add a generic invariant test that fails
if any such source is missing the edge, so future sources cannot
reintroduce the race.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The reconciled, hasListenIPsForAllPods, fullReplication and healthy gates
polled status fields with a 60s ceiling, while the strictly-earlier
health.available gate had 300s. The operator stamps generations.reconciled
only after every process holds its role for minimumUptimeSecondsForBounce
(60s), making reconciled and the health fields that depend on it the last
status fields to converge. The short ceiling closed the gate seconds before
the field landed on a healthy, fully-reconciled cluster, causing intermittent
jsonpath poll timeouts.
Raise those four gates to 300s to match the health.available budget. The waits
stay event-driven; only the ceilings widen.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
backupstrategy-controller creates the cozy-backups Bucket into the
tenant-root namespace, which cozystack-basics provisions. On a fresh
install the Bucket patch can race namespace creation and fail with
`namespaces "tenant-root" not found`, which then races the install
wait deadline and fails the run — on changes that have nothing to do
with backup ordering. Add the cozystack-basics dependsOn edge so the
target namespace exists first, pinned with a helm-unittest regression
guard.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The "Bootstrap Talos cluster" step times out the etcd-members health gate
at 180s. A 3-node etcd converges via Talos's serialized learner promotion
(one learner admitted at a time), so on a loaded ephemeral runner this
deterministic path can approach ~3m wall clock and photo-finish against the
180s ceiling — failing a cluster that is in fact healthy. The wait is
already event-driven; only the ceiling was too tight. Widen it to 300s.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Set the executable bit on migration 45 to match its siblings (44, 46). Replace
"PR #2610" references in migrations 45/46 with a description of the change.
Correct migration 45's comment: the KubeadmConfig children CAPI spawns from the
template are owned by their Machine (not Helm), so they carry no
app.kubernetes.io/managed-by=Helm label, are not matched by the selector, and
are never at risk of Helm pruning.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Add runAsNonRoot/runAsUser/seccompProfile to the bootstrap-token and
talos-reconcile hook Jobs so they satisfy PodSecurity "restricted" (matching the
csr-signer sidecar). Drop the keyEncipherment usage from the Ed25519 trustd
Certificate (RFC 9295 - Ed25519 keys cannot do key encipherment). Correct the
stale talos-pki comment that claimed the reconcile hook patches spec.ipAddresses:
the cert carries DNS SANs only and workers route to trustd via the Service DNS
name, as the reconcile job's own routing note explains.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Document two behaviour changes from the Phase 1 Talos rollover in the package
Breaking Changes: the worker MachineHealthCheck maxUnhealthy moved from a
hard-coded 0 (remediation effectively off) to nodeHealthCheck.maxUnhealthy
defaulting to "50%" (remediation enabled), and that GitOps-managed tenant
Kubernetes CRs must bump spec.version to v1.31 in Git or migration 46's patch is
reverted on the next source reconcile. Also fix the supported version range in
the intro (1.31-1.35; v1.30 was removed from the support matrix).
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
The kamaji source up to the 26.6.4-edge tag pins Go modules that carry
known advisories compiled into the manager binary: golang.org/x/net
below 0.55.0, golang.org/x/crypto below 0.52.0 and
go.opentelemetry.io/otel/sdk below 1.43.0. Unlike the distroless
OS-package findings, these advisories ship inside the Go binary and are
reported against the deployed image.
Force the fixed module versions during the build, after applying the
source patches and before compiling, so the binary embeds them. No edge
tag ships the fixes, and the tags that advance these modules also bring
the k8s 1.36 kubeadm change that regresses tenant control-plane RBAC, so
a source version bump is not a viable alternative.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The objectstorage-controller package ships the COSI control plane; its commits
use the objectstorage-controller scope, which had no area mapping and fell back
to area/uncategorized. Map it to area/storage alongside the other COSI/storage
scopes.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The bucket suite waited on '{.status.bucketReady}' without an expected value;
a bare jsonpath match is satisfied by the literal "false", so the wait passed
against an unready claim. Both suites also allowed 300s/600s, long enough to
tolerate the propagation race rather than catch it, so they stayed green
whether or not the BucketClaim ever converged.
Assert bucketReady=true on a 120s bound. Convergence is now controller-driven
(the controller requeues the claim until the backend Bucket's readiness
propagates) and completes within tens of seconds; a claim frozen by the
propagation race never converges and now fails the wait fast instead of being
masked.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The shipped kamaji binary was compiled with go1.26.3, which trivy flags
for the standard-library CVEs fixed in go1.26.4: crypto/x509
CVE-2026-27145, mime CVE-2026-42504, and net/textproto CVE-2026-42507.
The image base is distroless/static, so the remaining OS-package CVEs
from the scan do not apply to it.
Advance the build source from 26.3.5-edge to 26.3.6-edge, the next
patch on the same edge line (identical k8s.io/api v0.35.0 and
controller-runtime v0.22.4, staying clear of the k8s 1.36 line that
regresses tenant control-plane RBAC). The version change busts the
build-layer cache so the release rebuild compiles against the current
golang:1.26 toolchain, which resolves to 1.26.4 and closes the stdlib
CVEs.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The controller and the per-driver sidecar both run single-replica, so leader
election provides no high availability -- it only governs how quickly a freshly
recreated pod takes over. With the upstream 150s lease duration a recreated
provisioner pod stalls reconciliation for up to 150s while it waits out the
stale lease still held by the previous, now-deleted pod. Lower the lease timings
to the standard Kubernetes values (15s/10s/2s) so a recreated pod resumes
reconciliation in roughly 15s.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The dynamic-provisioning path re-reads the live Bucket after create and copies
its readiness onto the BucketClaim, but only during a reconcile. Nothing
re-drives that reconcile afterwards: the controller does not watch Buckets, and
the generic controller drops no-op resync deltas (reflect.DeepEqual), so once
the BucketClaim stops changing the listener is never re-invoked. The Bucket's
false->true transition, performed asynchronously by the sidecar on a different
object, is therefore reflected onto the claim only if the Bucket happened to be
ready during that single reconcile -- a race the sidecar usually loses, leaving
bucketReady frozen at false and BucketAccess never granted.
Return an error while the backend Bucket is not ready so the work queue
requeues the claim with exponential backoff; each retry re-reads the live
Bucket until its readiness has been propagated, converging deterministically
instead of depending on a resync that never fires.
Add a unit test pinning the contract (requeue while not ready, propagate true
once ready) and adjust the existing status-update conflict-retry test, which
pinned the old never-requeue behavior.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Refresh the vendored virt-operator manifest from v1.8.2 to v1.8.4. The
release backports validation of the VMI checksum status fields as a
uint32 range, required on Kubernetes 1.36 where strict CRD numeric
format validation otherwise leaves VMIs stuck in Scheduled. Remaining
changes are bug fixes and CVE remediations; the embedded CRDs and RBAC
are unchanged, so only the operator image tag moves.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Clear the reachable GO-2026-5026 advisory: idna.ToASCII is reached
through the KubeVirt client path, and the fix landed in x/net v0.55.0.
Bump to v0.56.0, pulling x/sys, x/term and x/text forward as required;
govulncheck then reports the module clean.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
A local `go build` emits the module-named binary (kube-ovn-webhook),
while the Dockerfile builds `-o webhook`. Ignore both names so neither
local binary is accidentally committed.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Several first-party images still built their Go binaries with EOL or
behind-latest toolchains (golang 1.20, 1.20.6, 1.23.4, 1.24, 1.25), so a
plain rebuild kept shipping standard-library versions that the current
1.26 toolchain fixes. Raise every Go builder image (FROM and the
builder_image ARG defaults) to golang:1.26 across cluster-autoscaler,
kubevirt-cloud-provider, redis-operator, multus-cni, keycloak-operator,
objectstorage, kilo and linstor-csi.
This is the build-time counterpart to raising the go directive in the
first-party Go modules: the toolchain that compiles the binary, not the
go.mod minimum, determines the standard-library version baked into the
image.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The token-proxy, kube-ovn-webhook and kubevirt-csi-driver image modules
still pinned go 1.26.2, so their binaries shipped standard-library
versions that the 1.26.4 toolchain fixes. Raise the go directive and
refresh module dependencies to current releases.
token-proxy and kube-ovn-webhook take a full dependency refresh;
kube-ovn-webhook also moves k8s.io/* from 0.26.0 to 0.36.2 (only
long-stable admission/core API is used, so no code change is needed).
kubevirt-csi-driver keeps its existing replace-based k8s.io/* alignment
at 0.36.0 and only raises the go directive; a blanket go get -u is
avoided there because it pulls transitive bumps that break compilation
of the pinned vendored upstream csi-driver source.
Also ignore the locally-built binaries under the two image directories so
they cannot be committed by accident.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The pre-delete cleanup Job ran kubectl as root with no securityContext.
Add a pod-level runAsNonRoot + runAsUser 65534 + seccompProfile
RuntimeDefault and a container-level allowPrivilegeEscalation: false plus
capabilities drop ALL, mirroring the dashboard adopt hook that runs the
same clastix/kubectl image non-root. Pin the securityContext in
helm-unittest.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The pre-delete cleanup Job ran kubectl from `bitnami/kubectl:latest`. The
bitnami Docker Hub catalog has moved to a legacy/sunset state, so that pull is
at risk of breaking — which would stop the cleanup hook from starting at all —
and the `:latest` tag leaves the kubectl version unpinned and non-reproducible.
Switch to `docker.io/clastix/kubectl:v1.32`, the version-pinned image already
used by every other first-party cleanup/delete hook in the repo (kubernetes,
keycloak-configure, dashboard, mariadb). That image is Alpine-based and ships
only `/bin/sh`, so the hook script (pure POSIX) now invokes `/bin/sh` instead
of `/bin/bash` — matching those sibling hooks.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The pre-delete cleanup hook deletes the tenant's HelmReleases with
`kubectl delete --wait=true`, which opens a watch on helmreleases to block
until finalization. The hook Role granted only get/list/delete, so the watch
was rejected with 403, the delete call hung, the Job never finished, and
tenant deletion stalled on the hook.
Add the watch verb, matching the kubernetes app chart's cleanup Role which
already includes it for the same --wait deletes.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The access-check helpers returned plain fmt.Errorf values, which the
apiserver coerces to 500 InternalError regardless of the actual failure.
A request without user identity now surfaces as 401 Unauthorized, and a
RoleBinding list failure as 500 InternalError with a typed Status body.
Get also repackages a NotFound from the backing Namespace lookup so the
Status body reports the tenantnamespaces resource instead of namespaces.
The Watch goroutine already logs access-check failures and skips the
event, so no untyped error escapes to the client from that path.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The helper header and run-migrations.sh claimed every migration-driven
stamp routes through the helper, but migrations 1-41 still stamp the
cozystack-version ConfigMap label-less and are backfilled by migration 42.
Scope the comments to go-forward stamps (migration 42 onward and the
bootstrap path), and add a unittest asserting no migration numbered >= 42
nor run-migrations.sh stamps cozystack-version without the helper, so the
no-drift invariant is enforced rather than aspirational.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Mirror render_cozystack_version_manifest's argument guard in
stamp_cozystack_version so a missing version aborts before the kubectl
pipe. Without it, a render failure in a shell without pipefail would
leave the pipeline exit status to kubectl, which would apply empty input.
Add a unittest covering the guard.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The bootstrap path that creates cozystack-version when the ConfigMap is
absent applied it without the platform.cozystack.io/no-delete label,
leaving it unguarded by the cozystack-no-delete-guardrail
ValidatingAdmissionPolicy. Route it through the shared helper so the
bootstrap ConfigMap carries the label like every migration-driven stamp.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Migrations 42, 43 and 44 each carried a copy-pasted heredoc that applied
the labeled cozystack-version ConfigMap. Source the shared helper and call
it instead, so the labeled manifest has a single definition and future
migrations cannot reintroduce a label-less apply.
Behavior-preserving: the rendered manifest is byte-identical to the
previous heredocs. Migration 42 keeps its MIGRATION_DRY_RUN path, now
building its manifest from the helper's pure renderer.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Extract the labeled cozystack-version ConfigMap stamp into a sourced
shell helper so migrations cannot drift back to a label-less apply that
strips the platform.cozystack.io/no-delete label.
render_cozystack_version_manifest emits the manifest on stdout (pure,
unit-testable); stamp_cozystack_version renders and applies it. The
namespace defaults to cozy-system and honors $NAMESPACE.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
## What this PR does
Bumps three of the small components listed in #2900, each picking up
published security fixes:
- **opencost** chart `1.41.0 → 2.5.23` (app `1.111.0 → 1.120.3`). The
1.x→2.x chart-major bump's values schema is additive, so the cozystack
overrides (external prometheus vmselect endpoint, customPricing) stay
valid. The Makefile now pins `--version 2.5.23` and drops the upstream
snapshot test suite (it can't resolve in the cozystack render context);
cozystack ships its own helm-unittest.
- **multus-cni** `v4.2.3 → v4.3.0`, rebuilt multi-arch (builder
`golang:1.24 → 1.26` for the Go-stdlib CVEs). This also **folds the
`multusMasterCNI: 05-cilium.conflist` CNI-ordering fix into
`customize-deployment.patch`** — it was previously a raw daemonset
hand-edit that the next `make update` would have wiped, re-introducing
the kube-ovn-before-cilium startup race. A new helm-unittest pins it.
- **clickhouse-backup tool** `altinity/clickhouse-backup 2.6.32 →
2.7.2`, rebuilt multi-arch. The separate `clickhouse-backup` image
(clickhouse-server base) and the managed clickhouse-server version are
intentionally left as-is (multi-major server upgrade, out of scope).
Each package gains helm-unittest coverage (none shipped before).
**Deferred from #2900 (this PR does NOT close the issue):**
- **bootbox** — already at the latest upstream tag (no-op).
- **kubeovn-plunger** — first-party cozystack component, no upstream
release to bump.
- **etcd-operator** — held at 0.4.5: the 0.5.x Helm chart is not
published on the `ghcr.io/aenix-io/charts` OCI registry yet (only GitHub
releases exist), so `helm pull` can't fetch it. Needs an aenix-io chart
publish first.
- **ingress-nginx** — deliberately pinned to 4.11.2 (revert `5c900a746`
after the CVE-2025-1974 mitigation attempt); bumping needs a separate
investigation, not a routine sweep.
### Release note
```release-note
chore(images): bump opencost to 2.5.23, multus-cni to v4.3.0, and clickhouse-backup to 2.7.2
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced OpenCost Helm support for plugin installation/config
validation, cloud integration (secret or JSON), MCP server, and
`extraObjects` rendering
* Added admin token secret creation and Pod Disruption Budget options
* Expanded UI/API/MCP exposure via Ingress, OpenShift Route, and Gateway
API HTTPRoute
* Improved internal Prometheus/Thanos endpoint handling and optional CA
trust update
* **Updates**
* Updated ClickHouse backup and Multus images; adjusted Multus “thick”
deployment behavior/resources and namespace usage
* Refreshed OpenCost chart values/docs and bumped chart/app versions
* **Tests**
* Added/extended Helm unittest suites and rendering invariants for
backup, Multus, and OpenCost
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Bumps cilium from 1.19.3 to 1.19.5 (latest 1.19 patch), picking up the
published security fixes the issue enumerates (incl. the
`moby/spdystream` and `golang.org/x/net` updates carried by 1.19.4) plus
upstream bug fixes: CiliumEgressGatewayPolicy NamespaceSelector
corruption, BGP PeerConfig status cleanup, and a reused-endpoint BPF
state race.
The change is a single self-contained commit:
- Re-vendors the chart via `make update` (helm pull cilium 1.19.5). The
two cozystack vendoring patches re-apply cleanly — the
SYS_MODULE/iptables capability lines and the k8s<1.30 AppArmor
annotation block are stripped (the Makefile guard confirms the AppArmor
block is gone, so there is no duplicate-mapping-key risk on the agent
DaemonSet).
- Points `images/cilium/Dockerfile` at `quay.io/cilium/cilium:v1.19.5`.
- Rebuilds the cozystack agent image multi-arch (linux/amd64 +
linux/arm64) and pins it in `values.yaml` (`1.19.5@sha256:8f0e898…`),
mirroring the rebuild-and-pin step done for the 1.19.3 bump — so the
deployed agent carries the fixes immediately, not just on the next
release rebuild.
Upstream dropped the dev-only `values.yaml.tmpl` / `README.md.gotmpl`
from the 1.19.5 release tarball (hence the large deletion). The defunct
`l2podAnnouncements.interface` and `loadBalancer.standalone` removals
don't affect cozystack, which uses neither.
Verified: `helm template` renders the agent at `1.19.5@sha256:8f0e898…`
across the talos/kilo/kubeovn/apparmor overlays; `helm lint` passes
aside from the expected wrapper-chart dependency artifact.
Closes#2893
### Release note
```release-note
chore(cilium): bump cilium to 1.19.5
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Upgraded the Cilium Helm chart to **1.19.5** and refreshed default
container images/digests across components.
* Adjusted Helm packaging to exclude `*.tmpl`/`*.gotmpl` artifacts.
* **New Features**
* Expanded RBAC to include **BGP peering policy** resources.
* Updated **L2 pod announcements** configuration to use
`interfacePattern`.
* **Bug Fixes**
* Improved ingress EndpointSlice readiness/labels for more consistent
endpoint behavior.
* Enabled IPv6 traffic to fall back to the kernel stack when the IPv6
datapath is disabled.
* **Refactor**
* Made the operator health `hostPort` configurable based on host
networking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Rebuilds the first-party `kilo` image to clear the published
`libpng16-16` / `libpng` advisories carried by the stale pinned layers,
and refreshes it onto current upstream.
- `images/kilo/Dockerfile`: `ARG VERSION` `fbce60d8` → `944dff57`
(current `squat/kilo` `main` HEAD); the rebuild also refreshes the
`golang:1.25` stdlib compiled into the `kg` / `kgctl` binaries. The
builder tag stays `golang:1.25` (upstream `go.mod` is `go 1.25.0`).
- runtime and `cni` base: `alpine:3.23` → `alpine:3.24`, which clears
the `libpng` advisories pulled in through the `graphviz` / `font-noto`
dependency chain.
- CNI plugins: `v1.9.0` → `v1.9.1`.
- `values.yaml`: re-pinned to the rebuilt multi-arch digest
(`linux/amd64` + `linux/arm64`); the human-readable tag handle moves
from `latest` to `v<version>` to match the sibling first-party images.
The in-repo cross-mesh-granularity feature patch still applies cleanly
on the new HEAD and is verified present in the built binary (`kg
--mesh-granularity` lists `cross`). Adds the package's first
helm-unittest coverage, pinning the image ref and the default mesh-arg
wiring.
### Release note
```release-note
chore(kilo): rebuild kilo image on current upstream and alpine 3.24 to clear the libpng advisories
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Upgraded Kilo to **v1.5.0**.
* Updated bundled CNI plugins to **v1.9.1**.
* Updated Kilo build/runtime base to **Alpine 3.24**.
* **Tests**
* Added Helm/unit coverage to validate Kilo DaemonSet image pinning and
default/overridden argument behavior (including compatibility options).
* Added a `test` make target to run Helm unit tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
The per-namespace Memory request/limit panels summed
kube_pod_container_resource_{requests,limits} over all pods regardless of
phase, while the sibling CPU panels were updated to multiply by
* on (namespace, pod) group_left(phase) (kube_pod_status_phase{phase="Running"} == 1).
Apply the same join so the paired CPU/Memory rows count the same pod population.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Enable Renovate's dockerfile and github-actions managers with
pinDigests so OpenSSF Scorecard Pinned-Dependencies (currently 1/10)
can pass and supply-chain refs stay reproducible.
PR #2849 SHA-pinned actions in 13 workflows, but the check also scores
container base images -- 58 unpinned FROM lines across 27 distinct
images -- and left build-main.yaml / release-e2e.yaml unpinned.
Nothing kept the new action SHAs updated either, so they would
silently stop receiving security patches.
Enabling the managers lets Renovate open the initial pin PR (and pin
the two missed workflows) and keep digests fresh thereafter.
Digest-only refreshes and the initial pin are batched into a single
weekly PR to avoid a trickle of rebuild-triggering PRs on the shared
CI runner; real tag bumps stay individual for review.
Also pin s3manager's floating tags (golang:1, alpine:latest) to
concrete versions first, otherwise digest tracking churns endlessly.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
## What this PR does
Restores the OpenSSF Scorecard **Vulnerabilities** check to green. It
regressed to **0/10** after #2852: in the weeks since,
`golang.org/x/crypto v0.51.0` accrued **13 new advisories**
(`GO-2026-5005`, `5006`, `5013`–`5021`, `5023`, `5033`). An
`osv-scanner` pass over `main` confirmed all 13 trace to that single
package.
### Change
- **`golang.org/x/crypto 0.51.0 → 0.52.0`** (the fixed version for all
13). No transitive cascade — `go mod tidy` changed only this line plus
its two `go.sum` hashes.
Bumped with `go get` / `go mod tidy`, not by hand-editing the generated
`go.mod`.
### Verification
- `osv-scanner` after the bump: **No issues found** (0 advisories across
both modules).
- `go build ./...` passes.
- `make generate` produces **no drift** — `codegen-drift` stays green.
> Scorecard's report mentioned advisory IDs as low as `GO-2025-3512`
(kubevirt-csi, CVE-2024-1725). That package is **not** in `go.mod` or
`go.sum` — it isn't a cozystack dependency, so nothing to bump there.
The real cluster is the 13 x/crypto advisories this PR closes.
### Why this keeps happening (and the durable fix)
`x/crypto` is an **indirect** dependency, and the Renovate app isn't
actually opening PRs on this repo yet (no `renovate[bot]` PRs, no
Dependency Dashboard) — the org Renovate install needs
`cozystack/cozystack` added to its repo list. Until then, indirect-dep
and Go-toolchain advisories will keep knocking this check back to 0 and
need manual bumps like this one. Recommended follow-ups: enable Renovate
on the repo, and allow it to bump indirect deps (and optionally the Go
patch version).
### Release note
```release-note
fix(deps): bump golang.org/x/crypto to v0.52.0, closing 13 known OSV advisories (GO-2026-5005…5033)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated cryptographic dependency to the latest patch version for
enhanced security and stability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Hardens GitHub Actions security posture to raise two OpenSSF Scorecard
checks that currently score **0/10**, as part of the push to ≥7.5 ahead
of CNCF Incubation due diligence. No workflow triggers, job graphs, or
step logic change — only `GITHUB_TOKEN` scopes and action pin refs.
### Token-Permissions (was 0/10)
The check failed because several workflows (`codegen-drift.yml`,
`pre-commit.yml`, `retest.yaml`, `tags.yaml`, …) declared **no
`permissions:` block at all**, so their jobs inherited the repository
default token — typically read/write across all scopes.
This PR applies the OpenSSF-recommended pattern to **every** workflow:
- A **top-level read-only default**: `permissions: { contents: read }`.
- An **explicit per-job `permissions:` block** granting only the writes
that job needs. Workflows that previously held write scopes at the
*workflow* level (`backport`, `pr-labeler`, `pr-size`, `stale`) now hold
them at *job* level, so the read-only default applies everywhere else.
Per-job scopes were derived from what each job actually calls:
| Workflow | Job | Scopes granted |
|---|---|---|
| auto-release | auto-release | `contents: write, pull-requests: read`
*(unchanged)* |
| backport | prepare | `contents: read` |
| backport | backport | `contents: write, pull-requests: write` |
| codegen-drift | codegen-drift | `contents: read` |
| labels | validate | `contents: read` |
| labels | sync | `contents: read, issues: write, pull-requests: write`
|
| pre-commit | pre-commit | `contents: read` |
| pr-labeler | label | `pull-requests: write` |
| pr-size | size | `pull-requests: write` |
| retest | retest | `actions: write, pull-requests: read` *(unchanged)*
|
| stale | stale | `issues: write, pull-requests: write` |
| update-releasenotes | update-releasenotes | `contents: write`
*(unchanged)* |
| pull-requests | detect-changes | `contents: read, pull-requests: read`
|
| pull-requests | build | `contents: read, packages: write`
*(unchanged)* |
| pull-requests | resolve_assets | `contents: read` |
| pull-requests | e2e | `contents: read, packages: read, checks: write`
*(unchanged)* |
| pull-requests-release | finalize | `contents: write` *(unchanged)* |
| tags | prepare-release | `contents/packages/pull-requests/actions:
write` *(unchanged)* |
| tags | generate-changelog | `contents: write, pull-requests: write`
*(unchanged)* |
| tags | update-website-docs | `contents: read` *(unchanged)* |
Self-hosted release jobs that push tags/branches/releases authenticate
via the `cozystack-ci` GitHub App token, so reducing the workflow
`GITHUB_TOKEN` does not affect them; the job-level write scopes they
already declared are preserved verbatim.
### Pinned-Dependencies (was 0/10)
Every GitHub-owned and third-party action is now pinned to a **full
commit SHA** with a human-readable version comment, e.g.
`actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4`.
`scorecard.yml` was already SHA-pinned in #2721; `breakpoint-action` was
already pinned. SHAs were resolved from each tag's current commit at
authoring time, so behavior is identical to the previously-referenced
tag.
> **Scope note:** this PR pins **GitHub Actions** only. Dockerfile
`FROM` digest pinning (≈40 images) is deliberately out of scope here —
it touches image builds and is higher blast-radius; tracked separately.
### Verification
- `actionlint` over all workflows: **exit 0**.
- All 14 workflow files parse as valid YAML.
- Diff is mechanical: only `permissions:` blocks and action `@ref`s
changed.
### Release note
```release-note
ci: declare least-privilege per-job GITHUB_TOKEN permissions and pin all GitHub Actions by commit SHA across CI workflows (OpenSSF Scorecard hardening)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Tightened CI/CD security with least-privilege workflow token
permissions: workflows now default to read-only access (`contents:
read`), with select jobs requesting write or additional scopes only when
needed.
* Improved supply-chain safety and consistency by pinning GitHub Actions
(and related third-party actions) to specific commit SHAs across
multiple workflows, reducing reliance on floating version tags.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Add an issue-template chooser and a feature-request template, and document
in CONTRIBUTING and the README where each kind of contribution belongs:
- bugs and concrete features -> issues in this repo
- questions and early ideas -> GitHub Discussions
- cross-cutting / architectural changes -> design proposals in cozystack/community
Also reconciles the README, which routed feature requests to a Discussions
category while the issue tracker is actually used for kind/feature issues.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Restores the OpenSSF Scorecard Vulnerabilities check to green. It
regressed to 0/10 after #2852 because golang.org/x/crypto v0.51.0 accrued
13 new advisories (GO-2026-5005, 5006, 5013-5021, 5023, 5033). osv-scanner
confirms all 13 trace to that one indirect package; v0.52.0 is the fixed
version. go mod tidy changed only this line and its two go.sum hashes.
After the bump osv-scanner reports no issues, both modules build, and
'make generate' produces no drift. Versions were changed with go get /
go mod tidy, not by hand-editing the generated go.mod.
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: Claude <noreply@anthropic.com>
Re-vendors the cilium chart from 1.19.3 to 1.19.5 (latest 1.19 patch) and
points the agent image build at v1.19.5. Picks up the published security
fixes plus upstream bug fixes (CiliumEgressGatewayPolicy NamespaceSelector
corruption, BGP PeerConfig status cleanup, reused-endpoint BPF state race).
The make-update vendoring patches re-apply cleanly: the SYS_MODULE/iptables
capability lines and the k8s<1.30 AppArmor annotation block are stripped (the
guard confirms the AppArmor block is gone, and the rendered agent DaemonSet
carries the annotation exactly once via cozystack podAnnotations). Upstream
dropped the dev-only values.yaml.tmpl / README.md.gotmpl from the 1.19.5
release tarball. The defunct l2podAnnouncements.interface removal does not
affect cozystack, which uses interfacePattern.
The BPF host patch (pass-ipv6-to-stack-when-ipv6-datapath-disabled.diff) is
refreshed against v1.19.5 bpf_host.c: identical logic, only context lines
re-synced.
The deployed agent image pin in values.yaml is intentionally left on the
current release-line digest; the multi-arch cilium agent is rebuilt and
re-pinned during release preparation.
Adaptations:
- charts/: re-vendored (helm pull cilium 1.19.5 + capability/AppArmor patches)
- images/cilium/Dockerfile: ARG VERSION v1.19.3 -> v1.19.5
- images/cilium/patches/: BPF host patch context refreshed for v1.19.5
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rebuilds the first-party kilo image to clear the published libpng advisories
carried by the stale pinned layers and to pick up current upstream:
- ARG VERSION: squat/kilo main fbce60d8 -> 944dff57 (the rebuild also refreshes
the golang:1.25 stdlib compiled into the kg / kgctl binaries)
- runtime and cni base: alpine 3.23 -> 3.24, which clears the libpng16-16 /
libpng advisories pulled in through the graphviz / font-noto chain
- CNI plugins: v1.9.0 -> v1.9.1
- values.yaml: re-pinned to the rebuilt multi-arch digest; the tag handle moves
from latest to v<version> to match the sibling first-party images
The in-repo cross-mesh-granularity feature patch still applies cleanly on the
new HEAD and is verified in the built binary (kg --mesh-granularity lists
`cross`). Adds the package's first helm-unittest coverage, pinning the image
ref and the default mesh-arg wiring.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Rebuilds the altinity-clickhouse-backup sidecar image off the upstream
altinity/clickhouse-backup 2.6.32 -> 2.7.2 release (multi-arch) and pins the
new digest. Picks up the published security fixes in the backup tool. The
clickhouse-server version (24.9.2) is intentionally left as-is: moving it is a
multi-major server upgrade, out of scope for this maintenance bump.
Extends the existing backup helm-unittest to assert the sidecar image tag, so
the bump is guarded against regressions.
Adaptations:
- images/altinity-clickhouse-backup/Dockerfile: FROM altinity/clickhouse-backup 2.6.32 -> 2.7.2
- images/altinity-clickhouse-backup.tag: pin rebuilt multi-arch image digest
- tests/backup_test.yaml: assert the backup sidecar image tag
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the multus-cni thick-mode daemonset from upstream v4.2.3 to v4.3.0
and rebuilds the cozystack image (multi-arch, golang:1.26 builder; v4.3.0's
go.mod and the published security fixes warrant the stdlib bump from 1.24). The
fix-del-without-cache and customize-deployment patches both re-apply cleanly to
v4.3.0. The rebuilt image is pinned by digest in the daemonset.
Folds the previously hand-edited `multusMasterCNI: "05-cilium.conflist"`
CNI-ordering fix into customize-deployment.patch so it survives every make
update (it was a raw daemonset edit that the re-vendor would otherwise wipe,
re-introducing the kube-ovn-before-cilium startup race).
Adds helm-unittest coverage (the package shipped none): asserts the daemonset
runs the cozystack-built image and that the daemon-config pins multusMasterCNI
to the cilium conflist.
Adaptations:
- images/multus-cni/Dockerfile: builder golang:1.24 -> 1.26, ARG VERSION v4.2.3 -> v4.3.0
- templates/multus-daemonset-thick.yml: re-vendored v4.3.0 + customize patch; pin rebuilt multi-arch image
- patches/customize-deployment.patch: carry the multusMasterCNI CNI-ordering fix
- Makefile: add test target
- tests/multus_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Re-vendors the opencost chart from 1.41.0 to 2.5.23 (app 1.111.0 -> 1.120.3),
crossing the 1.x -> 2.x chart-major boundary, picking up the published security
fixes. The major bump's values-schema changes are additive (new clusterName,
pdb, extraObjects keys; no removals), so the cozystack overrides (external
prometheus vmselect endpoint, customPricing) stay valid -- confirmed by render.
Pins the update target to --version 2.5.23 and drops the upstream chart's own
snapshot-based helm-unittest suite (it fails in the cozystack render context).
Adds cozystack helm-unittest coverage: pins the image and the prometheus
endpoint wiring.
Adaptations:
- charts/: re-vendored via make update (helm pull opencost --version 2.5.23; upstream tests removed)
- Makefile: pin --version 2.5.23; add test target; strip upstream chart tests
- tests/opencost_test.yaml: new helm-unittest coverage
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Bump the vendored Harbor Helm chart from 1.18.2 to 1.19.1, which
upgrades Harbor itself from appVersion 2.14.2 to 2.15.1.
The bump adds PodDisruptionBudget templates for the core, exporter,
jobservice, nginx, portal, registry and trivy components, improving
availability during voluntary disruptions (node drains/upgrades).
<!-- Thank you for making a contribution! Here are some tips for you:
- Use Conventional Commits for the PR title: `type(scope): description`
- Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore
- Scopes are not an exhaustive list — pick the most specific scope for
the change and extend the list when a genuinely new area appears.
Examples:
- System components: dashboard, platform, operator, cilium, kube-ovn,
linstor, fluxcd, cluster-api
- Managed apps: postgres, mariadb, redis, kafka, clickhouse,
virtual-machine, kubernetes
- Development and maintenance: api, hack, tests, ci, docs, maintenance
- Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or
add a `BREAKING CHANGE:` footer
- If it's a work in progress, consider creating this PR as a draft.
- Don't hesistate to ask for opinion and review in the community chats,
even if it's still a draft.
- Add the label `backport` if it's a bugfix that needs to be backported
to a previous version.
-->
## What this PR does
### Screenshots
<!-- REQUIRED for UI changes: attach screenshots or screen recordings
demonstrating
the visual impact of your changes. PRs with UI changes without
screenshots will not be merged. -->
### Release note
<!-- Write a release note:
- Explain what has changed internally and for users.
- Start with the same `type(scope):` prefix as in the PR title
- Follow the guidelines at
https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md.
-->
```release-note
feat(harbor): bump Harbor to 2.15.1 and add PodDisruptionBudgets for core, portal, jobservice, nginx, registry, trivy and exporter
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added Pod Disruption Budget support for core, portal, jobservice,
nginx, registry, trivy, and exporter.
* Made container health checks (liveness/readiness, plus core startup)
fully configurable per component.
* Added configurable registry HTTP client timeout.
* Enhanced Gateway API route configuration (hostnames formatting and
backend reference details).
* **Improvements**
* Improved external Redis credential handling for registry usage (safer
secret guarding and URL-encoding).
* Updated Harbor chart metadata and component images to **v2.15.1**.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Closes#2880
## What this PR does
Refreshes the `system/monitoring` image anchors onto current upstream to
clear the published advisories (8 CVEs, 1 critical) carried by the stale
pins.
- **grafana image**: base `grafana/grafana` `11.4.0` → `11.6.15` (latest
11.x). The rebuild clears the Go-dependency advisories in the grafana
binary (`x/crypto`, `moby/spdystream`, `jackc/pgx/v5`,
`russellhaering/goxmldsig`, `buger/jsonparser`, `golang-jwt/jwt/v4`,
`getkin/kin-openapi`). It stays on the 11.x line rather than 12.x to
keep the bundled panel plugins compatible (`grafana-worldmap-panel` is
deprecated upstream) and avoid a dashboard-schema migration.
- **victorialogs-datasource** plugin: `v0.14.1` → `v0.28.0`
(`grafanaDependency >=10.4.0`, so it loads on 11.6.15).
- **alerta-web**: `9.0.4` → `9.1.0`, clearing the python `cryptography`
advisory.
- `images/grafana.tag`: re-pinned to the rebuilt multi-arch digest
(`linux/amd64` + `linux/arm64`).
The built grafana image was verified in place: grafana `11.6.15`, all
four plugins present (`natel-discrete-panel`, `grafana-worldmap-panel`,
`marcusolsson-dynamictext-panel`, `victoriametrics-logs-datasource`),
vlogs `0.28.0`. helm-unittest passes (4/4).
Closes#2892
### Release note
```release-note
chore(monitoring): bump Grafana to 11.6.15, victorialogs-datasource to v0.28.0 and alerta-web to 9.1.0
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Grafana monitoring dashboard base image to 11.6.15 and
VictoriaLogs datasource plugin to v0.28.0
* Upgraded Alerta monitoring and alerting tool to 9.1.0
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## What this PR does
Rebuilds the e2e test sandbox on current upstream to clear the published
advisories (containerd, helm, go-git, moby/spdystream, Go stdlib)
carried by the stale tool binaries and base layers.
- base: `ubuntu:22.04` → `24.04`; `netcat` → `netcat-openbsd` (on 24.04
`netcat` is a candidate-less virtual package, so the concrete provider
must be named)
- kubectl: `1.33.2` → `1.33.13` — stays on the 1.33 line that the e2e
control plane pins at `v1.33.12` (`hack/e2e-prepare-cluster.bats`), so
client/server skew is unchanged
- talosctl: `1.10.4` → `1.13.5` — aligns the client with the v1.13.x
Talos nodes the sandbox provisions
- helm: `3.18.3` → `3.21.2` (latest helm 3) — the e2e flow drives the v3
CLI, so it stays on the 3.x line rather than moving to the breaking 4.x
- yq: `4.44.3` → `4.53.3`
- `values.yaml`: re-pinned to the rebuilt multi-arch digest
(`linux/amd64` + `linux/arm64`)
`crust-gather` and `cozyhr` already track their latest releases; `flux`
and `mc` install the current release at build time. The built image's
tool versions were verified in place (kubectl `v1.33.13`, talosctl
`v1.13.5`, helm `v3.21.2`, yq `v4.53.3`, flux `2.8.8`, ubuntu `24.04`).
Closes#2891
### Release note
```release-note
chore(testing): refresh e2e-sandbox tools and base (ubuntu 24.04, kubectl 1.33.13, talosctl 1.13.5, helm 3.21.2)
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated the e2e sandbox image to a new fixed digest for more
consistent test runs.
* Refreshed the sandbox build environment (Ubuntu 24.04) and upgraded
tooling (Kubernetes, Talos, Helm, and yq) to newer stable versions.
* Improved OS package setup by switching from `netcat` to
`netcat-openbsd` and consolidating apt cleanup to reduce image size.
* **Documentation**
* Updated e2e testing guidance: removed the recommendation to pin the
management cluster Kubernetes version, and added guidance to
snapshot/restore cluster state between test groups.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Aleksei Sviridkin