Files
Andrey Kolkov 647407545d fix(backupstrategy-controller): run backup clients on a package-owned image
The Altinity and RabbitMQ strategy Pods ran platform-migrations, a 349 MB
image carrying kubectl, helm and etcd-migrate whose ENTRYPOINT is the
migration runner, and whose pins were stamped from packages/core/platform
so every migration change moved the backup clients too.

Build images/backup-http-client (alpine + curl + jq + ca-certificates,
non-root) from this package's Makefile and stamp both client keys from
it. The cross-package stamp in packages/core/platform/Makefile and the
platform-migrations-specific check in hack/image-pin-consistency.bats go
away; the general one-digest-per-repository check still covers the two
keys sharing one image.

Fixes #3420

Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-10-12 09:54:52 +04:00

201 lines
12 KiB
YAML

backupStrategyController:
image: "ghcr.io/cozystack/cozystack/backupstrategy-controller:v1.6.0@sha256:e8bf0521d1d1d8b90ebe66e9013010887033ba2653901799d57f2df47c9d879e"
# chBackupClientImage is rendered into the Altinity strategy Pod, which drives
# clickhouse-backup's HTTP API with curl + jq. rabbitmqBackupClientImage is
# rendered into the RabbitMQ strategy Pod, which calls the management API and
# moves the definitions export to S3 with curl --aws-sigv4. Both run the
# images/backup-http-client/ image (alpine + curl + jq + ca-certificates) and
# must hold the same ref: one repository pinned at two digests fails the
# release promotion (hack/image-pin-consistency.bats). They are separate keys
# so either strategy can move to its own image later.
# DO NOT hand-edit: `make image` stamps both.
chBackupClientImage: "ghcr.io/cozystack/cozystack/backup-http-client:v1.6.0"
rabbitmqBackupClientImage: "ghcr.io/cozystack/cozystack/backup-http-client:v1.6.0"
# redisBackupClientImage is rendered into the Redis strategy Pod. The
# spotahome operator ships no backup CRD, so the strategy runs a one-shot Job
# that dumps an RDB snapshot to S3 and, on restore, loads it into a throwaway
# redis to emit a faithful replay stream. It ships its own image (redis 8 +
# curl) built from images/redis-backup/; `make image` re-digests this value.
redisBackupClientImage: "ghcr.io/cozystack/cozystack/redis-backup:v1.6.0"
# redisBackupActiveDeadlineSeconds bounds one backup/restore Job so a run that
# wedges (e.g. a --rdb dump blocking on a stalled master) becomes a Failed Job
# the driver reports, rather than a Running BackupJob a cron Plan stacks behind.
# The driver promotes this to the Job's activeDeadlineSeconds, so it bounds the
# whole run across retries, not each Pod attempt separately.
redisBackupActiveDeadlineSeconds: 3600
# redisBackupResources sizes the Redis strategy Pod. Restore is the demanding
# side: the throwaway loader holds the whole dataset in memory and writes the
# RDB plus its AOF rewrite to ephemeral storage, so a large Redis needs memory
# and ephemeral-storage well above the backup's. Raise these for tenants with
# large instances; the defaults suit small-to-medium ones.
redisBackupResources:
requests:
cpu: 10m
memory: 64Mi
ephemeral-storage: 256Mi
limits:
cpu: "1"
memory: 1Gi
ephemeral-storage: 4Gi
# The Kafka metadata strategy carries no client image here: unlike the Altinity
# and RabbitMQ drivers, which need only curl (plus jq for Altinity), its driver
# runs the Kafka Admin-API CLI (kafka-topics / kafka-configs), which only the
# Strimzi broker image ships. So the controller resolves the image from
# the target Kafka broker at reconcile time (strategy-kafka-default.yaml renders
# the container image as the "{{ .ClientImage }}" placeholder), and records it on
# the Backup so cleanup can reuse it once the broker is gone. No pin to drift.
replicas: 2
debug: false
metrics:
enabled: true
bindAddress: ":8443"
resources:
requests:
cpu: 10m
memory: 64Mi
limits:
cpu: 500m
memory: 128Mi
# backupStorage describes the platform-managed S3 bucket used by every
# default BackupClass strategy. The bucket lives in the backupStorage.namespace
# (tenant-root by default) and is provisioned through the apps.cozystack.io/Bucket CR.
# Tenants never see these coordinates directly — they reference cozy-default
# BackupClass and the backupstrategy-controller projects credentials per BackupJob.
backupStorage:
# provisionBucket controls whether to create the apps.cozystack.io/Bucket CR.
# Set to false when the cluster runs against an external S3 (no SeaweedFS)
# and the admin manages the credentials Secret manually.
provisionBucket: true
# The Bucket CR lives in tenant-root (a tenant namespace) because the
# apps.cozystack.io/Bucket chart's RBAC helper fails Helm render on any
# namespace that does not start with "tenant-" (see
# packages/library/cozy-lib/templates/_rbac.tpl). tenant-root is the
# root tenant present on every Cozystack install, so the system Bucket
# naturally lives there. Cross-namespace Secret read is granted to the
# backupstrategy-controller via cluster-scoped Secret RBAC.
namespace: tenant-root
bucketName: cozy-backups
# bucketNameOverride is an escape hatch for offline `helm template`
# renders (CI / local diffs). When set, the chart skips the live-cluster
# BucketClaim status lookup and uses this string as the S3 bucket name.
# Leave empty in production — the COSI driver assigns its own bucket
# name independent of .bucketName above, and the BucketClaim status
# is the only authoritative source.
bucketNameOverride: ""
# endpoint is the S3 endpoint URL shared by every default Strategy CR, the
# Velero BSL and the controller Deployment. For a provisioned bucket
# (provisionBucket: true, the default) it is NOT used verbatim: the
# "backupstrategy-controller.endpoint" helper derives the real endpoint from
# the COSI bucket's system credentials Secret (backupStorage.systemSecretName)
# and forces https://, because Cozystack's SeaweedFS runs with
# enableSecurity=true and serves TLS on :8333 behind the self-signed
# "SeaweedFS CA". The COSI-advertised endpoint is the external S3 ingress
# (ACME cert), which every consumer can verify — unlike the in-cluster
# service, whose CA is not plumbed to the backup operators and which the Etcd
# Strategy's S3 schema (no caCert field) cannot target at all.
#
# This value is the FALLBACK, used only when the derivation cannot run:
# - provisionBucket: false (external S3) — set it to your S3 endpoint,
# scheme included; it is used verbatim (the MariaDB/FDB strategies compute
# their secure flag from the scheme, so https:// enables TLS there — make
# the CA bundle reachable to those operators before pointing at a
# self-signed external endpoint);
# - offline `helm template` / unit renders and the pre-reconcile first
# install, where the Secret lookup returns nothing.
endpoint: http://seaweedfs-s3.tenant-root.svc.cozy.local:8333
# region is re-projected into cozy-backups-creds on the next reconcile
# (1-minute tick for system namespaces, on-demand for tenant namespaces).
# However, charts that consume the region via secretKeyRef in env vars —
# the chart-emitted ClickHouse sidecar on `useSystemBucket: true` is the
# only such consumer today — only pick up the new value on Pod restart.
# Plan a rollout of those Pods when you change this value.
region: us-east-1
forcePathStyle: true
# systemSecretName — the human-friendly Secret produced by the Bucket
# application in the backupStorage.namespace (tenant-root by default). The Bucket release is
# wrapped with the bucket-rd "bucket-" prefix (see
# packages/system/bucket-rd/cozyrds/bucket.yaml), so the human-friendly
# Secret rendered by packages/system/bucket/templates/user-credentials.yaml
# is named <bucket-release>-<user>-credentials. The projector also
# tolerates the raw COSI Secret format (single BucketInfo JSON key) so a
# cluster that wires an external S3 Secret manually still works.
systemSecretName: bucket-cozy-backups-system-credentials
# endpointCASecretName — optional Secret (key ca.crt) in the app namespace
# that the Redis strategy Pod trusts when the S3 endpoint is https with a
# self-signed cert (e.g. SeaweedFS). Empty by default: the projected endpoint
# is always https and the platform bucket's ACME cert verifies against the
# image's system CA store unaided, so no CA override is needed. Set it only for
# a private-CA endpoint. When set, the Secret is required — a missing one fails
# the Pod loudly rather than silently skipping verification.
endpointCASecretName: ""
# insecureSkipTLSVerify disables S3 certificate verification (curl -k). Off by
# default and an explicit opt-in, never a fallback: leaving it false keeps an
# untrusted-cert endpoint failing closed instead of silently insecure. Prefer
# endpointCASecretName for a self-signed endpoint.
insecureSkipTLSVerify: false
# systemNamespaces — namespaces where the controller eagerly projects
# cozy-backups-creds so long-lived static references (Velero BSL, FDB
# operator deployments) keep finding the Secret without on-demand
# rendering. Tenants are projected lazily during BackupJob reconcile.
systemNamespaces:
- cozy-velero
# reconcileDefaultObjects enables the controller's DefaultObjectsGate.
#
# The Strategy CRs and the Velero BSL below are Helm-templated behind a
# `lookup` of the BucketClaim THIS chart creates, because the S3 bucket
# name is assigned by the COSI driver (bucket-<claim-UID>) and cannot be
# computed at render time. On a fresh install the lookup is empty and those
# templates render nothing — and helm-controller does not re-render a
# release whose chart and values did not change (drift detection is off on
# operator-generated HelmReleases), so the skip is PERMANENT. Clusters have
# run for months with only the BackupClass present and no Strategy CRs at
# all; recovery needed a hand-stamped reconcile.fluxcd.io/forceAt +
# requestedAt, because a plain reconcile request does not re-render.
#
# The systemSecretName Secret above sits one release earlier in the same
# trap: packages/system/bucket/templates/user-credentials.yaml renders it
# behind a `lookup` of the COSI Secret, and skips it just as permanently.
# Without it the projector has no source at all, so nothing downstream can
# resolve — and neither can this gate, which reads the bucket name from it.
#
# With this enabled the controller repairs both, in order: while the
# credentials Secret is missing it forces the bucket's <bucket>-system
# release; once the bucket name resolves it checks that every object
# cozy-default routes to exists and forces THIS release when any is
# missing. Both are throttled, independently, and a no-op in the steady
# state. A suspended release is skipped rather than re-stamped forever. The
# gate does NOT create the objects itself: their bodies are values-driven
# and Helm stays their single owner.
#
# Turn this off only if you manage the default Strategy CRs out of band and
# accept that a missed install-time render is never repaired. Watch
# cozystack_backup_default_objects_missing (paired with
# cozystack_backup_default_objects_check_errors_total, which marks the
# gauge stale) if you do.
reconcileDefaultObjects: true
# velero — BSL rendering. The cozy-default BackupStorageLocation is
# created in cozy-velero by templates/velero-bsl.yaml so endpoint/bucket/
# region come from the same backupStorage block used by Strategy CRs and
# the projector, eliminating drift.
#
# bslEnabled defaults to TRUE because cozystack.velero is a hard
# dependsOn for this PackageSource (see
# packages/core/platform/sources/backupstrategy-controller.yaml) — the
# Velero CRDs are guaranteed to exist by the time this chart installs.
# The BSL template additionally gates itself on a populated BucketClaim
# status via the bucketName helper, so on a bootstrap install the BSL
# materialises only after the platform Bucket reconciles AND the
# DefaultObjectsGate has forced a real Helm upgrade (an interval reconcile
# alone does not re-render). Setting this
# to false is an explicit opt-out for clusters that disable VM backups
# entirely and want to avoid the cluster-default BSL. In that case the
# cozy-default-velero-vminstance / -vmdisk Strategy CRs are gated off the
# same flag and are NOT rendered either, so the DefaultObjectsGate does
# not expect them (it would otherwise force a Helm upgrade forever against
# a render that can never produce them). VM BackupJobs against cozy-default
# still fail, because the BackupClass keeps routing VMInstance / VMDisk to
# those absent strategies.
velero:
bslEnabled: true
namespace: cozy-velero