Files
Andrey KolkovandClaude Opus 4.8 b574e2176f fix(backups): cover Go-driven system-bucket ObjectStore + share the checksum helper
Address review on the S3 request-checksum fix:

- Extend the fix to the platform-managed useSystemBucket=true flow, whose
  ObjectStore is built in Go by applyClusterPluginBackup (SSA-applied), not
  chart-rendered. Add InstanceSidecarConfiguration to cnpgtypes.ObjectStoreSpec
  and set AWS_REQUEST_CHECKSUM_CALCULATION=when_required on it via
  barmanSidecarConfiguration(). The platform's own default system bucket is
  SeaweedFS, a non-AWS S3 gateway of the same class as Ceph RGW. Unit test in
  cnpgstrategy_controller_test.go asserts the applied ObjectStore carries it.

- Factor the duplicated instanceSidecarConfiguration block (3x across two
  charts) into a shared cozy-lib helper (cozy-lib.barman.checksumSidecarConfiguration).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
2026-07-22 15:35:32 +04:00

312 lines
15 KiB
YAML

{{- $tlsEnabled := (include "postgres.tls.enabled" .) | eq "true" -}}
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: {{ .Release.Name }}
spec:
instances: {{ .Values.replicas }}
{{- if and $tlsEnabled .Values.external }}
# Operator-managed TLS. CNPG auto-generates its own self-signed CA and signs
# server + client + replication certs from it. The operator's default SAN list
# already covers the three built-in services (rw/r/ro) in four DNS forms each
# (bare, .ns, .ns.svc, .ns.svc.<cluster-domain>); only the external hostname
# needs to be appended via serverAltDNSNames.
#
# serverAltDNSNames is mutually exclusive with serverTLSSecret per the CNPG
# admission webhook, which is why the chart no longer renders a cert-manager
# chain — the operator owns the chain end-to-end.
#
# Trust anchor for clients: the ca.crt key present in every user-credentials
# Secret CNPG creates (e.g. <release>-credentials). Mount that key into
# client pods to verify TLS.
certificates:
serverAltDNSNames:
- {{ printf "%s.%s" .Release.Name .Values._namespace.host }}
{{- end }}
{{/*
s3Credentials points at the operator-supplied Secret when
backup.s3CredentialsSecret.name is set; otherwise it falls back to the
chart-managed <release>-s3-creds Secret materialised from s3AccessKey/
s3SecretKey. The override path keeps credentials out of the Postgres
CR .spec — the CNPG backup driver uses it on restore.
*/}}
{{- $credCfg := default (dict) .Values.backup.s3CredentialsSecret }}
{{- $credSecretName := $credCfg.name | default (printf "%s-s3-creds" .Release.Name) }}
{{/*
Platform-managed default flow (useSystemBucket=true):
- The chart does NOT render spec.plugins or the barman-cloud
ObjectStore. The backup-controller's CNPG driver SSA-applies a
barmancloud.cnpg.io ObjectStore (carrying destinationPath/
endpointURL/credentials) and SSA-patches the live Cluster's
spec.plugins to reference it at first BackupJob time, using field
manager "cozystack-cnpg-backup-driver" with ForceOwnership (see
internal/backupcontroller/cnpgstrategy_controller.go
applyClusterPluginBackup). SSA semantics mean the chart's Helm
field manager never owns spec.plugins, so a subsequent Flux
re-render of this chart leaves the driver's patch intact — no
tug-of-war on the live Cluster.
Coordinates come from a single backupStorage values block in
the backupstrategy-controller chart. This eliminates the
endpoint-duplication failure mode (an operator switching the
platform endpoint would otherwise have to update both the
backupstrategy-controller and this chart).
- Consequence: WAL archiving via the plugin is not active until the
first BackupJob has run. WAL accumulates on the local PVC in the
meantime — fine for small / fresh clusters, but admins of large
Postgres releases switching mid-flight should run an ad-hoc
BackupJob immediately after enabling the flag.
- The chart-emitted <release>-s3-creds Secret is also skipped, so a
default install no longer leaks placeholder credentials into the
tenant namespace.
*/}}
{{- $destinationPath := .Values.backup.destinationPath }}
{{- $endpointURL := .Values.backup.endpointURL }}
{{/*
accessKeyIDKey/secretAccessKeyKey only apply when an external Secret was
supplied via backup.s3CredentialsSecret.name. The chart-managed
<release>-s3-creds Secret materialised by backup-secret.yaml hard-codes
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY, so applying overrides in
that mode would point CNPG at non-existent keys and break credential
reads. Default both names unconditionally and let the override take
effect only in external-Secret mode.
*/}}
{{- $credAccessKey := "AWS_ACCESS_KEY_ID" }}
{{- $credSecretKey := "AWS_SECRET_ACCESS_KEY" }}
{{- if $credCfg.name }}
{{- $credAccessKey = $credCfg.accessKeyIDKey | default "AWS_ACCESS_KEY_ID" }}
{{- $credSecretKey = $credCfg.secretAccessKeyKey | default "AWS_SECRET_ACCESS_KEY" }}
{{- end }}
{{/*
endpointCA references a Secret carrying the CA bundle the barman-cloud
plugin should trust. Required when the S3 endpoint serves a self-signed
certificate (the cozystack default seaweedfs-s3 ingress qualifies). The
same reference flows into both the backup ObjectStore's
spec.configuration.endpointCA and the recovery ObjectStore's
spec.configuration.endpointCA (referenced from externalClusters[].plugin)
so backup *and* recovery can verify TLS - missing it on the recovery
side made bootstrap.recovery panic in CNPG's instance-manager.
*/}}
{{- $caCfg := default (dict) .Values.backup.endpointCA }}
{{- $caKey := $caCfg.key | default "ca.crt" }}
{{/*
The barman-cloud plugin is chart-wired only when the legacy path has
real coordinates:
- backup.enabled=true AND
- useSystemBucket=false (platform flow lets the CNPG driver SSA-
patch spec.plugins + the ObjectStore instead) AND
- destinationPath is non-empty AND either an inline access/secret
pair is supplied OR an external Secret reference (s3Credentials-
Secret.name) is set.
Anything short of that means the user has enabled backup but not yet
supplied S3 coordinates — wiring the plugin against a half-empty
ObjectStore would just feed CNPG a broken archive_command. Tenants
opting into `useSystemBucket: true` keep spec.plugins unset until the
backup-controller's CNPG driver SSA-patches it on the first BackupJob.
The S3/barman settings live in the barmancloud.cnpg.io/ObjectStore CR
rendered at the bottom of this file (named after the release); the
Cluster only references it by name. Native spec.backup.barmanObjectStore
is deprecated in CNPG 1.27 and removed in 1.29. The bare (system-flavor)
image tags this chart pins still ship the barman-cloud-* binaries the
native path needs (unlike the `standard` variant keycloak-db pins), but
the system flavor is deprecated upstream, so this chart moves to the
plugin as well.
*/}}
{{- $hasInlineCreds := and (ne (default "" .Values.backup.s3AccessKey) "") (ne (default "" .Values.backup.s3SecretKey) "") }}
{{- $hasLegacyCreds := or $hasInlineCreds (ne (default "" $credCfg.name) "") }}
{{- $renderBarman := and .Values.backup.enabled (not .Values.backup.useSystemBucket) (ne $destinationPath "") $hasLegacyCreds }}
{{- if $renderBarman }}
plugins:
- name: barman-cloud.cloudnative-pg.io
isWALArchiver: true
parameters:
barmanObjectName: {{ .Release.Name }}
# Pin the WAL-archive S3 prefix explicitly (the plugin would default it
# to the Cluster name anyway). The prefix must never change on a live
# cluster, and the CNPG backup driver preserves whatever it finds here.
serverName: {{ .Release.Name }}
{{- end }}
{{/*
bootstrap.enabled + backup.useSystemBucket is unsupported in this
chart: bootstrap.recovery requires an explicit externalClusters entry
with destinationPath / endpointURL / s3Credentials, but useSystemBucket
deliberately leaves the platform coordinates out of values.yaml (they
live in the backupstrategy-controller chart). Rendering the bootstrap
block here with empty coordinates would produce a broken cnpg.io
Cluster CR and a confusing CNPG bootstrap failure.
The supported way to restore against the platform bucket is a
backups.cozystack.io/RestoreJob targeting a fresh Postgres app — the
backup-controller's CNPG driver SSA-patches the Cluster's
spec.bootstrap before CNPG sees it. See the BackupClass and Restore
flow in docs/operations/backup-classes.md.
Fail fast at chart-render time so tenants see the misconfiguration
immediately rather than during a Postgres bootstrap pod crash loop.
*/}}
{{- if and .Values.bootstrap.enabled .Values.backup.useSystemBucket }}
{{- fail "postgres: bootstrap.enabled=true is incompatible with backup.useSystemBucket=true. Use a backups.cozystack.io/RestoreJob against the cozy-default BackupClass to restore from the system bucket; see docs/operations/backup-classes.md." }}
{{- end }}
{{/*
The externalClusters bootstrap block below references $credSecretName
(default <release>-s3-creds) for recovery. That Secret is only
materialised by backup-secret.yaml when inline creds are present, so a
legacy-flow bootstrap with no usable credentials would render a cnpg.io
Cluster whose recovery s3Credentials points at a Secret that never
exists — CNPG then crash-loops with an opaque "secret not found".
Fail closed at render time (mirroring the useSystemBucket guard above)
so the misconfiguration is obvious immediately. useSystemBucket-mode
restore is the RestoreJob flow, already rejected by the guard above.
*/}}
{{- if and .Values.bootstrap.enabled (not .Values.backup.useSystemBucket) (not $credCfg.name) (not $hasInlineCreds) }}
{{- fail "postgres: bootstrap.enabled=true requires S3 credentials: set backup.s3AccessKey+backup.s3SecretKey, backup.s3CredentialsSecret.name, or backup.useSystemBucket=true (with a backups.cozystack.io/RestoreJob)." }}
{{- end }}
{{- if .Values.bootstrap.enabled }}
bootstrap:
recovery:
source: {{ .Values.bootstrap.oldName }}
{{- if .Values.bootstrap.recoveryTime }}
recoveryTarget:
targetTime: {{ .Values.bootstrap.recoveryTime }}
{{- end }}
externalClusters:
- name: {{ .Values.bootstrap.oldName }}
plugin:
name: barman-cloud.cloudnative-pg.io
parameters:
# References the recovery ObjectStore rendered at the bottom of this
# file, which carries the source cluster's S3 coordinates. serverName
# (the source's S3 path prefix) rides on the plugin parameter, not the
# ObjectStore (the plugin forbids serverName there). It must always be
# set: the native path defaulted an absent serverName to the external
# cluster's NAME (= oldName), but the plugin defaults it to the NEW
# cluster's name — which would read the wrong S3 prefix on restore.
barmanObjectName: {{ .Release.Name }}-recovery
serverName: {{ .Values.bootstrap.serverName | default .Values.bootstrap.oldName }}
{{- end }}
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.resourcesPreset .Values.resources $) | nindent 4 }}
imageName: ghcr.io/cloudnative-pg/postgresql:{{ include "postgres.versionMap" $ | trimPrefix "v" }}
enableSuperuserAccess: true
{{- if .Values._cluster.scheduling }}
{{- $rawConstraints := get .Values._cluster.scheduling "globalAppTopologySpreadConstraints" }}
{{- if $rawConstraints }}
{{- $rawConstraints | fromYaml | toYaml | nindent 2 }}
labelSelector:
matchLabels:
cnpg.io/cluster: {{ .Release.Name }}
{{- end }}
{{- end }}
postgresql:
parameters:
{{- $defaults := dict "max_wal_senders" "30" }}
{{- $params := merge (dict) .Values.postgresql.parameters $defaults }}
{{- /* Security: Block parameters that enable arbitrary code execution (shell commands or .so loading) */ -}}
{{- $dangerousParams := list "archive_command" "restore_command" "ssl_passphrase_command" "archive_cleanup_command" "recovery_end_command" "dynamic_library_path" "local_preload_libraries" "session_preload_libraries" "shared_preload_libraries" }}
{{- range $key, $value := $params }}
{{- $normalizedKey := lower $key }}
{{- if has $normalizedKey $dangerousParams }}
{{- fail (printf "SECURITY: Parameter '%s' is blocked. This parameter can enable arbitrary code execution and must not be set by tenants." $key) }}
{{- end }}
{{ $key }}: {{ $value | toString | quote }}
{{- end }}
minSyncReplicas: {{ .Values.quorum.minSyncReplicas }}
maxSyncReplicas: {{ .Values.quorum.maxSyncReplicas }}
monitoring:
enablePodMonitor: true
storage:
size: {{ required ".Values.size is required" .Values.size }}
{{- with .Values.storageClass }}
storageClass: {{ . }}
{{- end }}
inheritedMetadata:
labels:
policy.cozystack.io/allow-to-apiserver: "true"
app.kubernetes.io/name: postgres.apps.cozystack.io
app.kubernetes.io/instance: {{ $.Release.Name }}
---
apiVersion: cozystack.io/v1alpha1
kind: WorkloadMonitor
metadata:
name: {{ $.Release.Name }}
labels:
workloads.cozystack.io/resource-preset: {{ .Values.resourcesPreset | quote }}
spec:
replicas: {{ .Values.replicas }}
minReplicas: 1
kind: postgres
type: postgres
selector:
app.kubernetes.io/name: postgres.apps.cozystack.io
app.kubernetes.io/instance: {{ $.Release.Name }}
version: {{ $.Chart.Version }}
{{- if $renderBarman }}
---
# Backup ObjectStore for the barman-cloud plugin. Carries the S3/barman
# settings the deprecated spec.backup.barmanObjectStore used to hold; the
# Cluster references it by name via spec.plugins above. serverName is
# intentionally omitted (the plugin forbids it here and defaults it to the
# cluster name, preserving the s3://.../<cluster>/ layout).
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: {{ .Release.Name }}
spec:
{{- if .Values.backup.retentionPolicy }}
retentionPolicy: {{ .Values.backup.retentionPolicy | quote }}
{{- end }}
# Pin the barman-cloud sidecar's S3 request checksum to when_required
# (rationale in cozy-lib.barman.checksumSidecarConfiguration).
{{- include "cozy-lib.barman.checksumSidecarConfiguration" . | nindent 2 }}
configuration:
destinationPath: {{ $destinationPath }}
endpointURL: {{ $endpointURL }}
{{- if $caCfg.name }}
endpointCA:
name: {{ $caCfg.name }}
key: {{ $caKey }}
{{- end }}
s3Credentials:
accessKeyId:
name: {{ $credSecretName }}
key: {{ $credAccessKey }}
secretAccessKey:
name: {{ $credSecretName }}
key: {{ $credSecretKey }}
{{- end }}
{{- if .Values.bootstrap.enabled }}
---
# Recovery ObjectStore for bootstrap. Points at the source cluster's S3
# coordinates; referenced from externalClusters[].plugin above. serverName
# rides on that plugin parameter, not here.
apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
name: {{ .Release.Name }}-recovery
spec:
# Same S3 request-checksum pin as the backup ObjectStore above.
{{- include "cozy-lib.barman.checksumSidecarConfiguration" . | nindent 2 }}
configuration:
destinationPath: {{ $destinationPath }}
endpointURL: {{ $endpointURL }}
{{- if $caCfg.name }}
endpointCA:
name: {{ $caCfg.name }}
key: {{ $caKey }}
{{- end }}
s3Credentials:
accessKeyId:
name: {{ $credSecretName }}
key: {{ $credAccessKey }}
secretAccessKey:
name: {{ $credSecretName }}
key: {{ $credSecretKey }}
{{- end }}