Files
cozystack/packages/apps/postgres/tests/backup_storage_test.yaml
T
Andrey KolkovandClaude Opus 4.8 5bcdb8cfb8 fix(postgres): let in-place restore re-bootstrap instead of wedging
An in-place RestoreJob (targetApplicationRef omitted) re-bootstraps the
Postgres app under its own name. On stock v1.6.2 it never comes back up,
for two independent reasons in the same restore patch - both surface only
in place, because a to-copy restore targets a different app.

1. Archive-vs-recovery serverName collision. The driver set the restored
   cluster's WAL-archive serverName equal to the recovery source
   (bootstrap.serverName = bootstrap.oldName = sourceServerName) while the
   chart pinned the archiver serverName to the release name. In place those
   coincide, so the cluster archived onto the source's own WAL prefix and
   barman-cloud-check-wal-archive refused to start ("Expected empty
   archive"). Fixed by giving the restored cluster a fresh
   bootstrap.newServerName, keyed by the RestoreJob UID (unique per
   restore, stable across reconciles); the chart archives there while
   externalClusters still recovers from bootstrap.serverName. This is what
   the manual CNPG strategies hit (serverName defaults to the cluster
   name); the cozy-default flow dodged it (serverName is <ns>-<app>).

2. useSystemBucket guard. The restore populates explicit S3 coordinates
   (destinationPath / s3CredentialsSecret / endpointCA) but left
   backup.useSystemBucket=true, so the chart's "bootstrap.enabled is
   incompatible with useSystemBucket" guard failed the HelmRelease upgrade
   and the cluster never re-rendered. This is what the cozy-default
   (system-bucket) flow hits on an in-place restore. Fixed by clearing
   useSystemBucket in the restore patch, since the app is now on the
   explicit-creds path. The driver's app view (postgresapp.Backup) gains
   the field, without omitempty so a false survives the merge patch.

Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place restore
now guards reason 1 (manual-strategy flow); reason 2 is covered by a Go
unit assertion (a system-bucket in-place e2e needs the platform bucket in
CI and is left as a follow-up).

Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-25 17:31:53 +04:00

327 lines
11 KiB
YAML

suite: backup storage flow (barman-cloud plugin gating, chart Secret, bootstrap guards)
# Locks in the conditional rendering on the cnpg.io Cluster, the barman-cloud
# ObjectStore CR, and the chart-managed <release>-s3-creds Secret:
# - the legacy flow wires spec.plugins + renders the ObjectStore + the Secret
# only when real S3 coordinates are present;
# - the platform useSystemBucket flow leaves them out (the CNPG backup
# driver SSA-patches spec.plugins + the ObjectStore at first BackupJob time);
# - bootstrap.recovery fails fast at render time when it would otherwise
# emit a dangling s3Credentials reference (useSystemBucket, or no creds).
# These are template-rendering contracts, so helm-unittest is the right tool.
templates:
- templates/db.yaml
- templates/backup-secret.yaml
release:
name: pg-test
namespace: tenant-test
set:
_cluster:
scheduling: {}
tests:
- it: "useSystemBucket=true: chart skips spec.plugins, ObjectStore and <release>-s3-creds"
set:
backup:
enabled: true
useSystemBucket: true
asserts:
- notExists:
path: spec.plugins
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# No chart-rendered ObjectStore in the platform flow (the driver owns it):
# db.yaml emits only Cluster + WorkloadMonitor.
- hasDocuments:
count: 2
template: templates/db.yaml
- hasDocuments:
count: 0
template: templates/backup-secret.yaml
- it: "legacy backup with inline creds: wires spec.plugins + ObjectStore + <release>-s3-creds"
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: AKIAEXAMPLE
s3SecretKey: s3cr3t
asserts:
# The Cluster references the barman-cloud plugin, not native barmanObjectStore.
- notExists:
path: spec.backup.barmanObjectStore
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- equal:
path: spec.plugins[0].name
value: barman-cloud.cloudnative-pg.io
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- equal:
path: spec.plugins[0].parameters.barmanObjectName
value: pg-test
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# The WAL-archive S3 prefix is pinned explicitly to the release name and
# must never change on a live cluster (the driver preserves it).
- equal:
path: spec.plugins[0].parameters.serverName
value: pg-test
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# The ObjectStore carries the S3 credentials reference.
- equal:
path: metadata.name
value: pg-test
template: templates/db.yaml
documentSelector:
path: kind
value: ObjectStore
- equal:
path: spec.configuration.s3Credentials.accessKeyId.name
value: pg-test-s3-creds
template: templates/db.yaml
documentSelector:
path: kind
value: ObjectStore
# The barman-cloud sidecar's S3 request checksum is pinned to when_required
# (non-AWS S3 gateways reject the botocore default flexible checksum).
- equal:
path: spec.instanceSidecarConfiguration.env[0].name
value: AWS_REQUEST_CHECKSUM_CALCULATION
template: templates/db.yaml
documentSelector:
path: kind
value: ObjectStore
- equal:
path: spec.instanceSidecarConfiguration.env[0].value
value: when_required
template: templates/db.yaml
documentSelector:
path: kind
value: ObjectStore
- hasDocuments:
count: 1
template: templates/backup-secret.yaml
- equal:
path: metadata.name
value: pg-test-s3-creds
template: templates/backup-secret.yaml
- it: "empty retentionPolicy is omitted from the ObjectStore (plugin CRD rejects \"\")"
# The barman-cloud ObjectStore CRD validates retentionPolicy against
# ^[1-9][0-9]*[dwm]$, so an explicit empty value must be omitted rather than
# rendered as retentionPolicy: "" (which the API server rejects).
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: AKIAEXAMPLE
s3SecretKey: s3cr3t
retentionPolicy: ""
asserts:
- notExists:
path: spec.retentionPolicy
template: templates/db.yaml
documentSelector:
path: kind
value: ObjectStore
- it: "backup enabled, no creds, useSystemBucket=false: plugin + ObjectStore not rendered"
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: ""
s3SecretKey: ""
asserts:
- notExists:
path: spec.plugins
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# No ObjectStore rendered: db.yaml emits only Cluster + WorkloadMonitor.
- hasDocuments:
count: 2
template: templates/db.yaml
- hasDocuments:
count: 0
template: templates/backup-secret.yaml
- it: "bootstrap recovery: externalClusters[].plugin + a -recovery ObjectStore (serverName on the plugin param, not the ObjectStore)"
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: AKIAEXAMPLE
s3SecretKey: s3cr3t
destinationPath: s3://bucket/pg-test/
endpointURL: https://s3.example:9000
bootstrap:
enabled: true
oldName: old-pg
serverName: old-server
asserts:
# Recovery uses the barman-cloud plugin, not the removed externalClusters[].barmanObjectStore.
- notExists:
path: spec.externalClusters[0].barmanObjectStore
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- equal:
path: spec.externalClusters[0].plugin.name
value: barman-cloud.cloudnative-pg.io
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- equal:
path: spec.externalClusters[0].plugin.parameters.barmanObjectName
value: pg-test-recovery
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# serverName rides on the plugin parameter...
- equal:
path: spec.externalClusters[0].plugin.parameters.serverName
value: old-server
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# ...and the -recovery ObjectStore carries the source S3 config but MUST NOT set
# serverName inside spec.configuration (the plugin forbids it there).
- equal:
path: spec.configuration.destinationPath
value: s3://bucket/pg-test/
template: templates/db.yaml
documentSelector:
path: metadata.name
value: pg-test-recovery
- notExists:
path: spec.configuration.serverName
template: templates/db.yaml
documentSelector:
path: metadata.name
value: pg-test-recovery
# The non-AWS S3 request-checksum pin is applied to the recovery ObjectStore too.
- equal:
path: spec.instanceSidecarConfiguration.env[0].name
value: AWS_REQUEST_CHECKSUM_CALCULATION
template: templates/db.yaml
documentSelector:
path: metadata.name
value: pg-test-recovery
- equal:
path: spec.instanceSidecarConfiguration.env[0].value
value: when_required
template: templates/db.yaml
documentSelector:
path: metadata.name
value: pg-test-recovery
- it: "in-place restore: archiver serverName uses bootstrap.newServerName, distinct from the recovery source"
# Worst case: source serverName equals the restored cluster's own release
# name (pg-test). The chart must archive under bootstrap.newServerName (a
# fresh prefix) while recovering from bootstrap.serverName, or the restore
# would fail barman-cloud-check-wal-archive with "Expected empty archive".
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: AKIAEXAMPLE
s3SecretKey: s3cr3t
destinationPath: s3://bucket/pg-test/
endpointURL: https://s3.example:9000
bootstrap:
enabled: true
oldName: pg-test
serverName: pg-test
newServerName: pg-test-restore-deadbeef
asserts:
# WAL archiver writes to the fresh restore prefix, never the source's.
- equal:
path: spec.plugins[0].parameters.serverName
value: pg-test-restore-deadbeef
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
# Recovery still reads the source prefix.
- equal:
path: spec.externalClusters[0].plugin.parameters.serverName
value: pg-test
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- it: "bootstrap recovery without serverName: plugin serverName defaults to oldName (native parity)"
# Native CNPG defaulted an absent serverName to the externalClusters entry
# name (= oldName); the barman-cloud plugin defaults it to the NEW cluster's
# name, which would read the wrong S3 prefix. The chart must therefore
# always emit serverName, defaulting to oldName.
set:
backup:
enabled: true
useSystemBucket: false
s3AccessKey: AKIAEXAMPLE
s3SecretKey: s3cr3t
destinationPath: s3://bucket/pg-test/
endpointURL: https://s3.example:9000
bootstrap:
enabled: true
oldName: old-pg
serverName: ""
asserts:
- equal:
path: spec.externalClusters[0].plugin.parameters.serverName
value: old-pg
template: templates/db.yaml
documentSelector:
path: kind
value: Cluster
- it: "bootstrap + useSystemBucket: render fails fast (lock in existing guard)"
template: templates/db.yaml
set:
backup:
useSystemBucket: true
bootstrap:
enabled: true
oldName: old
asserts:
- failedTemplate:
errorMessage: "postgres: bootstrap.enabled=true is incompatible with backup.useSystemBucket=true. Use a backups.cozystack.io/RestoreJob against the cozy-default BackupClass to restore from the system bucket; see docs/operations/backup-classes.md."
- it: "bootstrap without credentials: render fails fast"
template: templates/db.yaml
set:
backup:
enabled: false
useSystemBucket: false
s3AccessKey: ""
s3SecretKey: ""
bootstrap:
enabled: true
oldName: old
asserts:
- failedTemplate:
errorMessage: "postgres: bootstrap.enabled=true requires S3 credentials: set backup.s3AccessKey+backup.s3SecretKey, backup.s3CredentialsSecret.name, or backup.useSystemBucket=true (with a backups.cozystack.io/RestoreJob)."