Andrey Kolkov
22b1e074a5
An in-place RestoreJob (targetApplicationRef omitted) re-bootstraps the
Postgres app under its own name. On stock v1.6.2 it never comes back up,
for two independent reasons in the same restore patch - both surface only
in place, because a to-copy restore targets a different app.
1. Archive-vs-recovery serverName collision. The driver set the restored
cluster's WAL-archive serverName equal to the recovery source
(bootstrap.serverName = bootstrap.oldName = sourceServerName) while the
chart pinned the archiver serverName to the release name. In place those
coincide, so the cluster archived onto the source's own WAL prefix and
barman-cloud-check-wal-archive refused to start ("Expected empty
archive"). Fixed by giving the restored cluster a fresh
bootstrap.newServerName, keyed by the RestoreJob UID (unique per
restore, stable across reconciles); the chart archives there while
externalClusters still recovers from bootstrap.serverName. This is what
the manual CNPG strategies hit (serverName defaults to the cluster
name); the cozy-default flow dodged it (serverName is <ns>-<app>).
2. useSystemBucket guard. The restore populates explicit S3 coordinates
(destinationPath / s3CredentialsSecret / endpointCA) but left
backup.useSystemBucket=true, so the chart's "bootstrap.enabled is
incompatible with useSystemBucket" guard failed the HelmRelease upgrade
and the cluster never re-rendered. This is what the cozy-default
(system-bucket) flow hits on an in-place restore. Fixed by clearing
useSystemBucket in the restore patch, since the app is now on the
explicit-creds path. The driver's app view (postgresapp.Backup) gains
the field, without omitempty so a false survives the merge patch.
Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place restore
now guards reason 1 (manual-strategy flow); reason 2 is covered by a Go
unit assertion (a system-bucket in-place e2e needs the platform bucket in
CI and is left as a follow-up).
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit 5bcdb8cfb8)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Assisted-By: LLM
PostgreSQL backup & restore (barman-cloud plugin)
End-to-end demo of backing up a cozystack Postgres application to S3 and restoring it, using the CloudNativePG barman-cloud plugin (the native spec.backup.barmanObjectStore path is deprecated in CNPG 1.27 and removed in 1.29). Backups flow through the platform BackupClass + CNPG strategy so one strategy CR serves every Postgres instance in the tenant.
What it provisions
The numbered manifests are the documented flow; a human can read and apply them by hand, and run-all.sh drives the same files for a smoke test.
00-bucket.yaml— a cozystack-managed S3Bucketfor the backups, with abackupuser.05-postgres-src.yaml— the sourcePostgres, withbackup.enabled: truesoarchive_commandstreams WALs from helm-install onwards.10-cnpg-strategy.yaml— theCNPGstrategy templated against the live application.15-backupclass.yaml— mapsKind=Postgresto that strategy.20-plan.yaml— an optional cronPlanfor recurring backups.25-backupjob-adhoc.yaml— an ad-hocBackupJob.30-postgres-target.yaml+40-restorejob-to-copy.yaml— restore into a fresh copy, leaving the source running.35-restorejob-in-place.yaml— the destructive in-place restore variant.
Placeholders and derived Secrets
05-postgres-src.yaml and 10-cnpg-strategy.yaml carry REPLACE_WITH_COSI_BUCKET_NAME, REPLACE_WITH_S3_ENDPOINT and REPLACE_WITH_PASSWORD. They also reference two Secrets, <app>-cnpg-backup-creds (the S3 credentials the barman-cloud sidecar reads) and <app>-cnpg-backup-ca (the CA it trusts for a self-signed endpoint). The strategy template renders those names against whichever application it drives, so the restore target needs its own pair too — a restore against pg-target looks up pg-target-cnpg-backup-*. run-all.sh resolves the placeholders from the provisioned Bucket's BucketInfo Secret and materialises the pair for both the source and the target; editing by hand, you copy the coordinates from kubectl -n <ns> get secret bucket-<name>-backup -o jsonpath='{.data.BucketInfo}' and the CA from the seaweedfs CA secret in tenant-root (seaweedfs-ca-cert by default; run-all.sh auto-discovers it from the cert-manager Certificate if the name differs).
Drop the endpointCA block from 05/10 (and set S3_CA_SECRET="" for run-all.sh) when the S3 endpoint is signed by a publicly-trusted CA.
Run it
# Defaults to NAMESPACE=tenant-root; override for a tenant namespace.
NAMESPACE=tenant-root examples/backups/postgres/run-all.sh
# Tear everything down afterwards (idempotent).
NAMESPACE=tenant-root examples/backups/postgres/cleanup.sh
run-all.sh writes a sentinel row into the source, waits for the BackupJob to reach Succeeded, restores to a copy, and asserts the sentinel round-tripped through S3 into the restored copy. Set SKIP_RESTORE=1 to stop after a successful backup.
Automated e2e
The Chainsaw suite at hack/e2e-chainsaw/postgres/ drives this same run-all.sh as a second test (postgres-2-backup-roundtrip), selected by CI whenever the postgres app (or these scripts) change and on every release cut. It runs in tenant-root against the in-cluster seaweedfs endpoint — the isolated e2e tenant cannot reach it across the Cilium egress policy, and the external ingress endpoint is an unroutable placeholder in the sandbox.