Seven guides in docs/ were written next to the code and left in the
source tree, where no operator looks. The website carries them now, so
a second copy here only lets the two drift again: the Backup Classes
page had already fallen behind its original.
docs/release.md, docs/deprecating-applications.md and the two SeaweedFS
runbooks in docs/operations stay. A migration, a bats suite, a workflow
or AGENTS.md reads each of them by path, which is the test for keeping
a document in this repository.
Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Without recoveryTarget.backupID the barman-cloud plugin chooses the
base backup itself: the newest one in the catalog when no recoveryTime
is set, and the newest one ending by recoveryTime otherwise, on any
timeline since no targetTLI is passed. A RestoreJob naming an older
Backup therefore started from another one, and on an archive holding
sibling timelines - left by in-place restores that archived back under
the source serverName - from another branch, bringing back that
branch's data.
The restore reads barman's backupId and the backup's stoppedAt from
the live cnpg.io/Backup behind the Backup, and passes the id through
the chart as bootstrap.recovery.recoveryTarget.backupID. The plugin
gives backupID precedence over time-based selection, and CNPG does not
count it among the mutually exclusive targets, so it combines with
targetTime.
Two cases keep the plugin's choice. A recoveryTime earlier than one
second past stoppedAt: the named backup cannot serve an instant before
it is consistent, and stoppedAt is truncated to the second while
barman compares its end time to the microsecond. And a cnpg.io/Backup
that is gone: plugin-barman-cloud's retention deletes it together with
the base backup it describes, and the driver deletes one only with its
artifact, so pinning its id would purge the target and then fail the
recovery on every retry. That restore records a BaseBackupGone warning
event instead.
A re-restore into the same target clears a backupID left by a previous
one.
Assisted-by: LLM
Signed-off-by: Mattia Eleuteri <mattia@hidora.io>
An in-place RestoreJob (targetApplicationRef omitted) re-bootstraps the
Postgres app under its own name. On stock v1.6.2 it never comes back up,
for two independent reasons in the same restore patch - both surface only
in place, because a to-copy restore targets a different app.
1. Archive-vs-recovery serverName collision. The driver set the restored
cluster's WAL-archive serverName equal to the recovery source
(bootstrap.serverName = bootstrap.oldName = sourceServerName) while the
chart pinned the archiver serverName to the release name. In place those
coincide, so the cluster archived onto the source's own WAL prefix and
barman-cloud-check-wal-archive refused to start ("Expected empty
archive"). Fixed by giving the restored cluster a fresh
bootstrap.newServerName, keyed by the RestoreJob UID (unique per
restore, stable across reconciles); the chart archives there while
externalClusters still recovers from bootstrap.serverName. This is what
the manual CNPG strategies hit (serverName defaults to the cluster
name); the cozy-default flow dodged it (serverName is <ns>-<app>).
2. useSystemBucket guard. The restore populates explicit S3 coordinates
(destinationPath / s3CredentialsSecret / endpointCA) but left
backup.useSystemBucket=true, so the chart's "bootstrap.enabled is
incompatible with useSystemBucket" guard failed the HelmRelease upgrade
and the cluster never re-rendered. This is what the cozy-default
(system-bucket) flow hits on an in-place restore. Fixed by clearing
useSystemBucket in the restore patch, since the app is now on the
explicit-creds path. The driver's app view (postgresapp.Backup) gains
the field, without omitempty so a false survives the merge patch.
Both are exercised only by an in-place restore; the postgres backup e2e
drove only to-copy, so neither surfaced in CI. A step 50 in-place restore
now guards reason 1 (manual-strategy flow); reason 2 is covered by a Go
unit assertion (a system-bucket in-place e2e needs the platform bucket in
CI and is left as a follow-up).
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
helm-unittest coverage for the S3 request-checksum fix (was mutation-negative
before): assert spec.instanceSidecarConfiguration.env pins
AWS_REQUEST_CHECKSUM_CALCULATION=when_required on all rendered barman-cloud
ObjectStores — postgres backup + bootstrap-recovery
(packages/apps/postgres/tests/backup_storage_test.yaml) and the Keycloak DB
(new packages/system/keycloak/tests/db_backup_test.yaml). Verified by mutation:
reverting the template change now fails these assertions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <andrey.kolkov@aenix.io>
Two defects of the same class — the WAL-archive S3 prefix (serverName)
changing or defaulting differently between components:
Recovery default regression (review blocker): native CNPG defaulted an
absent externalClusters serverName to the entry name (= bootstrap.oldName),
but the barman-cloud plugin defaults it to the NEW cluster's name, so a
legacy-flow restore with bootstrap.serverName unset would read the wrong
S3 prefix (hard failure, or silently restoring another cluster's data).
The chart now always emits serverName, defaulting to bootstrap.oldName.
Live-cluster serverName flip (found by the gated postgres backup e2e on a
real cluster): the chart renders spec.plugins without serverName
(effective prefix = cluster name) while a strategy template may name a
different one; the driver's ForceOwnership patch then flipped the prefix
mid-stream, splitting the WAL archive across two prefixes — the WAL
holding the backup's start was lost between them and the restore failed
with "WAL not found". The driver now preserves the live Cluster's
effective serverName (explicit parameter, or the cluster name when the
plugin is attached without one) and only applies the strategy's value to
a Cluster with no plugin attached; both charts pin serverName explicitly,
and the example strategy no longer sets one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Review follow-up (cozystack#3300, non-blocking): the barman-cloud ObjectStore CRD
validates spec.retentionPolicy against ^[1-9][0-9]*[dwm]$, so an operator who
explicitly sets backup.retentionPolicy="" would render retentionPolicy: "" and
the API server would reject the ObjectStore, breaking backup provisioning. Guard
the field in the apps/postgres and keycloak chart ObjectStores with
{{- if .Values.backup.retentionPolicy }} (the Go driver path is already safe via
omitempty). Add a helm-unittest asserting an empty retentionPolicy is omitted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Review follow-ups for the CNPG barman-cloud plugin migration (cozystack#3300):
- docs/operations/backup-classes.md: the 'Upgrade notes from chart-managed
backups' section still described the removed native spec.backup.barmanObjectStore
behavior. Rewrite it for the plugin flow (chart renders spec.plugins + a
barmancloud.cnpg.io/ObjectStore; the driver SSA-applies an ObjectStore and
patches spec.plugins), and document the real upgrade hazard: the
plugin-barman-cloud operator + cert-manager must be present in
cozy-postgres-operator or the Cluster references an unregistered plugin and
backups/WAL archiving stall.
- apps/postgres backup_storage_test.yaml: add a positive bootstrap-recovery
rendering test. Asserts externalClusters[].plugin references
barman-cloud.cloudnative-pg.io with barmanObjectName=<release>-recovery and
serverName on the plugin parameter, and that the -recovery ObjectStore carries
spec.configuration but NOT serverName (the plugin forbids it there). Previously
the recovery path had only fail-fast guard tests, no positive coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrey Kolkov <androndo@gmail.com>