mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-03 10:37:21 +09:00
Andrey KolkovandClaude Opus 4.8
6a1e2d0d06
Makes RestoreJob.spec.options.recoveryTime a genuinely supported, tested capability for the new backups.cozystack.io API (issue #2774). The recoveryTime -> CNPG bootstrap.recovery.recoveryTarget.targetTime plumbing already existed on the barman-cloud-plugin branch; this adds the reliability and verification layer the issue asked for: - Fail-fast for an unreachable target: when a RestoreJob has recoveryTime set and the recovery Cluster is not healthy, the driver reads the CNPG full-recovery pod logs and, once several distinct attempts have exited with PostgreSQL's "recovery ended before configured recovery target was reached" FATAL, fails the RestoreJob with conditions RecoveryConverged=False / Ready=False (reason RecoveryTargetUnreachable) instead of hanging to the restore deadline. Health is checked first, so a converged recovery always wins; counting distinct FATAL-ing attempts (not pod failures) tolerates a transient archive-lag miss on a near-now target and ignores transient recovery-pod crashes. Adds the pods/log RBAC grant the log read requires. - e2e (examples/backups/postgres/run-all.sh, wired into the Chainsaw postgres-2-backup-roundtrip suite): step 45 writes a before marker, captures the server timestamp, writes an after marker, restores to that timestamp and asserts the before row survived and the after row did not; step 46 submits a recoveryTime an hour ahead and asserts it fails fast with reason RecoveryTargetUnreachable. - Working, uncommented example (45-restorejob-pitr.yaml) and a docs section (docs/operations/backup-classes.md) covering the recoverable window, how to discover the earliest/latest restorable time from the backup catalog, the archive-lag tolerance, and idempotency under GitOps. Refs #2774 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Andrey Kolkov <androndo@gmail.com>
55 lines
2.7 KiB
Bash
Executable File
55 lines
2.7 KiB
Bash
Executable File
#!/bin/bash
|
|
# Best-effort teardown of everything the demo creates. Idempotent;
|
|
# missing resources are skipped silently.
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
source "$SCRIPT_DIR/00-helpers.sh"
|
|
|
|
print_header "Cleanup"
|
|
|
|
# The CNPG driver suspends the source/target Postgres HelmRelease while a
|
|
# RestoreJob runs. If the demo was aborted mid-restore the HR is left
|
|
# suspended; deleting the RestoreJob without resuming it first strands the app.
|
|
# Resume both best-effort BEFORE deleting the RestoreJobs.
|
|
for app in "$PG_SRC_NAME" "$PG_TARGET_NAME"; do
|
|
if kubectl -n "$NAMESPACE" get hr "postgres-${app}" >/dev/null 2>&1; then
|
|
log_substep "Resuming HelmRelease postgres-${app} (no-op if not suspended)..."
|
|
kubectl -n "$NAMESPACE" patch hr "postgres-${app}" --type=merge \
|
|
-p '{"spec":{"suspend":false}}' >/dev/null || true
|
|
fi
|
|
done
|
|
|
|
log_substep "Deleting RestoreJobs..."
|
|
kubectl -n "$NAMESPACE" delete restorejob.backups.cozystack.io \
|
|
"$RESTOREJOB_TOCOPY_NAME" "$RESTOREJOB_PITR_NAME" "$RESTOREJOB_UNREACHABLE_NAME" --ignore-not-found
|
|
|
|
log_substep "Deleting Plan + BackupJob + derived Backup artefacts..."
|
|
kubectl -n "$NAMESPACE" delete plan.backups.cozystack.io "$PLAN_NAME" --ignore-not-found
|
|
kubectl -n "$NAMESPACE" delete backupjob.backups.cozystack.io \
|
|
"$BACKUPJOB_NAME" "$BACKUPJOB_POSTMARKER_NAME" --ignore-not-found
|
|
# Only the Backups of THIS demo's apps — never `--all`: on a shared cluster the
|
|
# namespace can hold Backup artefacts of other applications and flows.
|
|
for b in $(kubectl -n "$NAMESPACE" get backups.backups.cozystack.io \
|
|
-o jsonpath='{range .items[*]}{.metadata.name} {.spec.applicationRef.name}{"\n"}{end}' 2>/dev/null |
|
|
awk -v src="$PG_SRC_NAME" -v tgt="$PG_TARGET_NAME" '$2 == src || $2 == tgt {print $1}'); do
|
|
kubectl -n "$NAMESPACE" delete backups.backups.cozystack.io "$b" --ignore-not-found
|
|
done
|
|
|
|
log_substep "Deleting Postgres apps..."
|
|
kubectl -n "$NAMESPACE" delete postgres.apps.cozystack.io "$PG_SRC_NAME" "$PG_TARGET_NAME" --ignore-not-found
|
|
|
|
log_substep "Deleting per-app backup Secrets..."
|
|
kubectl -n "$NAMESPACE" delete secret \
|
|
"${PG_SRC_NAME}-cnpg-backup-creds" "${PG_SRC_NAME}-cnpg-backup-ca" \
|
|
"${PG_TARGET_NAME}-cnpg-backup-creds" "${PG_TARGET_NAME}-cnpg-backup-ca" --ignore-not-found
|
|
|
|
log_substep "Deleting Bucket..."
|
|
kubectl -n "$NAMESPACE" delete bucket.apps.cozystack.io "$BUCKET_NAME" --ignore-not-found
|
|
|
|
log_substep "Deleting BackupClass + CNPG strategy (cluster-scoped)..."
|
|
kubectl delete backupclass.backups.cozystack.io "$BACKUPCLASS_NAME" --ignore-not-found
|
|
kubectl delete cnpg.strategy.backups.cozystack.io "$STRATEGY_NAME" --ignore-not-found
|
|
|
|
log_success "Cleanup complete."
|