# Backup Protection

ServerKit runs on your server, so backups can be **scheduled, smart, and
one-click restorable** — no plugins, no PHP timeouts, no middlemen. The
**Protection** panel turns the old "save a snapshot" button into real
protection for both WordPress sites and applications (Services).

You'll find it in two equivalent places (both render the same panel):

- **WordPress**: site detail → top **Backups** tab, or **Settings → Backups**.
- **Services**: application detail → **Settings → Backups**.

The global **Backups** page remains the archive / storage-management and
cost-settings view.

## What "protection" means

- **Automatic** — one toggle turns on a daily (or custom) schedule.
- **Visible** — a list *and* a calendar show what succeeded and what failed.
- **Measurable** — every backup shows its size and an estimated storage cost.
- **Restorable** — one-click restore (full / files only / database only), with a
  safety backup taken first by default.
- **Smart** — periodic full backups plus daily incrementals, retention, and
  compression tiers.
- **Connected** — each run is a job on the unified job bus (visible in **Jobs**),
  and failures raise a notification.

## The three cards

1. **Protection status** — the master on/off switch, a status pill
   (Protected / Pending / Failed / Off), KPIs (next backup, monthly cost,
   storage used), and a **Back up now** button (works even when protection is
   off — it takes a one-off backup).
2. **Schedule** — frequency (Daily / Weekly / Custom with a cron preview),
   retention (keep last *N* backups **and** delete older than *N* days — both
   rules apply), how often a **full** backup is taken, the compression tier, and
   an optional **remote copy**.
3. **Backup history** — a list or calendar of every run. Click a row for a
   detail drawer (metadata, cost, verify/restore/delete); restore opens a
   drawer where you pick the scope and safety options.

## Smart backups

- **Full vs incremental** (applications): the first run, and every
  `full_every_n_days`, is a **full** backup; the days in between are
  **incremental** (`tar --listed-incremental`), so daily storage growth stays
  small. Restoring an incremental automatically replays its full + intervening
  increments.
- **Compression tiers**: `fast` (gzip), `balanced` (zstd ‑3, the default), and
  `max` (zstd ‑19). If `zstd` isn't installed, ServerKit falls back to gzip.
- **Retention**: a backup is kept only if it's within the last *N* backups **and**
  newer than *N* days. The most recent successful backup is **never** deleted,
  and the full/increments a kept backup depends on are protected from cleanup.
- **Remote copy + verify**: when enabled, each run is uploaded to your configured
  remote (S3 / B2) and the primary archive is verified (size + checksum).

## Cost

ServerKit is free and open-source — **"cost" means your own storage cost, not a
ServerKit charge**:

- **Local** = your server's disk. It's **free by default** ($0/GB); set a
  `$/GB/month` rate only if you want to attribute server-disk cost.
- **S3 / B2** = your real cloud-provider bill (sensible list-price defaults:
  S3 `$0.023`, B2 `$0.006` per GB/month).

Set the rates under **Backups → Settings → Storage cost rates**. The panel shows
both the raw **storage used** (GB on your server) and an estimated **monthly
cost**; the global page adds an **estimated** and **projected** (at full
retention) monthly cost.

## Restore (≤ 3 clicks)

Open a backup from the history → **Restore** → choose the scope and confirm:

- **Full site** (files + database), **Files only**, or **Database only**
  (WordPress). Applications restore files.
- **Create a safety backup first** (on by default) snapshots the current state
  before overwriting.
- **Maintenance mode** during restore (WordPress, on by default).

Restore runs as a `restore.run` job; you're notified on completion.

## How restore proof works

A backup you have never restored is an assumption, not a safety net. ServerKit
records how strongly each backup has been *proven*, on a ladder:

| Level | What it means |
|-------|---------------|
| `none` | Produced, nothing checked yet. |
| `listed` | Every archive is readable end to end (`tar -t`). |
| `hashed` | The primary archive still matches the sha256 recorded in its `manifest.json` — catches silent corruption. |
| `drilled` | A real restore of this point actually succeeded (strongest proof). |

`listed` and `hashed` run automatically at the end of **every** backup (inside
the existing `verifying` window). Each run writes a `manifest.json` next to its
archives — file list, per-artifact sha256, chain refs, tool versions — and the
primary archive's hash is stored on the run.

A **restore drill** earns `drilled`. On a per-policy cadence (**Off / Weekly /
Monthly**, set on the Schedule card) — or on demand with **Run drill now** — the
latest restorable point is restored into a throwaway scratch location (a
`restores/drill-<id>/` dir for files/apps/WordPress, a `skdrill_<run_id>`
scratch database for databases), probe-verified (file count + bytes + a hashed
sample; table listing for databases), then torn down. It never touches the live
target. Drilling the *latest* point proves the whole incremental chain by
construction. A free-space precheck marks an under-provisioned drill
`skipped_no_space` rather than failing mid-extract, and only one drill runs at a
time per host.

The Protection panel shows a **Last proven restore** badge (never / ok / stale /
failed) and the Doctor raises a repairable warning when a cadenced policy is
overdue (> 1.5× its cadence) or its latest backup is unverified.

For **remote copies**, a weekly check compares the offsite object's size and —
for single-part uploads — its ETag/MD5; multipart uploads are honestly labelled
`size_only` until a **monthly deep check** downloads one sampled object and
hashes it against the *stored* sha256 (never a hash recomputed from the local
file, which could share the same corruption). Egress stays bounded to one object
per month.

Drill/verify failures and recoveries notify admins **once per transition**
(`backup.drill_failed` / `backup.verify_failed` and their recovery notices), not
on every sweep. The diagnostic support bundle includes a scrubbed
`restore_confidence` summary (levels + states only, no paths or secrets).

## Under the hood

- **Models**: `BackupPolicy` (one per target) and `BackupRun` (one per run, the
  history's source of truth) — `backend/app/models/backup_policy.py`,
  `backup_run.py`.
- **Services**: `backup_policy_service.py` (policy CRUD, schedule wiring, the
  `backup.policy.run` / `restore.run` job handlers, retention),
  `backup_cost_service.py` (cost), and the smart-backup helpers in
  `backup_service.py`. Restore proof lives in `backup_verify_service.py`
  (manifest + tier-1 verifier), `backup_drill_service.py` (`backup.drill.run` +
  cadence/orphan sweeps, `RestoreDrill` model), `backup_offsite_service.py`
  (`backup.offsite.verify`), and `backup_alert_service.py` (edge-triggered
  drill/verify notices).
- **Scheduling**: the policy's cron is mirrored into a `ScheduledJob` on the
  unified job bus, so firing is observable and retryable like any other job.