Andrey Kolkov
9cc3a9e459
The disabled hack/e2e-chainsaw/backup/ suite was a Chainsaw port of the retired
backup.bats, written against the old backup API, so the VMInstance and standalone
VMDisk Velero paths had no e2e coverage. Replace it with two round-trips in the
vminstance suite that drive the validated examples/backups/{vmi,vmdisk} scripts,
so the documented demo and the automated test stay in step.
Both run un-gated in CI:
- vminstance-2-backup-roundtrip boots a tiny cirros VM (u1.nano, 2Gi), backs it
up via BackupClass + BackupJob, restores in place, and asserts the VM returns
Running off a disk PVC the restore created: a new uid, Velero's restore-name
label and the -orig- PVC keepOriginalPVC leaves behind. The label alone is not
enough, because existingResourcePolicy: update also stamps it on a PVC Velero
only patched, which moves no data. A guest-side boot proof is unavailable: the
KubeVirt CR disables serial-console logging and cirros ships no guest-agent.
- vmdisk-2-backup-roundtrip imports a 5Gi disk, backs it up, and restores it in
place with the same three assertions. The controller prepares the target
(suspend HelmRelease, delete DataVolume, rename PVC). On a WaitForFirstConsumer
class the Velero data mover waits for a Pod to select a node for the restored
PVC, which a detached disk never gets, so the script starts a pause Pod on the
new PVC, standing in for the VM that would attach it.
The disk storageClass is an override (default replicated; the CI steps pass the
lane's COZY_E2E_STORAGE_CLASS, 'local' on the DRBD-off gate) and the script waits
are bounded so their worst case stays under the 40m op timeout, sized to keep the
whole parallel:1 suite inside the 215m job cap. On failure the catch also dumps
the Velero Restore, DataDownload and DataUpload objects. Each harness stands up a
throwaway in-cluster BSL (the platform cozy-default BSL derives the external S3
endpoint, unroutable in kind); its finally resumes the suspended HelmReleases,
reclaims the keepOriginalPVC orig PVC and its LINSTOR volume, and drains the
Velero backups before deleting the BSL. The example scripts are non-interactive
so the harness can drive them.
Assisted-by: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
3.3 KiB
VMDisk backup/restore example
This directory shows how to back up and restore a standalone Cozystack VMDisk (a HelmRelease plus a DataVolume-backed PVC, with no VMInstance attached) using the cluster's Velero backup strategy driver. Each BackupJob drives a Velero CSI-snapshot backup whose data mover (kopia) streams the LINSTOR block volume to S3; each in-place RestoreJob recreates the disk from that artefact.
The numbered scripts run in order (or via run-all.sh):
01-create-strategy.sh— aVelerostrategy for theVMDiskkind.02-create-backupclass.sh— aBackupClassbindingVMDiskto it.03-create-vmdisk.sh— a standaloneVMDisk, waited until imported.04-create-backupjob.sh— an ad-hocBackupJob, waited toSucceeded.05-restore-in-place.sh— an in-placeRestoreJob, then asserts that the PVC was recreated from the backup: it has a new uid, carries Velero'svelero.io/restore-namelabel, and the original survives as<name>-orig-<hash>.
cleanup.sh tears everything down non-interactively.
In-place restore
The driver prepares a VMDisk restore target the same way it does a VMInstance disk: it suspends the vm-disk-<name> HelmRelease and deletes its DataVolume (so Flux/CDI do not recreate the PVC and race the data mover), and, with keepOriginalPVC: true, renames the live PVC to <name>-orig-<hash> so that Velero creates a fresh PVC under the original name. cleanup.sh resumes the HelmRelease and reclaims the renamed PVC.
A disk that a running VM mounts cannot be restored in place: the RestoreJob fails before it changes anything and names the Pod that holds the disk. Stop the VM, or restore the VMInstance instead.
If the restore fails after this preparation, nothing is rolled back. The failure message and a Warning event name the HelmRelease that preparation suspended and the <name>-orig-<hash> PVC that holds the original data. If Velero got as far as restoring the HelmRelease it is no longer suspended, and Flux may already have recreated the disk; with no restored PVC under the disk's name, CDI imports a fresh disk there. Either way, check what the disk holds, and restore again or bind the original PV back to a PVC under the disk's name if it is not what you expect.
Detached disks and WaitForFirstConsumer
On a StorageClass with volumeBindingMode: WaitForFirstConsumer, such as the local class the CI lane uses, the Velero data mover does not start until a scheduled Pod has selected a node for the restored PVC. A VMInstance restore gets that from its virt-launcher Pod. A detached disk has no consumer, so its RestoreJob stays Running until something mounts the PVC, and fails once Velero's expose timeout passes. 05-restore-in-place.sh therefore starts a minimal pause Pod on the restored PVC, standing in for the VM that would attach the disk, and deletes it once the restore has succeeded. On a real cluster, attaching the disk to a VMInstance does the same.
Overrides
Environment variables (see 00-helpers.sh) tune the flow: NAMESPACE, BACKUP_STORAGE_LOCATION (defaults to the platform cozy-default BSL), VMI_DISK_STORAGE_CLASS (defaults to replicated; set it to your cluster's class), and the IMPORT_WAIT / BACKUP_WAIT / RESTORE_WAIT budgets. hack/e2e-chainsaw/vminstance/ drives this directory as the vmdisk-2-backup-roundtrip e2e test.