Myasnikov Daniil
a2d9cc766c
Image overrides let a tenant run an arbitrary container in the
management cluster, including helpers with access to the Talos CA
secrets and the tenant cluster's API credentials. Drop those overrides
and run the images the platform ships.
The Kubernetes and KubernetesNodes helpers (the wait-for-kubeconfig
and Proxmox cloud-config init containers, the bootstrap-token and OIDC
RBAC Jobs, the talos-csr-signer sidecar, the Talos reconcile and unpin
Jobs) take the bundled image references through cozy-lib.image, so
they still follow the configured registry. OpenSearch no longer sets
an image and leaves it to the operator, which follows the version the
chart's version map selects; an explicit version: null now fails the
render instead of falling back to v2. The Talos installer settings for
tenant worker VMs stay.
FoundationDB replaces the free-form cluster.version with version, v7.3
or v7.4, mapped to 7.3.63 and 7.4.1; the 7.1 line is no longer
offered. The chart refuses a render where cluster.version and version
name different release lines, which also covers variants without
platform migrations. Migration 60 carries an existing cluster.version
over to its release line and leaves the legacy value in place while
the old chart can still render it. It records what needs attention in
the cozystack.io/migration-60-needs-attention annotation on the
cozystack-version ConfigMap: a FoundationDB release it cannot carry
over or whose patch changes, a release that still sets one of the
removed image overrides, an OpenSearch release storing version: null.
A dry run reports them without patching, annotating or advancing the
stamp. Migration 56 stops carrying the removed kubectl override into
adopted node pools.
OpenSearch and FoundationDB get version-map update scripts that keep
the current default while it is still supported.
Stored image overrides are still accepted and the charts ignore them.
On the first reconcile OpenSearch nodes move to the image of their
selected version and Kubernetes components whose image changed
restart. Air-gapped installations need docker.io and ghcr.io mirrored
on the management-cluster nodes.
Assisted-by: LLM
(cherry picked from commit 749d401899)
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
FoundationDB backup/restore example
FoundationDB is currently the one supported Kind that is NOT bound by the platform-managed
cozy-defaultBackupClass. Restore runsfdbrestorefrom thecozy-foundationdb-operatorpod, which does not yet mount the projectedcozy-backups-credsSecret. Until that gap is closed, FDB backup/restore goes through this walk-through (per-app Bucket +blob_credentials.jsonSecret + custom BackupClass). Thecozy-default-foundationdbstrategy CR is shipped and you can bind it in a custom BackupClass when you wire the operator deployment manually.
This directory shows how to back up and restore a Cozystack-managed
FoundationDB application using the cluster's FoundationDB backup
strategy driver. The driver delegates to the FoundationDB Kubernetes
operator: each Cozystack BackupJob materialises an
apps.foundationdb.org/v1beta2 FoundationDBBackup CR (a continuous
backup_agent Deployment that streams data to S3), and each RestoreJob
materialises a FoundationDBRestore CR (a one-shot fdbrestore against
the destination cluster).
Because the operator only permits one running backup directory per
cluster, the driver flips any prior FoundationDBBackup to
backupState=Stopped before starting a new one. Each Cozystack
BackupJob therefore owns a discrete blob-store directory keyed by its
name.
Step order
| File | Role | Triggered by |
|---|---|---|
00-helpers.sh |
Shared bash helpers and env defaults; sourced by every step. | n/a |
01-create-strategy.sh |
Creates the cluster-scoped FoundationDB strategy. |
admin |
02-create-bucket.sh |
Provisions a Bucket, mints <app>-fdb-backup-creds Secrets, and creates the BackupClass bound to the FoundationDB strategy with the resolved S3 coordinates. Caches raw creds in .bucket-info.env (chmod 600). cleanup.sh removes it. |
tenant |
03-create-foundationdb-src.sh |
Provisions the source FoundationDB and writes a sentinel key/value. | tenant |
04-create-backupjob.sh |
Submits a BackupJob and waits for the first snapshot. |
tenant |
05-restore-in-place.sh |
Clears the sentinel and restores into the same cluster via RestoreJob. |
tenant |
06-restore-to-copy.sh |
Provisions a second FoundationDB and restores into it via RestoreJob.spec.targetApplicationRef. |
tenant |
cleanup.sh |
Removes everything created by the demo. | admin or tenant |
run-all.sh |
Convenience runner that executes 01..06 in order. | demo |
90-scenario-admin-prepare.md |
Narrative for the admin preparation steps. | docs |
91-scenario-user-backup.md |
Narrative for the user backup flow. | docs |
92-scenario-user-restore.md |
Narrative for the user restore flow (in-place and to-copy). | docs |
The
BackupClassis created in step 02 with already-resolved bucket coordinates rather than a placeholder pre-applied by an earlier admin step. An earlier draft of this demo split that across two steps and wroteaccountName: "REPLACE_ME"to the cluster first, which passed the operator'sMinLength=1validation but failed at backup_agent runtime — a tenant who skipped the patch step burned the 45-minute backup deadline on a half-configured class. The current step 02 fold avoids that trap; the BackupClass only ever exists with real coordinates.
Environment variables (overrides)
All variables come from 00-helpers.sh:
| Variable | Default | Description |
|---|---|---|
NAMESPACE |
tenant-root |
Tenant namespace for the demo. |
FDB_NAME |
fdb-src |
Source FoundationDB application name. |
FDB_RESTORE_NAME |
fdb-dst |
Target FoundationDB for the to-copy restore. |
BUCKET_NAME |
foundationdb-backups |
Cozystack Bucket to provision. |
BACKUPCLASS_NAME |
foundationdb-default |
BackupClass name (cluster-scoped). |
STRATEGY_NAME |
foundationdb-strategy-default |
Strategy name (cluster-scoped). |
BACKUPJOB_NAME |
foundationdb-backup-job |
BackupJob name. |
RESTOREJOB_INPLACE_NAME |
foundationdb-restore-inplace |
In-place RestoreJob name. |
RESTOREJOB_TOCOPY_NAME |
foundationdb-restore-to-copy |
To-copy RestoreJob name. |
Prerequisites
- Cozystack cluster with
backup-controllerandbackupstrategy-controllerrunning, with the FoundationDB dispatch wired (seeinternal/backupcontroller/foundationdbstrategy_controller.go). kubectl,jq.- The
apps.foundationdb.orgCRDs installed bypackages/system/foundationdb-operator(FoundationDBCluster,FoundationDBBackup,FoundationDBRestore). - The Cozystack
foundationdb-rdApplicationDefinition installed soapps.cozystack.io/FoundationDBrenders an HR.
Notes
- The chart-level
backup.*values block is markedDEPRECATED. New tenants should use this BackupClass + FoundationDB strategy flow instead. Existing tenants withbackup.enabled=truecontinue to render the legacyFoundationDBBackupCR unchanged. - The strategy template references a per-app blob credentials Secret named
{{ .Application.metadata.name }}-fdb-backup-creds. Step 02 materialises one Secret per app instance (source and to-copy target). - The to-copy variant is the stronger restore proof: it witnesses the sentinel key landing on an independent cluster.