The Redis backup demo now takes APP_KIND=Valkey, which switches the app
kind and the chart prefix the helpers derive sentinel and Secret names
from. The demo BackupClass routes Valkey to the same demo strategy.
The valkey suite gains valkey-backup-roundtrip, which drives that demo
end to end: seed a marker, back it up, restore it in place and into a
copy. It writes to tenant-root's seaweedfs through seaweedfs-s3, so a
seaweedfs change now selects the valkey suite too, and since the
round-trip runs examples/backups/redis as its harness, an edit there
selects both the redis and the valkey suites. The reduced E2E install
keeps the backup stack for the valkey suite, as it does for the other
backup round-trip suites.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
Valkey runs on the same spotahome RedisFailover operator as Redis and
speaks the same replication protocol and RDB format, so it needs no
strategy kind of its own. The only difference the backup Job sees is the
chart's resource-name prefix: valkey-<app> instead of redis-<app> for the
sentinel Service and the auth Secret.
The Redis driver now accepts applicationRef kind Valkey as well as Redis
and hands the template .Release.Prefix, derived from the app's kind, which
cozy-default-redis uses for every name it builds. A kind with no known
prefix fails the render instead of producing an empty one. cozy-default
routes apps.cozystack.io/Valkey to cozy-default-redis. A restore must
target an app of the same kind as the backup's source: replaying a Redis
dump into Valkey, or the reverse, is not something the strategy can
promise across engine releases.
The shipped template reads the prefix as `.Release.Prefix | default
"redis-"`: during an upgrade the previous controller, which has no Prefix
in its render context and serves only Redis, renders the new CR, and a
bare reference would print "<no value>" into the names.
The redis:8 image already loads RDB v11, which both shipped Valkey
versions (7.2, 8.1) write.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
The backup round-trips drive examples/backups, whose bucket waits gave
up at 300s and whose postgres waits at 360s, under the 600s install
timeout cozystack-api gives each of those releases. The kafka example's
bucket wait passed no budget and took the helper's 300s default. A
failed install there reported only that the wait ran out.
Raise each to 660s, and grow the Chainsaw op around every walkthrough
by what its waits gained, so a failure at any later wait keeps the
headroom it had and is still reported by the script rather than cut
off by the op. The kafka and kafka-metadata Kafka waits stay: their
7h install timeout cannot be cleared inside the E2E job cap.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Every backup walkthrough carried its own wait_for_field, and most their
own wait_hr_ready and wait_deleted: twelve, eight and two copies, which
had already drifted into five, three and two bodies. A fix made in one
copy did not reach the others, and a diagnostics change carried by hand
between two of them once went into the wrong function and was caught
only by reading the diff.
The helpers now live in examples/backups/_lib/wait-helpers.sh, sourced
by each walkthrough's 00-helpers.sh. Where copies differed, the shared
body keeps the diagnostics any of them had: wait_for_field leaves
kubectl's stderr attached, and wait_hr_ready prints the conditions and
history dump on Stalled or timeout. Call sites and budgets are unchanged.
A new unit test fails when any walkthrough defines one of these helpers
again, which replaces the byte-identity pin between copies.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The redis backup demo read cozy-default-redis once and gave up if it was
absent. That strategy is rendered behind a lookup of the platform
bucket, so it only exists after a Helm upgrade that runs once the
bucket is provisioned. The backupstrategy-controller forces that
upgrade, but helm-controller will not run it while a release in the
chart's dependsOn is not Ready.
A platform change shortly before the suite starts is enough to hold it
back. The e2e install patches the platform package late (exposed
services, then OIDC), the operator rewrites the cozystack-basics
release, which then waits on its own dependencies, and
backupstrategy-controller waits on cozystack-basics. On one run the
strategy was still missing eight minutes after those patches, and the
demo's single read failed the suite as if the backups stack were not
installed.
Wait for the object with a bound instead. 15 minutes covers the nine
minutes seen between those patches and the strategy's creation on
another run, and still leaves the demo inside its 30 minute step.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The redis backup demo read and wrote its marker key from a throwaway
`kubectl run -i --rm` Pod. kubectl returns that Pod's stdout over an
attach, the container runtime delivers only what the container writes
after the attach is registered, and kubectl falls back to the Pod log
only when the attach fails. A redis-cli that finishes in between comes
back as an empty reply with exit 0, which is how a GET of a key the
master held read as '' and failed the round-trip on PRs that do not
touch redis.
Run redis-cli by `kubectl exec` in a sentinel Pod instead, whose image
already carries it. An exec'd process writes into pipes created with
it, so nothing it prints is lost, and its exit status comes back too.
The demo's reads also stop discarding stderr. The same empty string used
to stand for a missing object, an API error and a lost stream alike, so
a red run could not say which one it was.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Store the Redis strategy Template schemaless, as the Rabbitmq strategy
already does: without the markers controller-gen emitted the full
PodTemplateSpec schema and the CRD reached ~602 KB, pushing the chart's
Helm-release Secret past the 1 MiB cap so backupstrategy-controller failed
to install at all.
Give the example strategy a cleanup branch, and both scripts an explicit
`elif restore` / `else exit 1`, so a deleted Backup rendered with
Mode=cleanup deletes its object instead of falling into the restore path
and FLUSHALLing the live master.
Forward a backupStrategyController block through the platform bundle
alongside backupStorage, so redisBackupResources — the restore-Pod sizing
knob the loader-not-ready message points at — is reachable via
spec.components.platform.values.backupStrategyController, not merely
settable on a Helm-managed Strategy CR that reverts on the next upgrade.
Correct the endpointCASecretName doc row to match the now-optional mount,
and `int` the activeDeadlineSeconds interpolation so a large values-file
override renders as an integer, not exponent form.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Wait for the throwaway loader's AOF rewrite to finish - no rewrite in
flight, last one ok, base AOF on disk - instead of a bare
aof_rewrite_in_progress:0 check, which right after `appendonly yes` can
fall through before the scheduled rewrite starts and replay an absent or
half-written base AOF. Same fix in the example strategy.
Declare the package image targets PHONY and serialize image-redis-backup
after image-backupstrategy-controller so their `yq -i values.yaml` digest
stamps cannot race under `make -j`. Run the demo cleanup under `set -e`
(every delete is already --ignore-not-found) so a real API failure aborts
instead of printing "Cleanup complete" over a half-torn-down demo.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Delete the backup object when its Backup is removed. The Redis driver
owns its S3 object (one per BackupJob, no engine retention prunes it), so
it now spawns a cleanup-mode Job that deletes the object and holds the
Backup's finalizer until it completes, mirroring the Rabbitmq driver.
Without this a retention-pruned Plan grew the bucket unboundedly and the
key was lost with the CR.
Size the strategy Pod from backupStrategyController.redisBackupResources
so an admin can raise the restore ceiling (the throwaway loader holds the
whole dataset in memory) via chart values, which a direct edit of the
cluster-scoped, Helm-managed Strategy CR could not survive. The
loader-not-ready diagnostic now names the memory limit, the likeliest
cause, instead of only the image version.
Make the endpoint CA volume optional: nothing projects that Secret into
the app namespace, so a required mount wedged every backup Pod on
FailedMount; a missing ca.crt now falls through to the system CA store
and fails the handshake fast, which is not the same as skipping verify.
Promote the Pod-level activeDeadlineSeconds to the Job so a wedged run is
bounded once, not (backoffLimit+1) times. Use find, not ls, for the base
AOF so its no-match diagnostic is not dead code under pipefail.
Pin the object key reaching the rendered Job: both fixtures now render
--key={{ .ObjectKey }} and assert it, so zeroing the key (which left the
package green) is caught. Document .ObjectKey and the cleanup mode in the
CRD render-context, list cozy-default-redis among the Helm-managed CRs,
and mirror the CA/TLS and find fixes into the example strategy.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
The demo strategy CR states its in-Pod script is identical to the shipped
cozy-default-redis strategy, but it kept the plaintext Sentinel and master
dial after the shipped template gained server-side TLS. Add the same
optional redis-<app>.ca-cert mount and RTLS auto-detection so a tenant who
copies this example to back up a tls.enabled Redis does not reproduce the
plaintext-against-a-TLS-only-listener failure.
Assisted-By: LLM
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
The branch was cut before main gained the redis-tls and
redis-tls-flip-rejected Chainsaw Tests, so its copy of the redis
chainsaw-test.yaml carried only the plaintext Test plus the new
redis-2-backup-roundtrip. Carry main's two TLS Tests here too, so a conflict
resolution at merge time cannot drop the TLS end-to-end coverage regardless
of which side it takes. The file still needs a rebase onto current main to
merge cleanly; this only removes the risk of losing the Tests.
Also pass the terminal Failed value to the BackupJob/RestoreJob phase waits
so a failed run aborts immediately with a legible cause instead of polling to
the 600s timeout.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
The Redis driver shipped no round-trip backup/restore e2e, unlike its
ClickHouse, MariaDB and RabbitMQ siblings, so the backup to S3 and restore
data path had no automated coverage. The existing demo backed up through the
platform cozy-default BackupClass, whose endpoint is the shared system
bucket; in the e2e environment that bucket's advertised external address is
not routable, so a cozy-default round-trip cannot run there.
Rework examples/backups/redis to provision its OWN Bucket + Redis strategy +
BackupClass (mirroring the MariaDB and RabbitMQ demos), projecting the
bucket's keys and the seaweedfs CA into the Secrets the strategy references
and honouring an S3_ENDPOINT override so the backup Pod targets the
in-cluster seaweedfs. Add a redis-2-backup-roundtrip Chainsaw test that
drives run-all.sh in tenant-root and asserts the sentinel round-trips
through object storage in place and into a copy.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
The spotahome RedisFailover operator ships no backup CRD, so add a
dedicated strategy.backups.cozystack.io/Redis driver that runs a one-shot
Job: it dumps an RDB snapshot from the current master (discovered via the
operator's sentinel Service) to the system bucket over S3, and on restore
replays it (RDB -> RESP via redis-cli --pipe) into the target's master.
Structurally a specialisation of the Job/Altinity script-Job pattern with
a Redis applicationRef gate and an app-readiness precondition. Backup
parameters round-trip through the Backup's DriverMetadata (no
underlyingResources snapshot); restore drives from .Backup.ApplicationRef
so a to-copy restore reads the source's object-storage prefix while
writing into a differently-named target. Both in-place and to-copy are
supported; PITR is not (RDB is a discrete snapshot).
- api: Redis/RedisList strategy type + generated CRD and deepcopy
- driver: reconcileRedis / reconcileRedisRestore + pure helpers, wired
into both dispatch switches, supportedBackupStrategyKinds and the no-op
restore cleanup case
- default: gated cozy-default-redis strategy (system-bucket coords via
cozy-backups-creds env secretKeyRef, https:// scheme + optional CA) and
the Redis route in the cozy-default BackupClass
- image: images/redis-backup bundling redis-cli + curl + an RDB->RESP
converter, wired via backupStrategyController.redisBackupClientImage
- tests: driver unit tests (gate, precondition, render, param round-trip,
both reconcile paths) and the helm rendering-gate coverage
- example: examples/backups/redis honest marker round-trip for both
restore flows
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-By: LLM
Andrey Kolkov