Commit Graph
15 Commits
Author SHA1 Message Date
Alexey Artamonov dae76d186c Merge branch 'main' into fix/clickhouse-idle-before-backup
main pins the archive name a backup Pod keeps across a wait for
late bucket credentials, against the controller's archive helper.
That helper now names a run's archive with part of the BackupJob's
UID, so the merged test has to give its BackupJob one.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-12 17:20:19 +03:00
Alexey Artamonov 5a95bcdd64 fix(clickhouse): keep a run's archive name within the old name's length
The archive name is part of every object key, a key is capped at 1024
bytes, and the legacy s3PathOverride prefix has no length limit. The
full UID made the name 37 characters longer than before, so a prefix
that fit the old name pushed the manifest key past the cap and failed
the backup.

The run's name now drops "clickhouse-" and the "T" and "Z" of the
timestamp and spends the room on the first 12 hex digits of the UID,
so it is exactly as long as the old name for every application.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-12 11:10:43 +03:00
Alexey Artamonov 2046649b4c fix(clickhouse): match a backup Job's archive by its whole name
The archive a backup Job carries was found as a substring of its Pod
spec. The name without the BackupJob's UID starts the name with it,
and a strategy may append to either, so a Job that wrote a longer
archive could get the shorter name recorded on its Backup, and a
restore of that Backup would read another archive.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-09 17:53:43 +03:00
Alexey Artamonov fc15ed7171 fix(clickhouse): keep the archive name of a backup running across the upgrade
A backup Job created before archive names carried the BackupJob's UID
writes its archive under the name without it, and the Job is not
re-rendered when the controller is upgraded. Matching only the new
name left that Backup with no archive recorded, so its restore fell
back to guessing by timestamp.

The name without the UID is matched second, because it is a prefix of
the one with it.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-09 16:42:13 +03:00
Alexey Artamonov 1291e616c7 fix(clickhouse): give every backup run an archive of its own
An archive was named from the application and the BackupJob's start
time, which has one-second resolution. Two runs of one application
started in the same second, as two Plans on the same schedule are,
wrote the same archive. The strategy takes a finished upload of its
archive as an earlier attempt of its own run, so the later run could
succeed on the other run's upload and record an archive missing any
table created since.

The BackupJob's UID now ends the name. It goes after the timestamp so
names keep sorting chronologically, and the restore of a Backup that
records no archive, which accepts only a bare timestamp after the
release prefix, never picks one of these.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-09 16:39:47 +03:00
Alexey Artamonov 86c541368d Merge branch 'main' into fix/clickhouse-backup-late-credentials
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-06 17:56:59 +03:00
Myasnikov Daniil d7ae34dfc3 fix(backups): fail a ClickHouse BackupJob whose Backup name is taken
A Backup outlives the BackupJob that made it, so a BackupJob created
again under a name already used meets the earlier run's Backup when its
own Job completes. That happens by hand, and from a Plan reconciled
again inside its starting deadline, since a Plan names each BackupJob
after its schedule slot. The Altinity driver took the earlier Backup as
its result: the BackupJob turned Succeeded with a BackupRef recording
the earlier run's archive, the archive this run wrote was recorded on
nothing, and restoring what the BackupJob reported brought back the
older data.

An existing Backup is now taken as this run's only when it records this
run's archive, which is what an earlier reconcile of the same run
leaves when its status write is lost. Any other fails the BackupJob,
naming both archives, and leaves the earlier Backup untouched.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-11-05 07:56:44 +00:00
Alexey Artamonov 6345b939d2 fix(clickhouse): stop reporting a credentials wait once the strategy job runs
The hold sets Ready=False/WaitingForCredentials with a countdown, and
nothing removed it once the Job was created, so a backup or restore
that ran for minutes kept saying it was still waiting. The condition is
dropped when the Job runs; a Ready condition set for another reason is
left alone.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-04 19:43:15 +03:00
Alexey Artamonov 5f636ce624 fix(clickhouse): do not hold strategy jobs on a credentials stamp that cannot expire
The projector rewrites the credentials-updated-at stamp only when the
projected content changes, so a stamp the hold cannot use stays put.
An unreadable one failed every reconcile with no condition on the job,
and one far in the future held each job again on every reconcile; in
both cases every ClickHouse backup and restore in the namespace stopped
for good. Such a stamp is now logged and ignored. A stamp only slightly
ahead, as a skewed clock writes it, still waits, and that wait ends.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-04 19:43:14 +03:00
Alexey Artamonov f5f50d2a77 fix(clickhouse): hold a strategy job until projected credentials reach the sidecar
The clickhouse-backup sidecar now reads the projected credentials from a
mounted Secret, which kubelet rewrites only after its sync period,
about 75 seconds after the change in practice. The first BackupJob
projects the Secret and started the strategy Job a second later, so its
attempts raced the volume update and the backup could still fail on a
sidecar without credentials.

The projector records when the projected content last changed, and the
ClickHouse driver holds back creating a backup or restore Job until two
minutes have passed since, with a WaitingForCredentials condition. An
unchanged re-projection keeps the time, so only the first projection
and a rotation cause a wait, and a Job already running is never held.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-11-04 17:15:10 +03:00
Myasnikov Daniil 47b497194e fix(backups): restore the ClickHouse archive a Backup was taken as
The Altinity strategy named each archive inside its Pod with date -u
and recorded nothing on the Backup, so a restore could only guess: it
listed the remote archives under the application's prefix and took the
newest. A RestoreJob of an older Backup therefore restored whichever
backup had run last.

The controller now names the archive before it creates the backup Job,
from the BackupJob's persisted StartedAt, so every reconcile and every
retried Pod of the Job writes the same one. The name is recorded in the
Backup's driverMetadata next to the strategy parameters, and a restore
renders it into the strategy Pod, which restores exactly that archive
and fails, naming it, when remote storage no longer holds it. The name
is recorded only when the Job's Pod was given it, so a Job created by
an earlier controller, or a strategy that names its own archives,
leaves the Backup without one instead of pointing it at an archive
that does not exist.

Because a retried Pod reuses the name, it waits on an attempt the
sidecar is still running instead of being refused with 423, and asks
for resume, so a local or partial remote backup an earlier attempt left
is completed rather than refused as existing.

Backups taken before names were recorded stay restorable: the Pod picks
the newest archive whose name is stamped no later than the Backup's
takenAt, which is set only after the backup Job finished, and logs a
warning that the choice is a best effort.

Assisted-by: LLM
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
2026-11-04 11:10:42 +00:00
Aleksei Sviridkin 7a0aab2bdc refactor: apply the Go 1.27 modernizers
Run go fix over every first-party module: interface{} becomes any,
hand-written loops become slices, maps, min/max and range-over-int,
errors.As becomes errors.AsType, pointer helpers become new(expr) and
are inlined at their call sites, and so on. Every rewrite keeps
behaviour.

Three rewrites stay out. embedlit flattens embedded-field literals
such as ObjectMeta, syntax only Go 1.27 accepts, which would break
backports to release branches still on Go 1.26. omitzero drops
omitempty from struct fields, and openapi-gen reads omitempty as
optional, so metadata, spec and status would turn required in the
published schemas. And errorsastype rewrote an errors.As whose
target, apierrors.APIStatus, does not implement error, which
errors.AsType rejects at compile time.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
2026-09-24 17:23:18 +03:00
Andrey Kolkov f17ead3d26 feat(platform): add default backupclass
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-05-28 13:49:59 +04:00
Andrey Kolkov 84874353ce fix(platform): align field of Altinity backup-strategy for similarity with others
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-05-13 17:15:51 +04:00
Andrey Kolkov 250e1b3376 feat(platform): implement backup strategy for clickhouse application
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
2026-05-11 14:46:48 +04:00