The restore existence check piped the live topic list into `grep -qxF`.
grep exits at the first match, so once the matched topic is far enough from
the end of a list larger than the 64 KiB pipe buffer, the producing printf
takes SIGPIPE, pipefail turns the pipeline into a failure, the `if` reads
false, and a topic that exists is sent to --create, which fails with
TopicExistsException. Read the list from a here-string instead: no pipe, no
SIGPIPE, decided by content rather than list size.
Also make the local metadata scratch file overridable (METADATA_FILE),
like BIN already is, so the bats suite can point each test at its own path
instead of sharing /tmp/kafka-metadata.txt across runs. The Job never sets
it and keeps the default.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
The script opens with `set -euo pipefail` and no shebang, so shellcheck
reports SC2148 and cannot tell which dialect to lint against. Declare it
with a `# shellcheck shell=bash` directive as the first line: shellcheck
and pre-commit now lint it as bash, and the line is an inert comment once
the script is embedded into the Job's pod argument.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
The restore path validated only the shape parsed from the live cluster,
not the partition count and replication factor read from the stored
object. A non-numeric recorded partition count made `[ "${lp}" -lt
"${parts}" ]` and its `elif` exit 2; an `if` condition is exempt from
errexit, so both read as false, the topic was skipped, and the loop still
printed its per-topic success line - so the RestoreJob reached Succeeded
without ever touching the topic. A well-formed but wrong-content object,
which curl -f cannot catch, must fail loudly like the unrecognised-record
guard.
Validate the recorded parts and rf are numeric before the comparisons and
--create, with a bats case that reddens when the guard is dropped.
Also correct a stale to-copy example comment: the strategy reads the exact
object the source backup recorded in status.artifact.uri (injected as
ARTIFACT_URI), not a key derived from .Backup.ApplicationRef.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
- delete_topic waited with `grep -qx` (BRE), so deleting `audit.events`
matched the live `audit-events` and spun to its timeout — the e2e
failed at step 25 on every run that reached it. Use `grep -qxF`.
- The two -k/--connect-to guards were written as `! grep`, which cannot
fail under hack/cozytest.sh (it wraps each @test in `set -e` + a
trailing `return 0`, and bash exempts a `!`-negated command from
errexit). Rewrite them as positive `if grep ...; then return 1; fi`;
verified the -k reinstatement now reddens the suite under cozytest.
- Cover the restore-side literal-topic guard that was unpinned at every
level: a bats case runs the existing-topic branch with a colliding
sibling and asserts the `--alter` argv carries `\Q…\E`, and run-all.sh
now leaves `audit.events` live so the e2e exercises that branch instead
of only `--create`. Add bats cases for the partition-shrink refusal, the
`-f` upload guard, and `--exclude-internal`.
- Bound a running Job: reconcileKafka / reconcileKafkaRestore now fail the
BackupJob/RestoreJob once kafkaDefaultBackupDeadline elapses in the
Job-running branch (buildJobStrategyBatchJob sets no
activeDeadlineSeconds), so an unschedulable pod no longer pins it in
Running forever. Pinned by TestReconcileKafka_FailsWhenJobNeverCompletes.
- Fail closed on an unknown record kind and an unknown MODE, and validate
the live partition/RF parsed on restore (an empty value silently skipped
reconciliation). Correct the stale "image pinned in the strategy" comment
in the example helpers and the chainsaw test (the controller now resolves
the target broker's image as .ClientImage).
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
The port workaround built --connect-to's HOST1 from ${host}, but the
virtual-hosted branch (forcePathStyle:false) sends to ${bucket}.${host}.
curl matches HOST1 against the request host, so on a virtual-hosted AND
non-default-port endpoint the redirect never fired and the request hit the
wrong port. Derive HOST1 from the host the URL actually uses; the real
connection target stays ${host}:${port}. Path-style is unaffected. Add a
bats case for the virtual-hosted ported mode, which the suite did not cover.
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM
- Regression fix: an earlier change dropped the curl --connect-to port workaround along
with -k, but curl 7.76.1 (the image's) 403s on ANY ported endpoint,
which is the shape backupStorage.endpoint ships (:8333) and what
provisionBucket:false and s3.ingress.enabled:false produce. Restore
--connect-to (sign a portless URL, redirect to the real port), keeping
-k off so TLS is still verified. IPv6-safe host/port split, path prefix
preserved. The port-less ingress passed; the ported case is the one that regressed.
- Correct the S3 envelope claim in values.yaml and backup-classes.md: the
403 is decided by the port, not a body-hash the backend recomputes, and
--connect-to handles it on SeaweedFS and MinIO alike.
- Hold the script fixes at the layer the contract lives at. Externalise
the driver to files/kafka-backup.sh (embedded verbatim via .Files.Get,
render byte-identical) and add hack/kafka-backup-script.bats, which
EXECUTES it against stubbed kafka CLIs + curl and asserts behaviour: \Q
literal match on the colliding pair, --connect-to with no -k on a ported
endpoint, fail-closed --list, comma-value bracket grouping, RF-divergence
abort, and the cleanup delete's HTTP handling. Not a template grep.
- Pin the Backup-delete dispatch and the readiness deadline in Go:
TestCleanupOnDelete_Kafka_RoutesToKafkaCleanup and
TestReconcileKafka_FailsAfterReadinessDeadline.
- Docs: config restore is additive (no --delete-config); export time
scales with topic count and the Job has no wall clock (as its siblings).
Signed-off-by: Andrey Kolkov <androndo@gmail.com>
Assisted-by: LLM