From af0db99c711ee19c105061d6df47299ed98d20e8 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 13 Oct 2026 05:52:57 +0300 Subject: [PATCH 1/9] feat(platform): move Strimzi resources to the v1 API before Strimzi 1.x Strimzi 1.0 removed v1beta2 from its CRDs, and the API server refuses a CRD that drops a version its status still lists as stored. Every stored object has to be rewritten as v1 before the new operator arrives, which needs the CRDs of Strimzi 0.49 to 0.51 that serve both versions and Strimzi's own conversion tool. The 0.45 operator is stopped first: Strimzi supports the conversion only under an operator that knows v1, and 0.45 keeps rewriting the Kafka status through v1beta2. The conversion Job's ClusterRole can patch CRDs and write ConfigMaps in every namespace, so it goes with its binding and ServiceAccount once the Job has finished, successfully or not; a failed Job stays for diagnosis, and a Job still running when the wait times out keeps them, because the retry waits for that Job. Two references to v1beta2 outlive the stored objects. Helm builds the objects of a release before it upgrades or uninstalls it, so every stored revision of every Kafka application is rewritten to v1. And the API server refuses server-side apply to an object whose managedFields still name a version the CRD no longer serves; when helm-controller recovers by wiping them, a field a later chart drops is never removed. Those entries move to v1 as well. A Helm action already running stores the releases it read when it finishes, v1beta2 included, so the version is not stamped while any Kafka release revision is still pending. Strimzi 1.x also runs neither ZooKeeper nor Kafka 3.x. A ZooKeeper cluster, or a shared broker pool still being drained, left by a platform that moved past v1.7 before its ZooKeeper to KRaft migration finished cannot be finished afterwards, so the migration stops instead; Kafka applications on v3.8 or v3.9 move to v4.3. The migration also stops, before it changes anything, while a Kafka application topic or a KafkaTopic sets a config Kafka 4 removed or a segment.bytes below the 1 MiB minimum of Kafka 4: the new kafka chart refuses to render such a topic, and a Kafka 4 broker whose log holds the small segment.bytes does not start. segment.bytes is read the way Kafka reads it: trimmed, with an optional sign, leading zeros decimal. Strimzi's conversion tool refuses some v1beta2 objects outright, and convert-resource stops at the first one, with the operators already stopped. While v1beta2 is served the migration lists every Strimzi kind first and refuses, naming each object and the reason, before it stops anything. The list follows the 0.51.0 tool the conversion Job runs, and a test fails when that image changes. The conversion deletes the Kafka's KRaft metadata state, and KRaft is the state Strimzi never leaves. Before it stops the operators, the migration records an annotation on every Kafka it accepts, so a retry after the conversion still accepts a NotReady Kafka that passed on its KRaft state. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- hack/migration-61-strimzi-v1-conversion.bats | 1013 +++++++++++++++++ .../testdata/migration-61-strimzi-v1/apps.txt | 4 + .../kafka-releases.txt | 2 + .../migration-61-strimzi-v1/kafkas-kraft.json | 4 + .../migration-61-strimzi-v1/kafkas-zk.json | 5 + hack/testdata/migration-61-strimzi-v1/kubectl | 342 ++++++ .../managed-kafkas.json | 54 + .../release-kafka-one.json | 1 + .../strimzi-crds-0.51.0.yaml | 1 + .../platform/images/migrations/Dockerfile | 8 + .../platform/images/migrations/migrations/61 | 661 +++++++++++ packages/core/platform/values.yaml | 2 +- 12 files changed, 2096 insertions(+), 1 deletion(-) create mode 100644 hack/migration-61-strimzi-v1-conversion.bats create mode 100644 hack/testdata/migration-61-strimzi-v1/apps.txt create mode 100644 hack/testdata/migration-61-strimzi-v1/kafka-releases.txt create mode 100644 hack/testdata/migration-61-strimzi-v1/kafkas-kraft.json create mode 100644 hack/testdata/migration-61-strimzi-v1/kafkas-zk.json create mode 100755 hack/testdata/migration-61-strimzi-v1/kubectl create mode 100644 hack/testdata/migration-61-strimzi-v1/managed-kafkas.json create mode 100644 hack/testdata/migration-61-strimzi-v1/release-kafka-one.json create mode 100644 hack/testdata/migration-61-strimzi-v1/strimzi-crds-0.51.0.yaml create mode 100755 packages/core/platform/images/migrations/migrations/61 diff --git a/hack/migration-61-strimzi-v1-conversion.bats b/hack/migration-61-strimzi-v1-conversion.bats new file mode 100644 index 000000000..6b82dd5c7 --- /dev/null +++ b/hack/migration-61-strimzi-v1-conversion.bats @@ -0,0 +1,1013 @@ +#!/usr/bin/env bats +# ----------------------------------------------------------------------------- +# Unit tests for platform migration 61 --> 62 (move the Strimzi custom +# resources to the v1 API, refuse ZooKeeper-based Kafkas, move Kafka 3.x +# applications to v4.3). See +# packages/core/platform/images/migrations/migrations/61 for the mechanism. +# +# The fake kubectl keeps the storedVersions every Strimzi CRD reports in a +# state file, and the conversion Job rewrites it when it succeeds, so the +# check after the Job reads what the Job did. +# +# The harness (docker, jq baked onto the migrations base image, the explicit +# `return` in run_migration) is the one hack/migration-55-fluxcd-orphan.bats +# describes; see there for why each piece is load-bearing. +# +# Run with: hack/cozytest.sh hack/migration-61-strimzi-v1-conversion.bats +# ----------------------------------------------------------------------------- + +load test_helper + +FIXTURES="$PWD/hack/testdata/migration-61-strimzi-v1" +FAKE_FIXTURES="$FIXTURES" +MIG_DIR="$PWD/packages/core/platform/images/migrations/migrations" +ALPINE=$(sed -n 's/^FROM \(alpine:[^ ]*\).*$/\1/p' \ + "$PWD/packages/core/platform/images/migrations/Dockerfile" | head -1) +TESTIMG="cozystack-migration61-test:$(printf '%s' "$ALPINE" | sed 's/[^a-zA-Z0-9]/-/g')" +WORKROOT="${TMPDIR:-/tmp}/cozy-migration-61-$$" + +cozy_cleanup() { + rm -rf "$WORKROOT" + return 0 +} + +# The wait for the conversion Job counts only its sleeps, so a zero poll +# interval would let a wait that misses the end of the Job spin for good. A +# one-second poll under a three-second timeout turns that into a quick failure. +run_migration() { + _run_migration_rc=0 + docker run --rm --network none \ + --user "$(id -u):$(id -g)" \ + -v "$MIG_DIR:/migrations:ro" \ + -v "$FIXTURES:/fakebin:ro" \ + -v "$WORK:/work" \ + -e PATH=/fakebin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \ + -e FAKE_CMDLOG=/work/cmdlog \ + -e FAKE_STATE=/work/state \ + -e FAKE_APPS=/fakebin/apps.txt \ + -e FAKE_RELEASES=/fakebin/kafka-releases.txt \ + -e FAKE_KAFKAS="${FAKE_KAFKAS_OVERRIDE:-/fakebin/${FAKE_KAFKAS_FILE:-kafkas-kraft.json}}" \ + -e FAKE_KAFKA_LIST_FAIL="${FAKE_KAFKA_LIST_FAIL-}" \ + -e FAKE_CONVERSION_CHANGES_SPEC="${FAKE_CONVERSION_CHANGES_SPEC-}" \ + -e FAKE_NO_STRIMZI="${FAKE_NO_STRIMZI-}" \ + -e FAKE_JOB="${FAKE_JOB-}" \ + -e FAKE_PATCH_FAIL="${FAKE_PATCH_FAIL-}" \ + -e FAKE_MANAGED_CONFLICT="${FAKE_MANAGED_CONFLICT-}" \ + -e FAKE_REBALANCES="${FAKE_REBALANCES-}" \ + -e FAKE_CRD_READ_FAIL="${FAKE_CRD_READ_FAIL-}" \ + -e FAKE_APIRES_FAIL="${FAKE_APIRES_FAIL-}" \ + -e FAKE_APPS_ABSENT="${FAKE_APPS_ABSENT-}" \ + -e FAKE_DISCOVERY_PARTIAL="${FAKE_DISCOVERY_PARTIAL-}" \ + -e FAKE_SECRET_LIST_FAIL="${FAKE_SECRET_LIST_FAIL-}" \ + -e FAKE_RELEASE_REWRITTEN_BACK="${FAKE_RELEASE_REWRITTEN_BACK-}" \ + -e FAKE_RELEASE_PENDING="${FAKE_RELEASE_PENDING-}" \ + -e FAKE_OLD_JOB="${FAKE_OLD_JOB-}" \ + -e FAKE_REBALANCE_FAIL="${FAKE_REBALANCE_FAIL-}" \ + -e FAKE_OPERATOR_READ_FAIL="${FAKE_OPERATOR_READ_FAIL-}" \ + -e FAKE_SERVED="${FAKE_SERVED-}" \ + -e FAKE_MANAGED_PATCH_FAIL="${FAKE_MANAGED_PATCH_FAIL-}" \ + -e FAKE_PULL="${FAKE_PULL-}" \ + -e FAKE_HELMRELEASES="${FAKE_HELMRELEASES-}" \ + -e FAKE_KAFKATOPICS="${FAKE_KAFKATOPICS-}" \ + -e FAKE_STRIMZI_OBJECTS="${FAKE_STRIMZI_OBJECTS-}" \ + -e FAKE_STRIMZI_LIST_FAIL="${FAKE_STRIMZI_LIST_FAIL-}" \ + -e STRIMZI_CRD_FILE="${STRIMZI_CRD_FILE-/fakebin/strimzi-crds-0.51.0.yaml}" \ + -e POLL_INTERVAL=1 \ + -e CONVERSION_TIMEOUT="${CONVERSION_TIMEOUT-3}" \ + "${MIG_IMAGE:-$TESTIMG}" "/migrations/$1" || _run_migration_rc=$? + return "$_run_migration_rc" +} + +prep() { + docker info >/dev/null 2>&1 || { + echo "docker is required: these tests run migration 61 inside a jq-enabled" >&2 + echo "build of $ALPINE, the migrations image's base." >&2 + return 1 + } + docker build -q -t "$TESTIMG" - >/dev/null < "$WORK/state/stored" + FAKE_CMDLOG="$WORK/cmdlog" + : > "$FAKE_CMDLOG" + unset FAKE_CONVERSION_CHANGES_SPEC FAKE_KAFKAS_OVERRIDE FAKE_KAFKA_LIST_FAIL FAKE_NO_STRIMZI FAKE_JOB FAKE_PATCH_FAIL FAKE_KAFKAS_FILE FAKE_MANAGED_CONFLICT FAKE_MANAGED_PATCH_FAIL FAKE_REBALANCES STRIMZI_CRD_FILE FAKE_CRD_READ_FAIL FAKE_APIRES_FAIL FAKE_SECRET_LIST_FAIL FAKE_RELEASE_REWRITTEN_BACK FAKE_OLD_JOB FAKE_REBALANCE_FAIL FAKE_OPERATOR_READ_FAIL FAKE_SERVED FAKE_PULL FAKE_HELMRELEASES FAKE_KAFKATOPICS FAKE_STRIMZI_OBJECTS FAKE_STRIMZI_LIST_FAIL FAKE_APPS_ABSENT FAKE_DISCOVERY_PARTIAL CONVERSION_TIMEOUT || true + return 0 +} + +release_json() { + base64 -d < "$WORK/state/release-$1" | base64 -d | gzip -dc +} + +@test "converts to v1, moves Kafka 3.x applications to v4.3 and stamps 62" { + prep + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + + grep -qxF "APPLY-CRDS /fakebin/strimzi-crds-0.51.0.yaml" "$FAKE_CMDLOG" + # convert-resource is the init container, so crd-upgrade cannot start first. + grep -qxF "APPLY-CONVERSION convert-resource crd-upgrade" "$FAKE_CMDLOG" + crds_line=$(grep -nxF "APPLY-CRDS /fakebin/strimzi-crds-0.51.0.yaml" "$FAKE_CMDLOG" | cut -d: -f1) + job_line=$(grep -nxF "APPLY-CONVERSION convert-resource crd-upgrade" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$crds_line" -lt "$job_line" ] + # The 0.45 operator is stopped before anything is converted. + scale_line=$(grep -nxF "SCALE-OPERATOR 0" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$scale_line" -lt "$crds_line" ] + # The entity operators only after the cluster operator, or it scales them back. + entity_line=$(grep -nxF "SCALE-ENTITY tenant-a/kafka-one-entity-operator" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$scale_line" -lt "$entity_line" ] + # A terminating topic or user operator still writes status through v1beta2. + wait_line=$(grep -nxF "WAIT-ENTITY tenant-a/strimzi.io/cluster=kafka-one,strimzi.io/name=kafka-one-entity-operator" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$entity_line" -lt "$wait_line" ] + [ "$wait_line" -lt "$crds_line" ] + + grep -qxF 'PATCH tenant-a/one {"spec":{"version":"v4.3"}}' "$FAKE_CMDLOG" + grep -qxF 'PATCH tenant-b/two {"spec":{"version":"v4.3"}}' "$FAKE_CMDLOG" + if grep -qE "^PATCH tenant-(c|d)/" "$FAKE_CMDLOG"; then echo "patched an application that needs no version change" >&2; return 1; fi + + grep -qxF "DELETE clusterrole strimzi-v1-api-conversion" "$FAKE_CMDLOG" + [ "$(tail -1 "$FAKE_CMDLOG")" = "STAMP 62" ] + rm -rf "$WORK" +} + +# The README documents a manual run outside the image. On Ubuntu /bin/sh is a +# dash without pipefail. +@test "runs on Ubuntu, where /bin/sh is dash" { + prep + docker build -q -t cozystack-migration61-test:ubuntu - >/dev/null <<'DOCKERFILE' +FROM ubuntu:24.04@sha256:2198e4c65076e9cb0b3d695219df667c0e3b0072da64ca3c2648aca17dcecaf5 +RUN apt-get update && apt-get install -y --no-install-recommends jq && rm -rf /var/lib/apt/lists/* +DOCKERFILE + rc=0 + MIG_IMAGE=cozystack-migration61-test:ubuntu run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + [ "$(tail -1 "$FAKE_CMDLOG")" = "STAMP 62" ] +} + +# Helm builds the objects of the deployed release before it upgrades or +# uninstalls it, and fails on an API version the server no longer serves. +@test "rewrites the deployed Helm release of each Kafka to the v1 API" { + prep + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + + grep -qxF "PATCH-RELEASE tenant-a/sh.helm.release.v1.kafka-one.v3" "$FAKE_CMDLOG" + grep -qxF "PATCH-RELEASE tenant-a/sh.helm.release.v1.kafka-one.v4" "$FAKE_CMDLOG" + # An application whose install failed has a release and no Kafka CR. + grep -qxF "PATCH-RELEASE tenant-f/sh.helm.release.v1.kafka-failed-install.v1" "$FAKE_CMDLOG" + # The applications leave Kafka 3.x before any release is rewritten. + version_line=$(grep -nF 'PATCH tenant-a/one ' "$FAKE_CMDLOG" | cut -d: -f1) + first_release_line=$(grep -nF 'PATCH-RELEASE ' "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + [ "$version_line" -lt "$first_release_line" ] + release_json v4 > "$WORK/release.json" + [ "$(jq -r '.manifest' "$WORK/release.json" | grep -c '^apiVersion: kafka.strimzi.io/v1$')" -eq 2 ] + if jq -r '.manifest' "$WORK/release.json" | grep -qF 'kafka.strimzi.io/v1beta2'; then echo "v1beta2 left in the release manifest" >&2; return 1; fi + # Only the manifest is rewritten; everything else in the release stays as it was. + jq 'del(.manifest)' "$FAKE_FIXTURES/release-kafka-one.json" > "$WORK/want" + jq 'del(.manifest)' "$WORK/release.json" > "$WORK/got" + cmp "$WORK/want" "$WORK/got" + release_line=$(grep -nxF "PATCH-RELEASE tenant-a/sh.helm.release.v1.kafka-one.v3" "$FAKE_CMDLOG" | cut -d: -f1) + stamp_line=$(grep -nxF "STAMP 62" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$release_line" -lt "$stamp_line" ] + + : > "$FAKE_CMDLOG" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + [ "$rc" -eq 0 ] + if grep -qF "PATCH-RELEASE" "$FAKE_CMDLOG"; then echo "a second run rewrote the release again" >&2; return 1; fi + rm -rf "$WORK" +} + +# Once the CRDs serve only v1, the API server refuses a server-side apply to +# an object whose managedFields still name v1beta2. +@test "moves managedFields entries recorded under old API versions to v1" { + prep + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "PATCH-MANAGED tenant-a/kafka-one rv=100 helm-controller=kafka.strimzi.io/v1,strimzi-cluster-operator=kafka.strimzi.io/v1" "$FAKE_CMDLOG" + # The status entry keeps its subresource and its fields. + [ "$(jq -r '.items[0].metadata.managedFields[1].subresource' "$WORK/state/managed")" = "status" ] + if grep -qF "PATCH-MANAGED tenant-b/" "$FAKE_CMDLOG"; then echo "patched an object already on v1" >&2; return 1; fi + + : > "$FAKE_CMDLOG" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + [ "$rc" -eq 0 ] + if grep -qF "PATCH-MANAGED" "$FAKE_CMDLOG"; then echo "a second run patched managedFields again" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a managedFields patch that loses a race is rebuilt from a fresh read" { + prep + export FAKE_MANAGED_CONFLICT=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "PATCH-MANAGED tenant-a/kafka-one rv=100 helm-controller=kafka.strimzi.io/v1,strimzi-cluster-operator=kafka.strimzi.io/v1" "$FAKE_CMDLOG" + grep -qxF "PATCH-MANAGED tenant-a/kafka-one rv=101 helm-controller=kafka.strimzi.io/v1,strimzi-cluster-operator=kafka.strimzi.io/v1" "$FAKE_CMDLOG" + [ "$(jq -r '.items[0].metadata.managedFields[0].apiVersion' "$WORK/state/managed")" = "kafka.strimzi.io/v1" ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a managedFields patch that fails for another reason aborts before stamping" { + prep + export FAKE_MANAGED_PATCH_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + [ "$(grep -c "PATCH-MANAGED" "$FAKE_CMDLOG")" -eq 1 ] + if grep -qF "STAMP" "$FAKE_CMDLOG"; then echo "stamped after a failed managedFields patch" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a cluster without Strimzi only stamps the version" { + prep + export FAKE_NO_STRIMZI=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + if grep -qE "APPLY-CRDS|APPLY-CONVERSION|PATCH" "$FAKE_CMDLOG"; then echo "touched Strimzi on a cluster without it" >&2; return 1; fi + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a ZooKeeper-based or still-migrating Kafka stops the migration before it changes anything" { + prep + export FAKE_KAFKAS_FILE=kafkas-zk.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " tenant-c/kafka-zk: ZooKeeper section in spec" "$WORK/out" + grep -qxF " tenant-d/kafka-mid: metadata state KRaftPostMigration" "$WORK/out" + if grep -qF "tenant-a/kafka-one" "$WORK/out"; then echo "named a KRaft cluster as ZooKeeper-based" >&2; return 1; fi + grep -qF "Cozystack v1.7" "$WORK/out" + if grep -qE "APPLY-CRDS|APPLY-CONVERSION|PATCH|STAMP|SCALE-OPERATOR" "$FAKE_CMDLOG"; then echo "changed the cluster despite a ZooKeeper Kafka" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a drain of the v1.7 ZooKeeper migration still in flight stops the migration" { + prep + export FAKE_REBALANCES="tenant-a/kafka-one-drain-kafka,tenant-a/my-own-rebalance" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " tenant-a/kafka-one-drain-kafka" "$WORK/out" + if grep -qxF " tenant-a/my-own-rebalance" "$WORK/out"; then echo "named an unrelated KafkaRebalance" >&2; return 1; fi + if grep -qE "SCALE-OPERATOR|APPLY-CRDS|STAMP" "$FAKE_CMDLOG"; then echo "changed the cluster despite an unfinished drain" >&2; return 1; fi + rm -rf "$WORK" +} + +# Run by hand outside the migrations image, the CRD file is easy to miss; +# the operators must not be stopped for a run that cannot go on. +@test "a missing CRD file aborts before the operators are stopped" { + prep + export STRIMZI_CRD_FILE=/nonexistent/strimzi-crds-0.51.0.yaml + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "/nonexistent/strimzi-crds-0.51.0.yaml is not readable" "$WORK/out" + if grep -qE "SCALE-|APPLY-CRDS|STAMP" "$FAKE_CMDLOG"; then echo "changed the cluster without the CRD file" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed conversion Job aborts with its logs, before any application changes" { + prep + export FAKE_JOB=fail + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "fake conversion log" "$WORK/out" + # The failure ends the wait at once, not at the timeout. + [ "$(grep -cF "get job strimzi-v1-api-conversion -o jsonpath={.status.succeeded}" "$FAKE_CMDLOG")" -eq 1 ] + if grep -qE "PATCH|STAMP" "$FAKE_CMDLOG"; then echo "went on after the conversion failed" >&2; return 1; fi + # The failed Job stays for diagnosis; the rights it ran with do not. + grep -qxF "DELETE clusterrolebinding strimzi-v1-api-conversion" "$FAKE_CMDLOG" + grep -qxF "DELETE clusterrole strimzi-v1-api-conversion" "$FAKE_CMDLOG" + grep -qxF "DELETE serviceaccount strimzi-v1-api-conversion" "$FAKE_CMDLOG" + [ "$(sed -n '/^APPLY-CONVERSION/,$p' "$FAKE_CMDLOG" | grep -cF "DELETE job strimzi-v1-api-conversion")" -eq 0 ] + rm -rf "$WORK" +} + +@test "a Job that leaves a non-v1 stored version behind aborts" { + prep + export FAKE_JOB=noop + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF 'kafkas.kafka.strimzi.io: ["v1beta2"]' "$WORK/out" + if grep -qE "PATCH|STAMP" "$FAKE_CMDLOG"; then echo "went on with v1beta2 still stored" >&2; return 1; fi + grep -qxF "DELETE clusterrolebinding strimzi-v1-api-conversion" "$FAKE_CMDLOG" + grep -qxF "DELETE clusterrole strimzi-v1-api-conversion" "$FAKE_CMDLOG" + grep -qxF "DELETE serviceaccount strimzi-v1-api-conversion" "$FAKE_CMDLOG" + [ "$(sed -n '/^APPLY-CONVERSION/,$p' "$FAKE_CMDLOG" | grep -cF "DELETE job strimzi-v1-api-conversion")" -eq 0 ] + rm -rf "$WORK" +} + +@test "a re-run after the conversion skips it and still moves the versions" { + prep + printf '["v1"]' > "$WORK/state/stored" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + if grep -qE "APPLY-CRDS|APPLY-CONVERSION" "$FAKE_CMDLOG"; then echo "converted again" >&2; return 1; fi + # The CRDs still serve v1beta2, so the operator may be 0.45: it is stopped. + grep -qxF "SCALE-OPERATOR 0" "$FAKE_CMDLOG" + # Nothing slow follows on this path, so the managedFields rewrite has to + # wait for the entity operators to be gone. + wait_line=$(grep -nF "WAIT-ENTITY tenant-a/" "$FAKE_CMDLOG" | cut -d: -f1) + managed_line=$(grep -nF "PATCH-MANAGED " "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + [ "$wait_line" -lt "$managed_line" ] + grep -qxF 'PATCH tenant-a/one {"spec":{"version":"v4.3"}}' "$FAKE_CMDLOG" + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a failing application patch aborts before stamping" { + prep + export FAKE_PATCH_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qF "STAMP" "$FAKE_CMDLOG"; then echo "stamped after a failed patch" >&2; return 1; fi + rm -rf "$WORK" +} + +# The migration runs once: a read that fails must stop it before the stamp, +# never pass for "nothing there". +@test "a failed read of the CRD stored versions aborts before stamping" { + prep + export FAKE_CRD_READ_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qE "STAMP|SCALE-|APPLY-CRDS|PATCH" "$FAKE_CMDLOG"; then echo "went on after a failed CRD read" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed read of the apps.cozystack.io API aborts before stamping" { + prep + export FAKE_APIRES_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "reading the apps.cozystack.io API failed" "$WORK/out" + if grep -qE "STAMP|PATCH-RELEASE" "$FAKE_CMDLOG"; then echo "went on after a failed apps.cozystack.io read" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "another API group failing discovery does not stop the migration" { + prep + export FAKE_DISCOVERY_PARTIAL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF 'PATCH tenant-a/one {"spec":{"version":"v4.3"}}' "$FAKE_CMDLOG" + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a cluster without the apps.cozystack.io API skips the applications and stamps" { + prep + export FAKE_APPS_ABSENT=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + if grep -qF "kafkas.apps.cozystack.io" "$FAKE_CMDLOG"; then echo "read Kafka applications without their API" >&2; return 1; fi + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a failed Helm release listing aborts before stamping" { + prep + export FAKE_SECRET_LIST_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qF "STAMP" "$FAKE_CMDLOG"; then echo "stamped after a failed release listing" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a release that names v1beta2 again after the rewrite stops the stamp" { + prep + export FAKE_RELEASE_REWRITTEN_BACK=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " Helm release tenant-a/sh.helm.release.v1.kafka-one.v3" "$WORK/out" + if grep -qF "STAMP" "$FAKE_CMDLOG"; then echo "stamped with a v1beta2 release left" >&2; return 1; fi + rm -rf "$WORK" +} + +# A Helm action that started before the rewrite stores the releases it read +# when it finishes, so a revision still pending blocks the stamp as well. +@test "a Helm action still running on a Kafka release stops the stamp" { + prep + export FAKE_RELEASE_PENDING=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " Helm release tenant-a/sh.helm.release.v1.kafka-one.v4 is pending-upgrade" "$WORK/out" + if grep -qF "STAMP" "$FAKE_CMDLOG"; then echo "stamped while a Helm action was running" >&2; return 1; fi + rm -rf "$WORK" +} + +# A Job failed on its deadline can still have a pod terminating, and +# convert-resource writes until that pod is gone. +@test "a finished Job of an earlier attempt is removed with its pods before the CRDs are touched" { + for state in failed succeeded; do + prep + export FAKE_OLD_JOB=$state + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + delete_line=$(grep -nxF "DELETE job strimzi-v1-api-conversion foreground" "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + scale_line=$(grep -nxF "SCALE-OPERATOR 0" "$FAKE_CMDLOG" | cut -d: -f1) + crds_line=$(grep -nF "APPLY-CRDS" "$FAKE_CMDLOG" | cut -d: -f1) + job_line=$(grep -nF "APPLY-CONVERSION" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$delete_line" -lt "$scale_line" ] + [ "$delete_line" -lt "$crds_line" ] + [ "$crds_line" -lt "$job_line" ] + rm -rf "$WORK" + done +} + +@test "a Job of an earlier attempt that fails while waited for is removed with its pods and started over" { + prep + export FAKE_OLD_JOB=running FAKE_JOB=fail + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + # The new Job fails as well, so the run stops after starting it. + [ "$rc" -ne 0 ] + # Each failure ends its wait at once, not at the timeout. + [ "$(grep -cF "get job strimzi-v1-api-conversion -o jsonpath={.status.succeeded}" "$FAKE_CMDLOG")" -eq 2 ] + delete_line=$(grep -nxF "DELETE job strimzi-v1-api-conversion foreground" "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + crds_line=$(grep -nF "APPLY-CRDS" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$delete_line" -lt "$crds_line" ] + grep -qF "APPLY-CONVERSION" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +# Without a deadline, a Job whose pod never starts never fails, and every +# retry waits on it and exits. The deadline must cover the wait of the run +# that started the Job plus the wait of the retry after it. +@test "the conversion Job carries a deadline of at least two waits for it" { + prep + export CONVERSION_TIMEOUT=100 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + deadline=$(sed -n 's/^CONVERSION-DEADLINE //p' "$FAKE_CMDLOG") + [ -n "$deadline" ] + [ "$deadline" -ge 200 ] + rm -rf "$WORK" +} + +# A pod the API server refused never fails, so the Job reports its deadline +# only in its Failed condition. +@test "a Job of an earlier attempt failed on its deadline without a pod is started over" { + prep + export FAKE_OLD_JOB=running FAKE_JOB=nopod CONVERSION_TIMEOUT=0 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qF "is still running; retry once it finishes" "$WORK/out"; then echo "took a failed Job for a running one" >&2; return 1; fi + delete_line=$(grep -nxF "DELETE job strimzi-v1-api-conversion foreground" "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + crds_line=$(grep -nF "APPLY-CRDS" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$delete_line" -lt "$crds_line" ] + grep -qF "APPLY-CONVERSION" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a conversion pod that cannot pull its image is named, with what to do" { + prep + export FAKE_JOB=hang FAKE_PULL=1 CONVERSION_TIMEOUT=0 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "cannot pull quay.io/strimzi/operator:0.51.0@sha256:" "$WORK/out" + grep -qF "mirror" "$WORK/out" + if grep -qE "PATCH|STAMP" "$FAKE_CMDLOG"; then echo "went on without a conversion" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a conversion Job still running when the wait times out keeps its rights for the retry" { + prep + export FAKE_JOB=hang CONVERSION_TIMEOUT=0 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if sed -n '/^APPLY-CONVERSION/,$p' "$FAKE_CMDLOG" | grep -qF "DELETE"; then echo "removed what the running Job needs" >&2; return 1; fi + rm -rf "$WORK" +} + +# A retry after the hook timed out must not touch the CRDs under a Job that +# is still converting. +@test "a Job of an earlier attempt still running is waited for before the CRDs are touched" { + prep + export FAKE_OLD_JOB=running + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qF "still running" "$WORK/out" + # It converted everything, so there is nothing left to apply or start. + if grep -qE "APPLY-CRDS|APPLY-CONVERSION" "$FAKE_CMDLOG"; then echo "touched the CRDs under a running Job" >&2; return 1; fi + grep -qxF "DELETE clusterrole strimzi-v1-api-conversion" "$FAKE_CMDLOG" + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "a Job of an earlier attempt still running after the wait aborts before the CRDs are touched" { + prep + export FAKE_OLD_JOB=running FAKE_JOB=hang CONVERSION_TIMEOUT=0 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "is still running; retry once it finishes" "$WORK/out" + if grep -qE "SCALE-|APPLY-CRDS|DELETE|STAMP" "$FAKE_CMDLOG"; then echo "went on under a running Job" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed read of the KafkaRebalances aborts before the operators are stopped" { + prep + export FAKE_REBALANCE_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qE "SCALE-|APPLY-CRDS|STAMP" "$FAKE_CMDLOG"; then echo "went on after a failed KafkaRebalance read" >&2; return 1; fi + rm -rf "$WORK" +} + +# Converting under a 0.45 operator the migration failed to see would let it +# record v1beta2 managedFields again after the rewrite. +@test "a failed read of the cluster operator Deployment aborts before anything is converted" { + prep + export FAKE_OPERATOR_READ_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qE "APPLY-CRDS|APPLY-CONVERSION|STAMP" "$FAKE_CMDLOG"; then echo "converted without knowing whether the 0.45 operator runs" >&2; return 1; fi + rm -rf "$WORK" +} + +# Run by hand after the kafka-operator release brought Strimzi 1.x: nothing +# would scale its operator back up. +@test "a re-run once the CRDs serve only v1 leaves the operators running" { + prep + printf '["v1"]' > "$WORK/state/stored" + export FAKE_SERVED=v1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + if grep -qF "SCALE-" "$FAKE_CMDLOG"; then echo "stopped the 1.x operator" >&2; return 1; fi + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} + +@test "the KRaft gate rejects missing and transitional state on both storage paths" { + for stored in '["v1beta2"]' '["v1"]'; do + for state in '""' '"PreKRaft"' '"KRaftMigration"' '"KRaftDualWriting"' '"KRaftPostMigration"' '"ZooKeeper"' false; do + prep + printf '%s' "$stored" > "$WORK/state/stored" + jq --argjson state "$state" '.items[0].status.kafkaMetadataState = $state | .items[1].status = {}' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "tenant-a/kafka-one" "$WORK/out" + grep -qF "tenant-b/kafka-two" "$WORK/out" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE" "$FAKE_CMDLOG"; then echo "mutated before proving KRaft" >&2; return 1; fi + done + done +} + +@test "a healthy modern Kafka without legacy state passes on both served APIs" { + for served in 'v1 v1beta2' v1; do + prep + export FAKE_SERVED="$served" + [ "$served" != v1 ] || printf '["v1"]' > "$WORK/state/stored" + broker_version=4.1.1 + [ "$served" != v1 ] || broker_version=4.3.1 + jq --arg broker "$broker_version" '.items |= map(.metadata.generation = 7 | .spec.kafka.version = $broker | .status = {kafkaVersion: $broker, kafkaMetadataVersion: "4.1-IV1", observedGeneration: 7, conditions: [{type: "Ready", status: "True"}]})' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + if [ "$served" = v1 ]; then + if grep -qE "SCALE-|APPLY-CRDS|v1beta2.kafka.strimzi.io" "$FAKE_CMDLOG"; then echo "touched legacy API on a v1-only rerun" >&2; return 1; fi + fi + done +} + +@test "a NotReady or malformed modern status cannot prove KRaft" { + for change in 'del(.status)' '.status = null' '.status.kafkaMetadataVersion = ""' '.status.kafkaMetadataVersion = false' '.status.kafkaMetadataVersion = 41' '.status.kafkaMetadataVersion = []' '.status.conditions = [{type:"NotReady",status:"True"}]' '.status.conditions = [{type:"Ready",status:"False"}]' '.status.kafkaMetadataState = "KRaftPostMigration"' '.spec.zookeeper = {replicas:3}'; do + prep + jq '.items |= map(.metadata.generation = 7 | .status = {kafkaMetadataVersion:"4.1-IV1",observedGeneration:7,conditions:[{type:"Ready",status:"True"}]})' "$FIXTURES/kafkas-kraft.json" | jq ".items[0] |= ($change)" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "tenant-a/kafka-one" "$WORK/out" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE" "$FAKE_CMDLOG"; then echo "accepted unproven modern status" >&2; return 1; fi + done +} + +@test "the KRaft gate reports every failing Kafka and its Helm release" { + prep + jq '.items |= map(.status = {} | .metadata.annotations["meta.helm.sh/release-name"] = ("release-" + .metadata.name))' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "tenant-a/kafka-one" "$WORK/out" + grep -qF "tenant-a/release-kafka-one" "$WORK/out" + grep -qF "tenant-b/kafka-two" "$WORK/out" + grep -qF "tenant-b/release-kafka-two" "$WORK/out" +} + +@test "a failed or malformed Kafka list fails closed before mutation" { + for response in failed empty invalid null '{}' '{"items":null}' '{"items":{}}' '{"items":[null]}' '{"items":[{}]}' '{"items":[]} {"items":[]}'; do + prep + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + case "$response" in + failed) export FAKE_KAFKA_LIST_FAIL=1; printf '{"items":[]}' > "$WORK/kafkas.json" ;; + empty) : > "$WORK/kafkas.json" ;; + *) printf '%s' "$response" > "$WORK/kafkas.json" ;; + esac + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE" "$FAKE_CMDLOG"; then echo "accepted a failed or malformed list" >&2; return 1; fi + done +} + +@test "an empty Kafka list passes the KRaft gate" { + prep + printf '{"items":[]}' > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" +} + +@test "a previous conversion Job is fenced before the KRaft gate rejects an unproven Kafka" { + for state in running failed succeeded; do + prep + export FAKE_OLD_JOB="$state" + jq '.items[0].status = {}' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "tenant-a/kafka-one" "$WORK/out" + if [ "$state" = running ]; then + wait_line=$(grep -nF 'get job strimzi-v1-api-conversion -o jsonpath={.status.succeeded}' "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + gate_line=$(grep -nF 'get kafkas.v1beta2.kafka.strimzi.io' "$FAKE_CMDLOG" | head -1 | cut -d: -f1) + [ "$wait_line" -lt "$gate_line" ] + fi + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE" "$FAKE_CMDLOG"; then echo "mutated after an unproven Kafka" >&2; return 1; fi + done +} + +@test "an unfinished drain blocks an already-v1 rerun" { + prep + printf '["v1"]' > "$WORK/state/stored" + export FAKE_SERVED=v1 FAKE_REBALANCES=tenant-a/kafka-one-drain-kafka + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "tenant-a/kafka-one-drain-kafka" "$WORK/out" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE" "$FAKE_CMDLOG"; then echo "mutated despite a pending drain" >&2; return 1; fi +} + +@test "a completed conversion with a newer spec generation does not block recovery of a proven KRaft cluster" { + prep + jq '.items |= map( + .metadata.generation = 7 + | .spec.kafka.version = "4.1.1" + | .spec.kafka.listeners = [{name:"plain",port:9092,type:"internal",tls:false}] + | .spec.kafka.replicas = 3 + | .spec.entityOperator.topicOperator.reconciliationIntervalSeconds = 90 + | .status = {kafkaMetadataVersion:"4.1-IV1",observedGeneration:7,conditions:[{type:"Ready",status:"True"}]})' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json FAKE_OLD_JOB=running FAKE_CONVERSION_CHANGES_SPEC=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + jq --exit-status 'all(.items[]; .metadata.generation == 8 and .status.observedGeneration == 7 and .spec.kafka.replicas == null and .spec.entityOperator.topicOperator.reconciliationIntervalSeconds == null and .spec.entityOperator.topicOperator.reconciliationIntervalMs == 90000)' "$WORK/state/kafkas-converted.json" + jq '.items | map(.status)' "$WORK/kafkas.json" > "$WORK/status-before.json" + jq '.items | map(.status)' "$WORK/state/kafkas-converted.json" > "$WORK/status-after.json" + cmp "$WORK/status-before.json" "$WORK/status-after.json" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + if grep -qE "APPLY-CRDS|APPLY-CONVERSION" "$FAKE_CMDLOG"; then echo "restarted an already completed conversion" >&2; return 1; fi +} + +# The converter deletes status.kafkaMetadataState, so anything the gate +# accepts before the conversion must still pass on a retry after it. +@test "an explicit KRaft state without a metadata version does not pass the gate" { + prep + jq '.items |= map(.status = {kafkaMetadataState:"KRaft",kafkaMetadataVersion:"3.9-IV0",conditions:[{type:"Ready",status:"True"}]}) | .items[0] |= del(.status.kafkaMetadataVersion)' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " tenant-a/kafka-one: no kafkaMetadataVersion" "$WORK/out" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "changed the cluster for an unproven Kafka" >&2; return 1; fi +} + +# KRaft is the final Strimzi metadata state, so it needs no Ready. The +# converter deletes the state, so the proof is recorded on the Kafka before the +# operators stop, and a retry after the conversion accepts it from there. +@test "a NotReady Kafka in the KRaft state passes, before and after the converter drops its state" { + prep + jq '.items |= map(.status = {kafkaMetadataState:"KRaft",kafkaMetadataVersion:"3.9-IV0",conditions:[{type:"Ready",status:"True"}]}) | .items[0].status.conditions = [{type:"NotReady",status:"True"}]' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json FAKE_CONVERSION_CHANGES_SPEC=1 FAKE_PATCH_FAIL=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + annotate_line=$(grep -nxF "ANNOTATE tenant-a/kafka-one cozystack.io/kraft-verified=3.9-IV0" "$FAKE_CMDLOG" | cut -d: -f1) + grep -qxF "ANNOTATE tenant-b/kafka-two cozystack.io/kraft-verified=3.9-IV0" "$FAKE_CMDLOG" + scale_line=$(grep -nxF "SCALE-OPERATOR 0" "$FAKE_CMDLOG" | cut -d: -f1) + [ "$annotate_line" -lt "$scale_line" ] + jq --exit-status '.items[0] | .status.kafkaMetadataState == null and .metadata.annotations["cozystack.io/kraft-verified"] == "3.9-IV0"' "$WORK/state/kafkas-converted.json" + + unset FAKE_PATCH_FAIL + : > "$FAKE_CMDLOG" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + if grep -qF "ANNOTATE" "$FAKE_CMDLOG"; then echo "annotated an already recorded Kafka again" >&2; return 1; fi +} + +@test "a Kafka accepted before the conversion is still accepted after the converter drops its state" { + prep + jq '.items |= map(.status = {kafkaMetadataState:"KRaft",kafkaMetadataVersion:"3.9-IV0",conditions:[{type:"Ready",status:"True"}]})' "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json FAKE_OLD_JOB=running FAKE_CONVERSION_CHANGES_SPEC=1 + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + jq --exit-status 'all(.items[]; .status.kafkaMetadataState == null)' "$WORK/state/kafkas-converted.json" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" +} + +@test "a Kafka application topic with a config Kafka 4 removed stops the migration before it changes anything" { + prep + cat > "$WORK/helmreleases.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"kafka-one","labels":{"apps.cozystack.io/application.kind":"Kafka","apps.cozystack.io/application.name":"one"}}, + "spec":{"values":{"topics":[ + {"name":"legacy","config":{"message.format.version":"3.0","message.timestamp.difference.max.ms":"1000","retention.ms":"1"}}, + {"name":"clean","config":{"retention.ms":"1"}}, + {"name":"bare"}]}}}, + {"metadata":{"namespace":"tenant-b","name":"kafka-two","labels":{"apps.cozystack.io/application.kind":"Kafka"}}, + "spec":{"values":{"topics":[{"name":"old","config":{"message.downconversion.enable":true}}]}}}, + {"metadata":{"namespace":"tenant-c","name":"kafka-three","labels":{"apps.cozystack.io/application.kind":"Kafka"}}, + "spec":{}} +]} +JSON + export FAKE_HELMRELEASES=/work/helmreleases.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF -- "-l apps.cozystack.io/application.kind=Kafka" "$FAKE_CMDLOG" + grep -qxF " tenant-a/one: topic legacy sets message.format.version" "$WORK/out" + grep -qxF " tenant-a/one: topic legacy sets message.timestamp.difference.max.ms" "$WORK/out" + grep -qxF " tenant-b/kafka-two: topic old sets message.downconversion.enable" "$WORK/out" + if grep -qE "clean|bare|retention" "$WORK/out"; then echo "named a topic without a removed config" >&2; return 1; fi + grep -qF "before running this migration" "$WORK/out" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "changed the cluster despite a removed topic config" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a topic below the Kafka 4 segment.bytes floor stops the migration before it changes anything" { + prep + cat > "$WORK/helmreleases.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"kafka-one","labels":{"apps.cozystack.io/application.kind":"Kafka","apps.cozystack.io/application.name":"one"}}, + "spec":{"values":{"topics":[ + {"name":"small","config":{"segment.bytes":524288}}, + {"name":"padded","config":{"segment.bytes":" 524288"}}, + {"name":"plus","config":{"segment.bytes":"+524288"}}, + {"name":"zeros","config":{"segment.bytes":"02000000"}}, + {"name":"exp","config":{"segment.bytes":"1e3"}}, + {"name":"floor","config":{"segment.bytes":"1048576"}}]}}} +]} +JSON + cat > "$WORK/kafkatopics.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-b","name":"by-hand"},"spec":{"topicName":"raw.events","config":{"segment.bytes":"65536"}}}, + {"metadata":{"namespace":"tenant-b","name":"named-by-cr"},"spec":{"config":{"segment.bytes":100}}}, + {"metadata":{"namespace":"tenant-b","name":"fine"},"spec":{"config":{"segment.bytes":1073741824}}}, + {"metadata":{"namespace":"tenant-b","name":"bare"},"spec":{}}, + {"metadata":{"namespace":"tenant-b","name":"padded-cr"},"spec":{"config":{"segment.bytes":"\t+100 "}}}, + {"metadata":{"namespace":"tenant-b","name":"negative-cr"},"spec":{"config":{"segment.bytes":"-2000000"}}} +]} +JSON + export FAKE_HELMRELEASES=/work/helmreleases.json FAKE_KAFKATOPICS=/work/kafkatopics.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qxF " tenant-a/one: topic small sets segment.bytes=524288, below the Kafka 4 minimum of 1048576" "$WORK/out" + grep -qxF " tenant-b/kafkatopic by-hand: topic raw.events sets segment.bytes=65536, below the Kafka 4 minimum of 1048576" "$WORK/out" + grep -qxF " tenant-b/kafkatopic named-by-cr: topic named-by-cr sets segment.bytes=100, below the Kafka 4 minimum of 1048576" "$WORK/out" + # Kafka trims the value and reads it with Integer.parseInt, which takes a + # sign and reads leading zeros as decimal. + grep -qxF " tenant-a/one: topic padded sets segment.bytes=524288, below the Kafka 4 minimum of 1048576" "$WORK/out" + grep -qxF " tenant-a/one: topic plus sets segment.bytes=524288, below the Kafka 4 minimum of 1048576" "$WORK/out" + grep -qxF " tenant-b/kafkatopic padded-cr: topic padded-cr sets segment.bytes=100, below the Kafka 4 minimum of 1048576" "$WORK/out" + grep -qxF " tenant-b/kafkatopic negative-cr: topic negative-cr sets segment.bytes=-2000000, below the Kafka 4 minimum of 1048576" "$WORK/out" + if grep -qE "floor|fine|bare|zeros|exp" "$WORK/out"; then echo "named a topic at or above the floor" >&2; return 1; fi + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "changed the cluster despite a topic below the floor" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed read of the KafkaTopics aborts before the operators are stopped" { + prep + export FAKE_KAFKATOPICS=/work/missing.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "get kafkatopics.v1beta2.kafka.strimzi.io" "$FAKE_CMDLOG" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "went on after a failed KafkaTopic read" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed read of the Kafka application HelmReleases aborts before the operators are stopped" { + prep + export FAKE_HELMRELEASES=/work/missing.json + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "helmreleases.helm.toolkit.fluxcd.io" "$FAKE_CMDLOG" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "went on after a failed HelmRelease read" >&2; return 1; fi + rm -rf "$WORK" +} + +# Strimzi's conversion tool refuses these objects, and convert-resource stops +# at the first one, after the operators are stopped. +refused_objects() { + jq '.items[0].spec.kafka += {resources: {requests: {cpu: "1"}}, authorization: {type: "keycloak"}, listeners: [{name: "oauth", authentication: {type: "oauth"}}, {name: "custom", authentication: {type: "custom", secrets: [{secretName: "s", key: "k"}]}}]} + | .items[1].spec.kafka += {authorization: {type: "opa"}, listeners: [{name: "custom", authentication: {type: "custom", sasl: true}}, {name: "tls", authentication: {type: "tls"}}]} + | .items += [.items[1] | .metadata = {namespace: "tenant-c", name: "kafka-clean"} | .spec.kafka.authorization = {type: "simple"}]' \ + "$FIXTURES/kafkas-kraft.json" > "$WORK/kafkas.json" + mkdir -p "$WORK/objects" + cat > "$WORK/objects/kafkaconnects.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"connect-bad"},"spec":{"authentication":{"type":"oauth"},"externalConfiguration":{"env":[]},"build":{"output":{"additionalKanikoOptions":["--a"],"additionalBuildOptions":["--b"]}}}}, + {"metadata":{"namespace":"tenant-a","name":"connect-clean"},"spec":{"authentication":{"type":"tls"},"build":{"output":{"additionalKanikoOptions":["--a"]}}}}, + {"metadata":{"namespace":"tenant-a","name":"connect-build-clean"},"spec":{"build":{"output":{"additionalBuildOptions":["--b"]}}}} +]} +JSON + cat > "$WORK/objects/kafkabridges.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"bridge-bad"},"spec":{"authentication":{"type":"oauth"}}}, + {"metadata":{"namespace":"tenant-a","name":"bridge-clean"},"spec":{"authentication":{"type":"scram-sha-512"}}} +]} +JSON + cat > "$WORK/objects/kafkamirrormaker2s.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"mm2-old-bad"},"spec":{"externalConfiguration":{"env":[]},"connectCluster":"missing","clusters":[{"alias":"src","authentication":{"type":"oauth"}},{"alias":"dst"}],"mirrors":[{"sourceCluster":"nope","targetCluster":"other"}]}}, + {"metadata":{"namespace":"tenant-a","name":"mm2-no-clusters"},"spec":{"mirrors":[{}]}}, + {"metadata":{"namespace":"tenant-a","name":"mm2-new-oauth"},"spec":{"target":{"alias":"dst","authentication":{"type":"oauth"}},"mirrors":[{"source":{"alias":"src","authentication":{"type":"oauth"}}}]}}, + {"metadata":{"namespace":"tenant-a","name":"mm2-old-clean"},"spec":{"connectCluster":"dst","clusters":[{"alias":"src"},{"alias":"dst"}],"mirrors":[{"sourceCluster":"src","targetCluster":"dst"}]}}, + {"metadata":{"namespace":"tenant-a","name":"mm2-new-clean"},"spec":{"target":{"alias":"dst"},"mirrors":[{"source":{"alias":"src"},"targetCluster":"dst"}]}} +]} +JSON + cat > "$WORK/objects/kafkausers.json" <<'JSON' +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"user-no-spec"}}, + {"metadata":{"namespace":"tenant-a","name":"user-clean"},"spec":{"authentication":{"type":"tls"}}} +]} +JSON + cat > "$WORK/objects/kafkanodepools.json" <<'JSON' +{"items":[{"metadata":{"namespace":"tenant-a","name":"pool-null-spec"},"spec":null}]} +JSON + export FAKE_KAFKAS_OVERRIDE=/work/kafkas.json FAKE_STRIMZI_OBJECTS=/work/objects +} + +@test "an object the Strimzi v1 conversion refuses stops the migration before the operators are stopped" { + prep + refused_objects + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + while IFS= read -r want; do + grep -qxF " $want" "$WORK/out" || { echo "not reported: $want" >&2; return 1; } + done <<'LINES' +kafkas.kafka.strimzi.io tenant-a/kafka-one: .spec.kafka.resources (use a KafkaNodePool) +kafkas.kafka.strimzi.io tenant-a/kafka-one: OAuth authentication on a listener +kafkas.kafka.strimzi.io tenant-a/kafka-one: secrets in a custom listener authentication +kafkas.kafka.strimzi.io tenant-a/kafka-one: Keycloak authorization +kafkas.kafka.strimzi.io tenant-b/kafka-two: Open Policy Agent authorization +kafkaconnects.kafka.strimzi.io tenant-a/connect-bad: OAuth client authentication +kafkaconnects.kafka.strimzi.io tenant-a/connect-bad: .spec.externalConfiguration +kafkaconnects.kafka.strimzi.io tenant-a/connect-bad: both .spec.build.output.additionalKanikoOptions and additionalBuildOptions +kafkabridges.kafka.strimzi.io tenant-a/bridge-bad: OAuth client authentication +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-old-bad: .spec.externalConfiguration +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-old-bad: OAuth client authentication +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-old-bad: .spec.connectCluster is not an alias in .spec.clusters +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-old-bad: a mirror whose targetCluster is not the target cluster +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-old-bad: a mirror whose sourceCluster is not an alias in .spec.clusters +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-no-clusters: no target cluster in .spec.target, or in .spec.connectCluster and .spec.clusters +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-no-clusters: a mirror without .source or .sourceCluster +kafkamirrormaker2s.kafka.strimzi.io tenant-a/mm2-new-oauth: OAuth client authentication +kafkausers.kafka.strimzi.io tenant-a/user-no-spec: no .spec +kafkanodepools.kafka.strimzi.io tenant-a/pool-null-spec: no .spec +LINES + [ "$(grep -c '^ kafkas.kafka.strimzi.io tenant-b/kafka-two: ' "$WORK/out")" -eq 1 ] + if grep -qF "clean" "$WORK/out"; then echo "named an object the conversion accepts" >&2; return 1; fi + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "changed the cluster despite an object the conversion refuses" >&2; return 1; fi + rm -rf "$WORK" +} + +@test "a failed read of a Strimzi kind for the conversion refusal check aborts before the operators are stopped" { + prep + export FAKE_STRIMZI_LIST_FAIL=kafkaconnects + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -ne 0 ] + grep -qF "get kafkaconnects.v1beta2.kafka.strimzi.io --all-namespaces -o json" "$FAKE_CMDLOG" + if grep -qE "SCALE-|APPLY-|PATCH|STAMP|DELETE|ANNOTATE" "$FAKE_CMDLOG"; then echo "went on after a failed Strimzi list read" >&2; return 1; fi + rm -rf "$WORK" +} + +# The refusal check copies what the 0.51.0 tool rejects, so a different tool +# version needs that list read again from its source. +@test "the conversion image stays on Strimzi 0.51.0, the version the refusal check was read from" { + grep -qE '^CONVERSION_IMAGE="\$\{CONVERSION_IMAGE:-quay\.io/strimzi/operator:0\.51\.0@sha256:[0-9a-f]{64}\}"$' "$MIG_DIR/61" +} + +# With the CRDs serving v1 alone, every object is stored as v1 and no +# conversion runs. +@test "objects the conversion would refuse do not block a re-run once the CRDs serve only v1" { + prep + refused_objects + export FAKE_SERVED=v1 + printf '["v1"]' > "$WORK/state/stored" + rc=0 + run_migration 61 >"$WORK/out" 2>&1 || rc=$? + cat "$WORK/out"; cat "$FAKE_CMDLOG" + [ "$rc" -eq 0 ] + grep -qxF "STAMP 62" "$FAKE_CMDLOG" + rm -rf "$WORK" +} diff --git a/hack/testdata/migration-61-strimzi-v1/apps.txt b/hack/testdata/migration-61-strimzi-v1/apps.txt new file mode 100644 index 000000000..deb2852f1 --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/apps.txt @@ -0,0 +1,4 @@ +tenant-a one v3.9 +tenant-b two v3.8 +tenant-c three v4.2 +tenant-d four diff --git a/hack/testdata/migration-61-strimzi-v1/kafka-releases.txt b/hack/testdata/migration-61-strimzi-v1/kafka-releases.txt new file mode 100644 index 000000000..e617bdcb9 --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/kafka-releases.txt @@ -0,0 +1,2 @@ +tenant-a kafka-one +tenant-b diff --git a/hack/testdata/migration-61-strimzi-v1/kafkas-kraft.json b/hack/testdata/migration-61-strimzi-v1/kafkas-kraft.json new file mode 100644 index 000000000..9bb3daab3 --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/kafkas-kraft.json @@ -0,0 +1,4 @@ +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"kafka-one","annotations":{"strimzi.io/kraft":"enabled"}},"spec":{"kafka":{"version":"3.9.0"}},"status":{"kafkaMetadataState":"KRaft","kafkaMetadataVersion":"3.9-IV0","conditions":[{"type":"Ready","status":"True"}]}}, + {"metadata":{"namespace":"tenant-b","name":"kafka-two"},"spec":{"kafka":{"version":"3.8.1"}},"status":{"kafkaMetadataState":"KRaft","kafkaMetadataVersion":"3.9-IV0","conditions":[{"type":"Ready","status":"True"}]}} +]} diff --git a/hack/testdata/migration-61-strimzi-v1/kafkas-zk.json b/hack/testdata/migration-61-strimzi-v1/kafkas-zk.json new file mode 100644 index 000000000..1c0611fae --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/kafkas-zk.json @@ -0,0 +1,5 @@ +{"items":[ + {"metadata":{"namespace":"tenant-a","name":"kafka-one"},"spec":{"kafka":{"version":"3.9.0"}},"status":{"kafkaMetadataState":"KRaft","kafkaMetadataVersion":"3.9-IV0","conditions":[{"type":"Ready","status":"True"}]}}, + {"metadata":{"namespace":"tenant-c","name":"kafka-zk"},"spec":{"kafka":{"version":"3.9.0"},"zookeeper":{"replicas":3}},"status":{"kafkaMetadataState":"ZooKeeper"}}, + {"metadata":{"namespace":"tenant-d","name":"kafka-mid"},"spec":{"kafka":{"version":"3.9.0"}},"status":{"kafkaMetadataState":"KRaftPostMigration"}} +]} diff --git a/hack/testdata/migration-61-strimzi-v1/kubectl b/hack/testdata/migration-61-strimzi-v1/kubectl new file mode 100755 index 000000000..e2d899bad --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/kubectl @@ -0,0 +1,342 @@ +#!/bin/sh +# Fake kubectl for hack/migration-61-strimzi-v1-conversion.bats. It serves only +# the calls migration 61 and lib/cozystack-version.sh make, and logs each one +# to $FAKE_CMDLOG. +# +# FAKE_CMDLOG file to append a one-line record of each call to +# FAKE_STATE directory; its "stored" file holds the storedVersions +# every Strimzi CRD reports, and the conversion Job +# rewrites it to ["v1"] when it succeeds +# FAKE_NO_STRIMZI non-empty => the Kafka CRD is absent +# FAKE_KAFKAS fixture served for the Strimzi Kafka list; annotations +# written to it are kept in $FAKE_STATE +# FAKE_JOB succeed (default) | fail | noop (succeeds, converts nothing) +# | hang (neither succeeds nor fails) +# | nopod (Failed condition, no failed pod: the pod was +# never created) +# FAKE_PULL non-empty => the conversion pod cannot pull its image +# FAKE_PATCH_FAIL non-empty => every application patch exits 1 +# FAKE_CRD_READ_FAIL, FAKE_APIRES_FAIL, FAKE_SECRET_LIST_FAIL: that read +# exits 1 +# FAKE_APPS_ABSENT non-empty => the apps.cozystack.io API is not installed +# FAKE_DISCOVERY_PARTIAL non-empty => another API group fails discovery, so +# api-resources exits 1 after listing what it found +# FAKE_RELEASE_REWRITTEN_BACK: release patches do not stick +# FAKE_RELEASE_PENDING: revision v4 of kafka-one is pending-upgrade, a +# Helm action that has not finished +# FAKE_REBALANCES comma-separated ns/name served for the KafkaRebalance list +# FAKE_HELMRELEASES fixture served for the Kafka application HelmRelease +# list; none by default, and a missing file fails the read +# FAKE_KAFKATOPICS fixture served for the KafkaTopic list, the same way +# FAKE_STRIMZI_OBJECTS directory; .json in it is served for the JSON +# list of that Strimzi kind other than Kafka and +# KafkaTopic, an empty list when it is absent +# FAKE_STRIMZI_LIST_FAIL plural of one such kind whose JSON list read exits 1 +# The deployed Helm release of kafka-one is release-kafka-one.json, encoded +# the way Helm stores it; a patch of its Secret is written to +# $FAKE_STATE/release, so a second run reads what the first one wrote. +set -u + +log() { [ -n "${FAKE_CMDLOG:-}" ] && printf '%s\n' "$*" >> "$FAKE_CMDLOG"; } +log "KUBECTL $*" + +args="$*" +# The Kafka list as it stands: converted, annotated, or the fixture. +kafkas_src() { + for f in "$FAKE_STATE/kafkas-converted.json" "$FAKE_STATE/kafkas-live.json"; do + [ -f "$f" ] && { printf '%s' "$f"; return; } + done + printf '%s' "$FAKE_KAFKAS" +} +arg_after() { k="$1"; shift; p=""; for a in "$@"; do [ "$p" = "$k" ] && { printf '%s' "$a"; return; }; p="$a"; done; } + +case "$args" in + *"apply --filename -"*) + manifest=$(cat) + v=$(printf '%s\n' "$manifest" | sed -n 's/^ version: "\{0,1\}\([0-9]\{1,\}\)"\{0,1\}$/\1/p' | head -1) + log "STAMP ${v:-unknown}" + exit 0 + ;; + *"apply -f -"*) + manifest=$(cat) + case "$manifest" in + *"kind: Job"*) + order=$(printf '%s\n' "$manifest" | sed -n 's/.*v1-api-conversion.sh", "\([a-z-]*\)".*/\1/p' | tr '\n' ' ') + log "APPLY-CONVERSION ${order% }" + log "CONVERSION-DEADLINE $(printf '%s\n' "$manifest" | sed -n 's/^ activeDeadlineSeconds: \([0-9]*\)$/\1/p')" + exit 0 + ;; + esac + log "APPLY-UNKNOWN" + exit 1 + ;; + *"apply --server-side"*) + log "APPLY-CRDS $(arg_after -f "$@")" + exit 0 + ;; + *"get crd kafkas.kafka.strimzi.io --ignore-not-found -o name"*) + [ -n "${FAKE_NO_STRIMZI:-}" ] || echo "customresourcedefinition.apiextensions.k8s.io/kafkas.kafka.strimzi.io" + exit 0 + ;; + *"get crd kafkas.kafka.strimzi.io -o jsonpath={.spec.versions[*].name}"*) + printf '%s' "${FAKE_SERVED:-v1 v1beta2}" + exit 0 + ;; + *"get crd "*"storedVersions"*) + if [ -n "${FAKE_CRD_READ_FAIL:-}" ]; then echo "Error from server: etcdserver: request timed out" >&2; exit 1; fi + cat "$FAKE_STATE/stored" + exit 0 + ;; + *" annotate kafkas.v1beta2.kafka.strimzi.io "*|*" annotate kafkas.v1.kafka.strimzi.io "*) + ns=$(arg_after -n "$@"); kind=$(arg_after annotate "$@"); name=$(arg_after "$kind" "$@") + kv=""; for a in "$@"; do case "$a" in cozystack.io/*=*) kv="$a" ;; esac; done + log "ANNOTATE $ns/$name $kv" + src=$(kafkas_src); dst="$FAKE_STATE/kafkas-live.json" + [ "$src" = "$FAKE_STATE/kafkas-converted.json" ] && dst="$src" + jq --arg ns "$ns" --arg name "$name" --arg k "${kv%%=*}" --arg v "${kv#*=}" \ + '.items |= map(if .metadata.namespace == $ns and .metadata.name == $name then .metadata.annotations[$k] = $v else . end)' \ + "$src" > "$dst.new" && mv "$dst.new" "$dst" + exit 0 + ;; + *"get "*".kafka.strimzi.io --all-namespaces -o json"|*"get strimzipodsets."*".core.strimzi.io --all-namespaces -o json") + resource=$(arg_after get "$@") + case "${resource%%.*}" in + kafkas|kafkatopics) ;; + *) + if [ "${resource%%.*}" = kafkarebalances ] && [ -n "${FAKE_REBALANCE_FAIL:-}" ]; then echo "Error from server: etcdserver: request timed out" >&2; exit 1; fi + if [ "${resource%%.*}" = "${FAKE_STRIMZI_LIST_FAIL:-}" ]; then echo "Error from server: etcdserver: request timed out" >&2; exit 1; fi + f="${FAKE_STRIMZI_OBJECTS:-/nonexistent}/${resource%%.*}.json" + if [ -f "$f" ]; then cat "$f"; else echo '{"items":[]}'; fi + exit 0 + ;; + esac + ;; +esac +case "$args" in + *"get kafkarebalances.v1beta2.kafka.strimzi.io"*|*"get kafkarebalances.v1.kafka.strimzi.io"*) + if [ -n "${FAKE_REBALANCE_FAIL:-}" ]; then echo "Error from server: etcdserver: request timed out" >&2; exit 1; fi + [ -z "${FAKE_REBALANCES:-}" ] || printf '%s\n' "$FAKE_REBALANCES" | tr ',' '\n' + exit 0 + ;; + *"get helmreleases.helm.toolkit.fluxcd.io --all-namespaces -l apps.cozystack.io/application.kind=Kafka -o json"*) + if [ -n "${FAKE_HELMRELEASES:-}" ]; then cat "$FAKE_HELMRELEASES" || exit 1; else echo '{"items":[]}'; fi + exit 0 + ;; + *"get kafkatopics.v1beta2.kafka.strimzi.io --all-namespaces -o json"*|*"get kafkatopics.v1.kafka.strimzi.io --all-namespaces -o json"*) + if [ -n "${FAKE_KAFKATOPICS:-}" ]; then cat "$FAKE_KAFKATOPICS" || exit 1; else echo '{"items":[]}'; fi + exit 0 + ;; + *"get kafkas.v1beta2.kafka.strimzi.io"*) + if [ -n "${FAKE_KAFKA_LIST_FAIL:-}" ]; then echo "Error from server: Kafka list failed" >&2; exit 1; fi + case "${FAKE_SERVED:-v1 v1beta2}" in v1) echo "v1beta2 is not served" >&2; exit 1 ;; esac + cat "$(kafkas_src)" + exit 0 + ;; + *"get kafkas.v1.kafka.strimzi.io --all-namespaces -o json"|*"get kafkas.v1.kafka.strimzi.io --all-namespaces --output json") + if [ -n "${FAKE_KAFKA_LIST_FAIL:-}" ]; then echo "Error from server: Kafka list failed" >&2; exit 1; fi + cat "$(kafkas_src)" + exit 0 + ;; + *"get kafkas.v1.kafka.strimzi.io"*) + cat "$FAKE_RELEASES" + exit 0 + ;; + *"get secret -l owner=helm,name=kafka-one -o"*) + if [ -n "${FAKE_SECRET_LIST_FAIL:-}" ]; then echo "Error from server (Forbidden): secrets is forbidden" >&2; exit 1; fi + # v4 stands for a failed upgrade: still the latest revision, so uninstall + # builds it. + echo "sh.helm.release.v1.kafka-one.v3 sh.helm.release.v1.kafka-one.v4" + exit 0 + ;; + *"get secret -l owner=helm,name=kafka-failed-install -o"*) + echo "sh.helm.release.v1.kafka-failed-install.v1" + exit 0 + ;; + *"get secret sh.helm.release.v1.kafka-failed-install.v1 -o jsonpath={.data.release}"*) + if [ ! -f "$FAKE_STATE/release-failed" ]; then + gzip -c "$(dirname "$0")/release-kafka-one.json" | base64 | tr -d '\n' | base64 | tr -d '\n' > "$FAKE_STATE/release-failed" + fi + cat "$FAKE_STATE/release-failed" + exit 0 + ;; + *"patch secret sh.helm.release.v1.kafka-failed-install.v1 --type merge --patch-file "*) + jq -r '.data.release' "$(arg_after --patch-file "$@")" > "$FAKE_STATE/release-failed" + log "PATCH-RELEASE tenant-f/sh.helm.release.v1.kafka-failed-install.v1" + exit 0 + ;; + *"get deployment --all-namespaces -l app.kubernetes.io/name=entity-operator"*) + [ -n "${FAKE_NO_OPERATOR:-}" ] || echo "tenant-a kafka-one-entity-operator" + exit 0 + ;; + *"scale deployment kafka-one-entity-operator --replicas=0"*) + log "SCALE-ENTITY tenant-a/kafka-one-entity-operator" + exit 0 + ;; + *"get deployment kafka-one-entity-operator -o json"*) + echo '{"spec":{"selector":{"matchLabels":{"strimzi.io/cluster":"kafka-one","strimzi.io/name":"kafka-one-entity-operator"}}}}' + exit 0 + ;; + *"wait --for=delete pod -l strimzi.io/"*) + log "WAIT-ENTITY $(arg_after -n "$@")/$(arg_after -l "$@")" + exit 0 + ;; + *"get secret -l owner=helm,name="*) + exit 0 + ;; + *"get secret sh.helm.release.v1.kafka-one.v"*" -o jsonpath={.data.release}"*) + rev=$(arg_after secret "$@"); rev=${rev##*.} + if [ ! -f "$FAKE_STATE/release-$rev" ]; then + status=deployed + [ "$rev" != v4 ] || [ -z "${FAKE_RELEASE_PENDING:-}" ] || status=pending-upgrade + jq -c --arg s "$status" '.info.status = $s' "$(dirname "$0")/release-kafka-one.json" | + gzip -c | base64 | tr -d '\n' | base64 | tr -d '\n' > "$FAKE_STATE/release-$rev" + fi + cat "$FAKE_STATE/release-$rev" + exit 0 + ;; + *"patch secret sh.helm.release.v1.kafka-one.v"*" --type merge --patch-file "*) + rev=$(arg_after secret "$@"); rev=${rev##*.} + # FAKE_RELEASE_REWRITTEN_BACK: a Helm upgrade that was already running + # stores its v1beta2 revision over the rewrite. + [ -n "${FAKE_RELEASE_REWRITTEN_BACK:-}" ] || jq -r '.data.release' "$(arg_after --patch-file "$@")" > "$FAKE_STATE/release-$rev" + log "PATCH-RELEASE tenant-a/sh.helm.release.v1.kafka-one.$rev" + exit 0 + ;; + *"get deployment strimzi-cluster-operator --ignore-not-found -o name"*) + if [ -n "${FAKE_OPERATOR_READ_FAIL:-}" ]; then echo "Error from server: etcdserver: request timed out" >&2; exit 1; fi + [ -n "${FAKE_NO_OPERATOR:-}" ] || echo "deployment.apps/strimzi-cluster-operator" + exit 0 + ;; + *"scale deployment strimzi-cluster-operator --replicas=0"*) + log "SCALE-OPERATOR 0" + exit 0 + ;; + *"wait --for=delete pod -l name=strimzi-cluster-operator"*) + exit 0 + ;; + *"get kafkas.kafka.strimzi.io --all-namespaces --show-managed-fields -o json"*) + if [ -f "$FAKE_STATE/managed" ]; then cat "$FAKE_STATE/managed"; else cat "$(dirname "$0")/managed-kafkas.json"; fi + exit 0 + ;; + *"--all-namespaces --show-managed-fields -o json"*) + echo '{"items":[]}' + exit 0 + ;; + *"get kafkas.kafka.strimzi.io "*" --show-managed-fields -o json"*) + ns=$(arg_after -n "$@"); name=$(arg_after kafkas.kafka.strimzi.io "$@") + src="$FAKE_STATE/managed"; [ -f "$src" ] || src="$(dirname "$0")/managed-kafkas.json" + jq --arg ns "$ns" --arg n "$name" '.items[] | select(.metadata.namespace == $ns and .metadata.name == $n)' "$src" + exit 0 + ;; + *"patch kafkas.kafka.strimzi.io "*"--patch-file "*) + ns=$(arg_after -n "$@"); name=$(arg_after kafkas.kafka.strimzi.io "$@") + patch=$(arg_after --patch-file "$@") + src="$FAKE_STATE/managed"; [ -f "$src" ] || src="$(dirname "$0")/managed-kafkas.json" + rv=$(jq -r '.metadata.resourceVersion' "$patch") + log "PATCH-MANAGED $ns/$name rv=$rv $(jq -r '[.metadata.managedFields[] | "\(.manager)=\(.apiVersion)"] | join(",")' "$patch")" + # FAKE_MANAGED_CONFLICT: a concurrent write bumps the object before the + # first patch lands, so that patch carries a stale resourceVersion. + if [ -n "${FAKE_MANAGED_CONFLICT:-}" ] && [ ! -f "$FAKE_STATE/bumped" ]; then + : > "$FAKE_STATE/bumped" + jq --arg ns "$ns" --arg n "$name" '(.items[] | select(.metadata.namespace == $ns and .metadata.name == $n) | .metadata.resourceVersion) |= (tonumber + 1 | tostring)' "$src" > "$FAKE_STATE/managed.new" && mv "$FAKE_STATE/managed.new" "$FAKE_STATE/managed" + fi + if [ -n "${FAKE_MANAGED_PATCH_FAIL:-}" ]; then echo "Error from server (Forbidden): forbidden" >&2; exit 1; fi + src="$FAKE_STATE/managed"; [ -f "$src" ] || src="$(dirname "$0")/managed-kafkas.json" + cur=$(jq -r --arg ns "$ns" --arg n "$name" '.items[] | select(.metadata.namespace == $ns and .metadata.name == $n) | .metadata.resourceVersion' "$src") + if [ "$cur" != "$rv" ]; then + echo "Error from server (Conflict): Operation cannot be fulfilled on kafkas.kafka.strimzi.io \"$name\": the object has been modified; please apply your changes to the latest version and try again" >&2 + exit 1 + fi + jq --arg ns "$ns" --arg n "$name" --slurpfile p "$patch" \ + '(.items[] | select(.metadata.namespace == $ns and .metadata.name == $n) | .metadata) |= (.managedFields = $p[0].metadata.managedFields | .resourceVersion = (.resourceVersion | tonumber + 1 | tostring))' \ + "$src" > "$FAKE_STATE/managed.new" && mv "$FAKE_STATE/managed.new" "$FAKE_STATE/managed" + exit 0 + ;; + *"wait --for=condition=Established"*) + exit 0 + ;; + *"get job strimzi-v1-api-conversion --ignore-not-found -o jsonpath={.metadata.name} {.status.succeeded}{.status.failed}"*) + # FAKE_OLD_JOB: a Job an earlier attempt left behind, running, succeeded + # or failed. + case "${FAKE_OLD_JOB:-}" in + running) printf 'strimzi-v1-api-conversion ' ;; + succeeded|failed) printf 'strimzi-v1-api-conversion 1' ;; + esac + exit 0 + ;; + *"get job strimzi-v1-api-conversion -o jsonpath={.status.succeeded}"*) + case "${FAKE_JOB:-succeed}" in + succeed) + if [ -n "${FAKE_CONVERSION_CHANGES_SPEC:-}" ] && [ ! -f "$FAKE_STATE/kafkas-converted.json" ]; then + jq '.items |= map( + .apiVersion = "kafka.strimzi.io/v1" + | .metadata.generation += 1 + | del(.spec.kafka.replicas, .spec.kafka.storage) + | del(.status.kafkaMetadataState) + | if .spec.entityOperator.topicOperator.reconciliationIntervalSeconds != null then + .spec.entityOperator.topicOperator |= + (.reconciliationIntervalMs = (.reconciliationIntervalSeconds * 1000) + | del(.reconciliationIntervalSeconds)) + else . end)' "$(kafkas_src)" > "$FAKE_STATE/kafkas-converted.json" + fi + printf '["v1"]' > "$FAKE_STATE/stored" + echo 1 + ;; + noop) echo 1 ;; + esac + exit 0 + ;; + *"get job strimzi-v1-api-conversion -o jsonpath={.status.failed}"*) + case "${FAKE_JOB:-succeed}" in + fail) printf '1' ;; + esac + case "$args" in + *'conditions[?(@.type=="Failed")]'*) + case "${FAKE_JOB:-succeed}" in fail|nopod) printf ' True' ;; esac + ;; + esac + exit 0 + ;; + *"get pod -l batch.kubernetes.io/job-name=strimzi-v1-api-conversion -o jsonpath="*) + [ -z "${FAKE_PULL:-}" ] || printf ' ImagePullBackOff' + exit 0 + ;; + *" logs "*) + echo "fake conversion log" + exit 0 + ;; + "delete "*|*" delete "*) + cascade="" + case "$args" in *"--cascade=foreground"*) cascade=" foreground" ;; esac + log "DELETE $(arg_after delete "$@") $(arg_after "$(arg_after delete "$@")" "$@")$cascade" + exit 0 + ;; + *"api-resources --api-group=apps.cozystack.io"*) + echo "kafkas.apps.cozystack.io" + if [ -n "${FAKE_DISCOVERY_PARTIAL:-}" ]; then echo "error: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request" >&2; exit 1; fi + exit 0 + ;; + *"get --raw /apis/apps.cozystack.io/v1alpha1"*) + if [ -n "${FAKE_APIRES_FAIL:-}" ]; then echo "Error from server (ServiceUnavailable): the server is currently unable to handle the request" >&2; exit 1; fi + if [ -n "${FAKE_APPS_ABSENT:-}" ]; then echo "Error from server (NotFound): the server could not find the requested resource" >&2; exit 1; fi + echo '{"kind":"APIResourceList","groupVersion":"apps.cozystack.io/v1alpha1","resources":[{"name":"kafkas","kind":"Kafka"},{"name":"kafkas/status","kind":"Kafka"}]}' + exit 0 + ;; + *"get kafkas.apps.cozystack.io"*" kafka-"*) + # Release names of the applications; failed-install has no Kafka CR. + printf '%s\n' 'tenant-a kafka-one' 'tenant-f kafka-failed-install' + exit 0 + ;; + *"get kafkas.apps.cozystack.io"*) + cat "$FAKE_APPS" + exit 0 + ;; + *"patch kafkas.apps.cozystack.io"*) + if [ -n "${FAKE_PATCH_FAIL:-}" ]; then echo "Error from server (Conflict)" >&2; exit 1; fi + log "PATCH $(arg_after -n "$@")/$(arg_after kafkas.apps.cozystack.io "$@") $(arg_after -p "$@")" + exit 0 + ;; +esac +log "UNMODELLED" +printf 'fake kubectl: unmodelled call: %s\n' "$args" >&2 +exit 1 diff --git a/hack/testdata/migration-61-strimzi-v1/managed-kafkas.json b/hack/testdata/migration-61-strimzi-v1/managed-kafkas.json new file mode 100644 index 000000000..bea24efd5 --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/managed-kafkas.json @@ -0,0 +1,54 @@ +{ + "items": [ + { + "metadata": { + "namespace": "tenant-a", + "name": "kafka-one", + "managedFields": [ + { + "manager": "helm-controller", + "operation": "Apply", + "apiVersion": "kafka.strimzi.io/v1beta2", + "fieldsType": "FieldsV1", + "fieldsV1": { + "f:spec": { + "f:kafka": { + "f:version": {} + } + } + } + }, + { + "manager": "strimzi-cluster-operator", + "operation": "Update", + "apiVersion": "kafka.strimzi.io/v1beta2", + "subresource": "status", + "fieldsType": "FieldsV1", + "fieldsV1": { + "f:status": {} + } + } + ], + "resourceVersion": "100" + } + }, + { + "metadata": { + "namespace": "tenant-b", + "name": "kafka-two", + "managedFields": [ + { + "manager": "helm-controller", + "operation": "Apply", + "apiVersion": "kafka.strimzi.io/v1", + "fieldsType": "FieldsV1", + "fieldsV1": { + "f:spec": {} + } + } + ], + "resourceVersion": "101" + } + } + ] +} \ No newline at end of file diff --git a/hack/testdata/migration-61-strimzi-v1/release-kafka-one.json b/hack/testdata/migration-61-strimzi-v1/release-kafka-one.json new file mode 100644 index 000000000..f995a4926 --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/release-kafka-one.json @@ -0,0 +1 @@ +{"name":"kafka-one","version":3,"info":{"status":"deployed"},"manifest":"---\n# Source: kafka/templates/kafka.yaml\napiVersion: kafka.strimzi.io/v1beta2\nkind: Kafka\nmetadata:\n name: kafka-one\n---\n# Source: kafka/templates/topics.yaml\napiVersion: kafka.strimzi.io/v1beta2\nkind: KafkaTopic\nmetadata:\n name: kafka-one-orders\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: kafka-one-metrics\n","hooks":[{"name":"kafka-one-kafka-migration","manifest":"apiVersion: batch/v1\nkind: Job\n# script applies apiVersion: kafka.strimzi.io/v1beta2\n"}]} \ No newline at end of file diff --git a/hack/testdata/migration-61-strimzi-v1/strimzi-crds-0.51.0.yaml b/hack/testdata/migration-61-strimzi-v1/strimzi-crds-0.51.0.yaml new file mode 100644 index 000000000..76a5f108a --- /dev/null +++ b/hack/testdata/migration-61-strimzi-v1/strimzi-crds-0.51.0.yaml @@ -0,0 +1 @@ +# Stand-in for strimzi-crds-0.51.0.yaml; the fake kubectl never reads it. diff --git a/packages/core/platform/images/migrations/Dockerfile b/packages/core/platform/images/migrations/Dockerfile index 6084a9e5e..7a6bcf6b5 100644 --- a/packages/core/platform/images/migrations/Dockerfile +++ b/packages/core/platform/images/migrations/Dockerfile @@ -29,6 +29,14 @@ RUN set -eux; \ echo "${sha256} /usr/bin/etcd-migrate" | sha256sum -c -; \ chmod +x /usr/bin/etcd-migrate +# CRDs of Strimzi 0.51.0, the last release serving both v1beta2 and v1. +# Migration 61 applies them so Strimzi's conversion tool can move every stored +# object to v1 before the kafka-operator release installs the v1-only CRDs. +RUN mkdir /strimzi-crds && \ + curl -fsSL -o /strimzi-crds/strimzi-crds-0.51.0.yaml \ + https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.51.0/strimzi-crds-0.51.0.yaml && \ + echo "a11257e67985a946a6facfb17d729aac5670b89675d410d13162fe455a5ac7a8 /strimzi-crds/strimzi-crds-0.51.0.yaml" | sha256sum -c - + COPY migrations /migrations # v1alpha2 etcd CRDs, vendored into the build context by `make image-migrations`. # migration 50 applies these before adoption so it does not deadlock waiting for diff --git a/packages/core/platform/images/migrations/migrations/61 b/packages/core/platform/images/migrations/migrations/61 new file mode 100755 index 000000000..c70895a61 --- /dev/null +++ b/packages/core/platform/images/migrations/migrations/61 @@ -0,0 +1,661 @@ +#!/bin/bash +# Migration 61 --> 62 +# Move the Strimzi custom resources to the v1 API before Strimzi 1.x arrives. +# +# Strimzi 1.0 removed the v1beta2 API from its CRDs. The kafka-operator +# release replaces the CRDs on upgrade (upgradeCRDs: CreateReplace), and the +# API server refuses a CRD whose spec.versions no longer lists a version that +# status.storedVersions still records. So every stored object has to be +# rewritten as v1, and storedVersions narrowed to [v1], before the new +# operator arrives. That needs CRDs serving both versions, which only Strimzi +# 0.49 to 0.51 ship: the 0.51.0 CRDs are baked into this image (see the +# Dockerfile). The conversion itself is Strimzi's v1-api-conversion tool, run +# as a Job from the 0.51.0 operator image: convert-resource rewrites the +# objects, crd-upgrade switches the storage version and narrows +# storedVersions. +# +# Strimzi 0.46 also removed ZooKeeper. A ZooKeeper-based Kafka that did not +# reach KRaft on the release carrying the chart's ZooKeeper to KRaft +# migration cannot be run, converted or migrated by anything after this +# point, so this migration refuses to proceed while one is left. +# +# Strimzi 1.2 runs neither Kafka 3.8 nor 3.9, so every Kafka application on +# v3.8 or v3.9 is moved to v4.3. Until the new kafka chart arrives, the +# release renders with the old one, which rejects v4.3, and the HelmRelease +# reports a values error; no Helm action runs in that window. +# +# Ordering: this runs as the platform chart's pre-upgrade hook, before the +# upgrade re-points cozystack-packages, so on a variant that runs migrations +# neither the new operator nor the new kafka chart is rendered before it. The +# default variant renders no migration hook, and there these steps are manual. +# +# Re-runnable: every step tolerates having run before. + +set -euo pipefail + +# shellcheck source=lib/cozystack-version.sh +. "$(dirname "$0")/lib/cozystack-version.sh" + +STRIMZI_CRD_FILE="${STRIMZI_CRD_FILE:-/strimzi-crds/strimzi-crds-0.51.0.yaml}" +CONVERSION_IMAGE="${CONVERSION_IMAGE:-quay.io/strimzi/operator:0.51.0@sha256:ef3e406aa09449dd0241d53cdc81e15e58bf59b51f2425fb64384a179d676b1d}" +CONVERSION_NS="${CONVERSION_NS:-cozy-kafka-operator}" +CONVERSION_NAME=strimzi-v1-api-conversion +CONVERSION_TIMEOUT="${CONVERSION_TIMEOUT:-1800}" +# The run that starts the Job waits up to CONVERSION_TIMEOUT for it. As the +# platform hook, a run is cut off sooner by the Helm upgrade timeout +# (cozystack-operator --helmrelease-upgrade-timeout, 10m by default), and the +# retry, the next pod of the hook Job or a new hook Job once helm-controller +# retries the upgrade, waits up to CONVERSION_TIMEOUT again for the same Job. +# The Job is not owned by the hook Job and nothing else bounds it, so the +# deadline is the only limit: a conversion gets two waits, and a Job whose pod +# never starts then fails and the next run starts the conversion over. +CONVERSION_DEADLINE=$((CONVERSION_TIMEOUT * 2)) +POLL_INTERVAL="${POLL_INTERVAL:-5}" +KAFKA_TARGET_VERSION=v4.3 + +# The CRDs the conversion tool rewrites. kafkamirrormakers.kafka.strimzi.io +# is not among them: MirrorMaker 1 left in Strimzi 0.46 and its CRD is left +# behind untouched. +STRIMZI_CRDS="kafkas.kafka.strimzi.io kafkanodepools.kafka.strimzi.io +kafkaconnects.kafka.strimzi.io kafkaconnectors.kafka.strimzi.io +kafkabridges.kafka.strimzi.io kafkamirrormaker2s.kafka.strimzi.io +kafkarebalances.kafka.strimzi.io kafkatopics.kafka.strimzi.io +kafkausers.kafka.strimzi.io strimzipodsets.core.strimzi.io" + +# --ignore-not-found separates a cluster without Strimzi (no output, rc 0) +# from an API error, which aborts under set -e. +crd=$(kubectl get crd kafkas.kafka.strimzi.io --ignore-not-found -o name) +if [ -z "$crd" ]; then + echo "Strimzi is not installed; nothing to convert" + stamp_cozystack_version 62 + exit 0 +fi + +# The running 0.45 operator must not write to the resources from here on. +# Strimzi supports the conversion only under an operator that knows v1 +# (0.49 or later), and 0.45 rewrites the Kafka status through v1beta2 on every +# reconcile, because the v1 schema prunes its kafkaMetadataState field. Each +# such write records a v1beta2 managedFields entry again after the rewrite +# below. The kafka-operator release brings the operator back at Strimzi 1.2: +# its chart sets the replica count, so the upgrade restores it. Kafka pods keep +# running while no operator reconciles them. +stop_old_operator() { + operator=$(kubectl -n "$CONVERSION_NS" get deployment strimzi-cluster-operator --ignore-not-found -o name) + if [ -n "$operator" ]; then + kubectl -n "$CONVERSION_NS" scale deployment strimzi-cluster-operator --replicas=0 + kubectl -n "$CONVERSION_NS" wait --for=delete pod -l name=strimzi-cluster-operator --timeout=300s + fi + # The topic and user operators of each cluster write KafkaTopic and + # KafkaUser status through v1beta2 the same way, also while they shut down. + # Only now that the cluster operator is gone does a scale-down stick; the + # 1.2 operator scales them back up when it rolls them onto its own image. + kubectl get deployment --all-namespaces -l app.kubernetes.io/name=entity-operator,app.kubernetes.io/managed-by=strimzi-cluster-operator \ + -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{"\n"}{end}' | + while read -r ns name; do + kubectl -n "$ns" scale deployment "$name" --replicas=0 + # An empty selector would wait on every pod in the namespace. + selector=$(kubectl -n "$ns" get deployment "$name" -o json | + jq -er '.spec.selector.matchLabels // {} | to_entries | map("\(.key)=\(.value)") | join(",") | select(length > 0)') + kubectl -n "$ns" wait --for=delete pod -l "$selector" --timeout=300s + done + } + +# Called as an if condition, where set -e does not apply: a failed read exits +# here, or it would pass for an absent CRD. +stored_v1_only() { + for c in $STRIMZI_CRDS; do + if ! v=$(kubectl get crd "$c" --ignore-not-found -o jsonpath='{.status.storedVersions}'); then + echo "ERROR: reading the stored versions of $c failed" >&2 + exit 1 + fi + if [ -n "$v" ] && [ "$v" != '["v1"]' ]; then + return 1 + fi + done + return 0 + } + +# A Job whose pod the API server refused fails on its deadline with no failed +# pod to count, so its Failed condition is read as well. +job_failed() { + f=$(kubectl -n "$CONVERSION_NS" get job "$CONVERSION_NAME" -o jsonpath='{.status.failed} {.status.conditions[?(@.type=="Failed")].status}') + case "$f" in + [1-9]*|*True*) return 0 ;; + esac + return 1 + } + +# wait_for_conversion: 0 once the conversion Job has succeeded; on a failure +# or a timeout, print its logs and return 1. +wait_for_conversion() { + echo "Waiting for the conversion Job" + waited=0 + pull_reported= + while :; do + succeeded=$(kubectl -n "$CONVERSION_NS" get job "$CONVERSION_NAME" -o jsonpath='{.status.succeeded}') + # A pod that cannot pull its image leaves no logs to print, and its Job + # fails only at the deadline. + if [ -z "$pull_reported" ]; then + waiting=$(kubectl -n "$CONVERSION_NS" get pod -l "batch.kubernetes.io/job-name=$CONVERSION_NAME" \ + -o jsonpath='{.items[*].status.initContainerStatuses[*].state.waiting.reason} {.items[*].status.containerStatuses[*].state.waiting.reason}' || true) + case "$waiting" in + *ErrImagePull*|*ImagePullBackOff*) + echo "ERROR: the conversion pod cannot pull $CONVERSION_IMAGE. A cluster that cannot reach quay.io needs that image mirrored under the same name and digest, for example through a registry mirror for quay.io in the container runtime of the nodes. The Job fails ${CONVERSION_DEADLINE}s after it started, and the next run of the migration starts it again." >&2 + pull_reported=1 + ;; + esac + fi + if [ "${succeeded:-0}" -ge 1 ]; then + kubectl -n "$CONVERSION_NS" logs "job/$CONVERSION_NAME" --all-containers --prefix || true + return 0 + fi + if job_failed || [ "$waited" -ge "$CONVERSION_TIMEOUT" ]; then + echo "ERROR: the Strimzi v1 API conversion Job did not succeed; its logs:" >&2 + kubectl -n "$CONVERSION_NS" logs "job/$CONVERSION_NAME" --all-containers --prefix >&2 || true + return 1 + fi + sleep "$POLL_INTERVAL" + waited=$((waited + POLL_INTERVAL)) + done + } + +# The Job's ClusterRole can patch CRDs and write ConfigMaps everywhere, so +# none of it outlives a conversion that has finished, whether it succeeded or +# failed. A failed Job itself stays for diagnosis. A Job still running when +# the wait times out keeps its rights: the retry waits for that same Job. +remove_conversion_rbac() { + kubectl delete clusterrolebinding "$CONVERSION_NAME" --ignore-not-found + kubectl delete clusterrole "$CONVERSION_NAME" --ignore-not-found + kubectl -n "$CONVERSION_NS" delete serviceaccount "$CONVERSION_NAME" --ignore-not-found + } + +remove_conversion_job() { + kubectl -n "$CONVERSION_NS" delete job "$CONVERSION_NAME" --ignore-not-found --wait=true + remove_conversion_rbac + } + +# The platform hook can time out while the Job converts, and the retry kills +# the hook at whatever point it reached. A Job still running then has to +# finish before anything here touches the CRDs, or the CRD apply below would +# switch the storage version back under it. Its outcome is read from the CRDs +# afterwards, not from the Job. +job_state=$(kubectl -n "$CONVERSION_NS" get job "$CONVERSION_NAME" --ignore-not-found -o jsonpath='{.metadata.name} {.status.succeeded}{.status.failed}{.status.conditions[?(@.type=="Failed")].status}') +case "$job_state" in + "$CONVERSION_NAME ") + echo "A conversion Job from an earlier attempt is still running" + # backoffLimit is 0, so a Job reporting no failure has not finished. + if ! wait_for_conversion; then + if ! job_failed; then + echo "ERROR: the conversion Job of an earlier attempt is still running; retry once it finishes" >&2 + exit 1 + fi + fi + ;; +esac +# A previous conversion can still be changing these objects, so the gate runs +# after its wait but before a new operator stop or any CRD mutation. +served=$(kubectl get crd kafkas.kafka.strimzi.io -o jsonpath='{.spec.versions[*].name}') +case " $served " in + *" v1beta2 "*) kafka_api=v1beta2 ;; + *" v1 "*) kafka_api=v1 ;; + *) echo "ERROR: no supported Strimzi Kafka API is served" >&2; exit 1 ;; +esac +# KRaft is the last Strimzi metadata state, so the state with a metadata +# version proves KRaft. The conversion deletes kafkaMetadataState, so without +# it the proof is Ready=True, or the annotation a previous run recorded below. +# Ready because the assembly operator retains kafkaMetadataVersion on failure: +# https://github.com/strimzi/strimzi-kafka-operator/blob/0.51.0/cluster-operator/src/main/java/io/strimzi/operator/cluster/operator/assembly/KafkaAssemblyOperator.java#L189-L192 +KRAFT_ANNOTATION=cozystack.io/kraft-verified +kafkas=$(kubectl get "kafkas.$kafka_api.kafka.strimzi.io" --all-namespaces -o json) +unproven=$(printf '%s' "$kafkas" | jq --slurp --exit-status --raw-output --arg verified "$KRAFT_ANNOTATION" ' + def nonempty_string: type == "string" and length > 0; + def unproven_reason: + if .spec.zookeeper != null then "ZooKeeper section in spec" + elif (.status | type) != "object" then "no status" + elif .status.kafkaMetadataState != null and .status.kafkaMetadataState != "KRaft" + then "metadata state \(.status.kafkaMetadataState)" + elif (.status.kafkaMetadataVersion | nonempty_string | not) then "no kafkaMetadataVersion" + elif .status.kafkaMetadataState == "KRaft" or + any(.status.conditions[]?; .type == "Ready" and .status == "True") or + (.metadata.annotations[$verified] | nonempty_string) + then null + else "not Ready" end; + if length != 1 or (.[0] | type) != "object" or (.[0].items | type) != "array" + then error("invalid Strimzi Kafka list") else .[0].items end + | map( + if type != "object" or (.metadata.namespace | nonempty_string | not) or + (.metadata.name | nonempty_string | not) or (.spec | type) != "object" + then error("invalid Strimzi Kafka object") else . end + | unproven_reason as $reason + | select($reason != null) + | "\(.metadata.namespace)/\(.metadata.name)" + + (if (.metadata.annotations["meta.helm.sh/release-name"] | nonempty_string) + then " (Helm release: \(.metadata.namespace)/\(.metadata.annotations["meta.helm.sh/release-name"]))" + else "" end) + ": \($reason)") + | join("\n")') +if [ -n "$unproven" ]; then + echo "ERROR: KRaft is not confirmed for these Kafka clusters:" >&2 + printf '%s\n' "$unproven" | sed 's/^/ /' >&2 + echo "Strimzi 1.x cannot run ZooKeeper or an unfinished migration. Upgrade through Cozystack v1.7 and let each Kafka finish its KRaft migration and report Ready before upgrading past it. A Kafka created after this migration stopped the Strimzi operators has no status until the upgrade finishes: delete that application and create it again afterwards." >&2 + exit 1 +fi + +# The v1.7 chart removes its drain rebalance only when the shared pool is free. +rebalances=$(kubectl get "kafkarebalances.$kafka_api.kafka.strimzi.io" --all-namespaces \ + -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}{"\n"}{end}') +drains=$(printf '%s\n' "$rebalances" | grep -- '-drain-kafka$' || true) +if [ -n "$drains" ]; then + echo "ERROR: a ZooKeeper to KRaft migration of Cozystack v1.7 is still draining a shared broker pool:" >&2 + printf '%s\n' "$drains" | sed 's/^/ /' >&2 + echo "Let the kafka application upgrade on v1.7 finish, which removes these KafkaRebalances, before upgrading past it." >&2 + exit 1 +fi + +# Kafka 4.2/4.3 reject these topic configs that Kafka 3.x accepted, and the +# new kafka chart refuses to render them, so such an application would stay on +# its pre-upgrade spec under an operator that cannot run it; a broker whose log +# holds a segment.bytes below the floor does not start. The same limits are in +# packages/apps/kafka/templates/topics.yaml, +# packages/core/platform/templates/_helpers.tpl and +# packages/system/backupstrategy-controller/files/kafka-backup.sh. +# Application values are read from the HelmReleases, the same source as the +# platform chart's guard; KafkaTopics cover topics created outside the chart. +SEGMENT_BYTES_MIN=1048576 +# shellcheck disable=SC2016 # a jq program, expanded by jq +rejected_topic_configs=' + (("message.downconversion.enable", "message.format.version", "message.timestamp.difference.max.ms") + as $key | select($config | has($key)) | "\($owner): topic \($topic) sets \($key)"), + ($config["segment.bytes"] + # Kafka trims characters up to U+0020 and reads the rest with + # Integer.parseInt, which allows a sign and leading zeros. + | (if type == "number" then . + elif type == "string" then + capture("^[\\x00-\\x20]*(?:\\+|(?-))?0*(?[0-9]+)[\\x00-\\x20]*$") + | "\(.sign // "")\(.digits)" | tonumber + else empty end) + | select(. < $min) + | "\($owner): topic \($topic) sets segment.bytes=\(.), below the Kafka 4 minimum of \($min)")' +releases=$(kubectl get helmreleases.helm.toolkit.fluxcd.io --all-namespaces -l apps.cozystack.io/application.kind=Kafka -o json) +topics=$(kubectl get "kafkatopics.$kafka_api.kafka.strimzi.io" --all-namespaces -o json) +rejected=$(printf '%s' "$releases" | jq --raw-output --argjson min "$SEGMENT_BYTES_MIN" ' + .items[] + | "\(.metadata.namespace)/\(.metadata.labels["apps.cozystack.io/application.name"] // .metadata.name)" as $owner + | (.spec.values.topics // []) | if type == "array" then .[] else empty end + | select(type == "object") | .name as $topic + | (.config | if type == "object" then . else {} end) as $config + | '"$rejected_topic_configs") +rejected_crs=$(printf '%s' "$topics" | jq --raw-output --argjson min "$SEGMENT_BYTES_MIN" ' + .items[] + | "\(.metadata.namespace)/kafkatopic \(.metadata.name)" as $owner + | (.spec.topicName // .metadata.name) as $topic + | (.spec.config | if type == "object" then . else {} end) as $config + | '"$rejected_topic_configs") +rejected=$(printf '%s\n%s' "$rejected" "$rejected_crs" | sed '/^$/d') +if [ -n "$rejected" ]; then + echo "ERROR: Kafka 4 rejects topic configs these Kafka applications and KafkaTopics still set:" >&2 + printf '%s\n' "$rejected" | sed 's/^/ /' >&2 + echo "Remove the configs Kafka 4 removed and raise segment.bytes to the minimum before running this migration." >&2 + exit 1 +fi + +# Strimzi's conversion tool refuses some v1beta2 objects outright, and +# convert-resource stops at the first one, with the operators stopped and the +# objects it reached before already converted. These are the refusals of the +# tool in CONVERSION_IMAGE, read from the strimzi-kafka-operator 0.51.0 source +# (v1-api-conversion/src/main/java/io/strimzi/kafka/api/conversion/v1/converter/): +# a missing .spec on every kind, and what KafkaConversions, +# SharedConversions, MirrorMaker2Conversions and the Conversion.move of +# KafkaConnectConverter reject. Re-read them when CONVERSION_IMAGE moves off +# 0.51.0; a Bats case fails on that change. The tool cannot do a dry run here +# instead: it is Java, and this image has no JVM. Once the CRDs serve v1 +# alone, every object is stored as v1 and nothing is converted. +# shellcheck disable=SC2016 # a jq program, expanded by jq +conversion_refusals=' + def oauth: .authentication.type? == "oauth"; + .spec as $s + | if $s == null then "no .spec" + elif $crd == "kafkas.kafka.strimzi.io" then + (select($s.kafka.resources != null) | ".spec.kafka.resources (use a KafkaNodePool)"), + (select(any($s.kafka.listeners[]?; oauth)) | "OAuth authentication on a listener"), + (select(any($s.kafka.listeners[]?; .authentication.type? == "custom" and .authentication.secrets != null)) + | "secrets in a custom listener authentication"), + (select($s.kafka.authorization.type? == "keycloak") | "Keycloak authorization"), + (select($s.kafka.authorization.type? == "opa") | "Open Policy Agent authorization") + elif $crd == "kafkabridges.kafka.strimzi.io" then + select($s | oauth) | "OAuth client authentication" + elif $crd == "kafkaconnects.kafka.strimzi.io" then + (select($s | oauth) | "OAuth client authentication"), + (select($s.externalConfiguration != null) | ".spec.externalConfiguration"), + (select($s.build.output.additionalKanikoOptions != null and $s.build.output.additionalBuildOptions != null) + | "both .spec.build.output.additionalKanikoOptions and additionalBuildOptions") + elif $crd == "kafkamirrormaker2s.kafka.strimzi.io" then + [$s.clusters[]?.alias] as $aliases + | (if $s.target != null then $s.target.alias else $s.connectCluster end) as $target + | (select($s.externalConfiguration != null) | ".spec.externalConfiguration"), + (select(any($s.clusters[]?, $s.target, $s.mirrors[]?.source; oauth)) | "OAuth client authentication"), + (select($s.target == null and ($aliases == [] or $s.connectCluster == null)) + | "no target cluster in .spec.target, or in .spec.connectCluster and .spec.clusters"), + (select($s.target == null and $aliases != [] and $s.connectCluster != null and + ($aliases | index([$s.connectCluster])) == null) + | ".spec.connectCluster is not an alias in .spec.clusters"), + ($s.mirrors[]? | + (select(.targetCluster != null and .targetCluster != $target) + | "a mirror whose targetCluster is not the target cluster"), + (select(.source == null and ($aliases == [] or .sourceCluster == null)) + | "a mirror without .source or .sourceCluster"), + (select(.source == null and $aliases != [] and .sourceCluster != null and + (.sourceCluster as $source | $aliases | index([$source])) == null) + | "a mirror whose sourceCluster is not an alias in .spec.clusters")) + else empty end' +case " $served " in + *" v1beta2 "*) + refused= + for c in $STRIMZI_CRDS; do + list=$(kubectl get "${c%%.*}.$kafka_api.${c#*.}" --all-namespaces -o json) + found=$(printf '%s' "$list" | jq --raw-output --arg crd "$c" '.items[] + | "\($crd) \(.metadata.namespace)/\(.metadata.name)" as $object + | [ '"$conversion_refusals"' ] | unique | .[] | "\($object): \(.)"') + refused=$(printf '%s\n%s' "$refused" "$found" | sed '/^$/d') + done + if [ -n "$refused" ]; then + echo "ERROR: the Strimzi v1 API conversion refuses these objects:" >&2 + printf '%s\n' "$refused" | sed 's/^/ /' >&2 + echo "Change each one as the Strimzi v1 API requires and run this migration again." >&2 + exit 1 + fi + ;; +esac + +# Record the proof before anything converts, so a retry after the conversion +# accepts a Kafka that passed only on its KRaft state. +printf '%s' "$kafkas" | jq --raw-output --arg verified "$KRAFT_ANNOTATION" '.items[] + | select(.metadata.annotations[$verified] | type == "string" and length > 0 | not) + | "\(.metadata.namespace) \(.metadata.name) \(.status.kafkaMetadataVersion)"' | +while read -r ns name version; do + kubectl -n "$ns" annotate "kafkas.$kafka_api.kafka.strimzi.io" "$name" --overwrite "$KRAFT_ANNOTATION=$version" +done + +# A Job failed on its deadline can still have a pod terminating, and +# convert-resource writes until it is gone. A foreground deletion returns +# only once the Job's pods are. +if [ -n "$job_state" ]; then + kubectl -n "$CONVERSION_NS" delete job "$CONVERSION_NAME" --ignore-not-found --cascade=foreground --wait=true +fi + +if stored_v1_only; then + echo "Every Strimzi CRD already stores only v1; skipping the conversion" + # The 0.45 operator can only still be running while the CRDs serve + # v1beta2. Once they serve v1 alone the operator is 1.x, and a run by hand + # must not stop it: no Helm upgrade would bring it back. + case " $served " in + *" v1beta2 "*) stop_old_operator ;; + esac + remove_conversion_job +else + if [ ! -r "$STRIMZI_CRD_FILE" ]; then + echo "ERROR: $STRIMZI_CRD_FILE is not readable; set STRIMZI_CRD_FILE to strimzi-crds-0.51.0.yaml from the Strimzi 0.51.0 release" >&2 + exit 1 + fi + stop_old_operator + # These CRDs mark v1beta2 as the storage version, so a re-run after a + # partial crd-upgrade switches storage back and converts from the start. + echo "Applying the Strimzi 0.51.0 CRDs, which serve both v1beta2 and v1" + kubectl apply --server-side --force-conflicts --field-manager=cozystack-migration-61 -f "$STRIMZI_CRD_FILE" + for c in $STRIMZI_CRDS; do + kubectl wait --for=condition=Established --timeout=120s "crd/$c" + done + + # Whatever an earlier attempt left has finished by now, and its result was + # undone by the CRD apply above, so the conversion starts over. + kubectl -n "$CONVERSION_NS" delete job "$CONVERSION_NAME" --ignore-not-found --wait=true + kubectl apply -f - <&2 + for c in $STRIMZI_CRDS; do + echo " $c: $(kubectl get crd "$c" --ignore-not-found -o jsonpath='{.status.storedVersions}')" >&2 + done + remove_conversion_rbac + exit 1 + fi + + remove_conversion_job +fi + +# Moving the applications off Kafka 3.x first makes the old chart reject +# their values, so no Helm action can store a new v1beta2 revision behind the +# rewrite below. +# kubectl api-resources fails when any API group fails discovery, so only +# this group is read: NotFound means it is not installed, any other error +# aborts. +work=$(mktemp -d) +trap 'rm -rf "$work"' EXIT +if app_group=$(kubectl get --raw /apis/apps.cozystack.io/v1alpha1 2>"$work/app-group.err"); then + kafka_apps=$(printf '%s' "$app_group" | jq --raw-output 'any(.resources[]?; .name == "kafkas")') +elif grep -qF '(NotFound)' "$work/app-group.err"; then + kafka_apps=false +else + echo "ERROR: reading the apps.cozystack.io API failed: $(cat "$work/app-group.err")" >&2 + exit 1 +fi +has_kafka_apps() { [ "$kafka_apps" = true ]; } +if has_kafka_apps; then + kubectl get kafkas.apps.cozystack.io --all-namespaces \ + -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.name}{" "}{.spec.version}{"\n"}{end}' | + while read -r ns name version; do + case "$version" in + v3.8|v3.9) + echo "Moving Kafka $ns/$name from $version to $KAFKA_TARGET_VERSION" + kubectl -n "$ns" patch kafkas.apps.cozystack.io "$name" --type merge \ + -p "{\"spec\":{\"version\":\"$KAFKA_TARGET_VERSION\"}}" + ;; + esac + done +fi + +# Helm builds the objects of a stored release before it upgrades, +# uninstalls or rolls back to it, and fails on an API version the server no +# longer serves, so every Kafka release would be stuck once the v1-only CRDs +# land. Uninstall reads the latest revision whatever its status, and upgrade +# falls back to the last failed one, so every revision of each release is +# rewritten, the way helm-mapkubeapis does: the Secret holds base64 of +# gzipped JSON, inside the base64 of the Secret data. A release whose install +# failed has no Kafka CR, so the releases are listed from the applications as +# well. The chart's hooks carry no Strimzi kinds, so only the manifest needs +# the rewrite. + +list_releases() { + kubectl get kafkas.v1.kafka.strimzi.io --all-namespaces \ + -o jsonpath='{range .items[*]}{.metadata.namespace}{" "}{.metadata.annotations.meta\.helm\.sh/release-name}{"\n"}{end}' > "$work/releases.all" + if has_kafka_apps; then + kubectl get kafkas.apps.cozystack.io --all-namespaces \ + -o jsonpath='{range .items[*]}{.metadata.namespace}{" kafka-"}{.metadata.name}{"\n"}{end}' >> "$work/releases.all" + fi + sort -u "$work/releases.all" > "$work/releases" + } + +# scan_releases rewrite|check: rewrite every revision that still names +# v1beta2, or list the ones that do and the ones still pending. +scan_releases() { + list_releases + while read -r ns release; do + [ -n "$release" ] || continue + secrets=$(kubectl -n "$ns" get secret -l "owner=helm,name=$release" -o jsonpath='{.items[*].metadata.name}') + for secret in $secrets; do + kubectl -n "$ns" get secret "$secret" -o jsonpath='{.data.release}' | base64 -d | base64 -d | gzip -dc > "$work/release.json" + if [ "$1" = check ]; then + status=$(jq -r '.info.status' "$work/release.json") + case "$status" in + pending-*) + echo " Helm release $ns/$secret is $status" + continue + ;; + esac + fi + if ! jq -e '.manifest | contains("apiVersion: kafka.strimzi.io/v1beta2")' "$work/release.json" > /dev/null; then + continue + fi + if [ "$1" = check ]; then + echo " Helm release $ns/$secret" + continue + fi + echo "Rewriting Helm release $ns/$secret to kafka.strimzi.io/v1" + jq -c '.manifest |= gsub("apiVersion: kafka.strimzi.io/v1beta2\n"; "apiVersion: kafka.strimzi.io/v1\n")' "$work/release.json" | + gzip -c | base64 | tr -d '\n' | base64 | tr -d '\n' > "$work/release.b64" + jq -n --rawfile r "$work/release.b64" '{data: {release: $r}}' > "$work/patch.json" + kubectl -n "$ns" patch secret "$secret" --type merge --patch-file "$work/patch.json" + done + done < "$work/releases" + } + +# The objects keep managedFields entries recorded under the old API versions, +# and once the CRDs serve only v1 the API server refuses every server-side +# apply to such an object ("request to convert CR to an invalid group/version", +# kubernetes/kubernetes#111937). helm-controller recovers by wiping the +# entries, and the fields Helm owned pass to "before-first-apply", so a field a +# later chart drops is never removed. Rewrite the entries to v1 while both +# versions are still served; the conversion is None, so field paths are +# spelled the same under either, and an entry for a field v1 dropped only +# names a field that no longer exists. +# +# The patch replaces the whole managedFields list, so a patch built from a +# stale read could drop an entry a concurrent writer added in between. The +# patch carries the resourceVersion it was built from: the API server then +# answers a concurrent change with a conflict, and the object is read again. +stale_fields='[.metadata.managedFields[]? | select(.apiVersion | test("/v1(alpha1|beta1|beta2)$"))] | length > 0' + +# list_stale_managed : the objects of that kind with entries to rewrite. +list_stale_managed() { + kubectl get "$1" --all-namespaces --show-managed-fields -o json | + jq -r ".items[] | select($stale_fields) | \"\\(.metadata.namespace) \\(.metadata.name)\"" > "$work/managed" + } + +rewrite_managed() { + for c in $STRIMZI_CRDS; do + list_stale_managed "$c" + while read -r ns name; do + echo "Moving the managedFields of $c $ns/$name to v1" + attempt=0 + while :; do + kubectl -n "$ns" get "$c" "$name" --show-managed-fields -o json > "$work/object.json" + if ! jq -e "$stale_fields" "$work/object.json" > /dev/null; then + break + fi + jq '{metadata: {resourceVersion: .metadata.resourceVersion, managedFields: (.metadata.managedFields | map(.apiVersion |= sub("/v1(alpha1|beta1|beta2)$"; "/v1")))}}' \ + "$work/object.json" > "$work/managed-patch.json" + if kubectl -n "$ns" patch "$c" "$name" --type merge --patch-file "$work/managed-patch.json" 2> "$work/patch.err"; then + break + fi + attempt=$((attempt + 1)) + if ! grep -q 'the object has been modified' "$work/patch.err" || [ "$attempt" -ge 5 ]; then + cat "$work/patch.err" >&2 + exit 1 + fi + done + done < "$work/managed" + done + } + +scan_releases rewrite +rewrite_managed + +# A Helm action already running when the releases were rewritten keeps the +# releases it read in memory and stores them when it finishes: its own +# revision, and the previous one as superseded, both naming v1beta2 again. +# Scan once more and refuse to stamp the version while a revision is still +# pending or names v1beta2: the hook then fails and runs the migration again. +scan_releases check > "$work/leftover" +for c in $STRIMZI_CRDS; do + list_stale_managed "$c" + sed "s|^| managedFields of $c |" "$work/managed" >> "$work/leftover" +done +if [ -s "$work/leftover" ]; then + echo "ERROR: these still name a removed Strimzi API version, or a Helm action on them has not finished:" >&2 + cat "$work/leftover" >&2 + echo "A pending release clears once helm-controller finishes or unlocks that HelmRelease action." >&2 + exit 1 +fi + +stamp_cozystack_version 62 diff --git a/packages/core/platform/values.yaml b/packages/core/platform/values.yaml index 8ea4a054a..bb690233c 100644 --- a/packages/core/platform/values.yaml +++ b/packages/core/platform/values.yaml @@ -14,7 +14,7 @@ migrations: # refuses to advance past a migration file missing from the image (exit 1), so # a stale pin fails loudly rather than silently skipping the etcd adoption. image: ghcr.io/cozystack/cozystack/platform-migrations:v1.6.0@sha256:6777042e3e9c6bad76e7d96fb64baa115061975f39061cb3ede84e21d0e2213f - targetVersion: 61 + targetVersion: 62 # Adopt legacy etcd clusters onto the v1alpha2 operator WITHOUT the mandatory # pre-adoption safety snapshot (migration 50). Leave false unless migration 50 # has actually refused to proceed and you accept the risk: adoption rewires From 2cd41b411f94752bc8c3e44a01fcb9e076caa142 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 13 Oct 2026 05:52:56 +0300 Subject: [PATCH 2/9] fix(backups): read the Strimzi Kafka CR through the v1 API Strimzi 1.x no longer serves kafka.strimzi.io/v1beta2, so the Kafka backup driver's readiness check would fail on every backup and restore. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- internal/backupcontroller/kafkatypes/types.go | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/internal/backupcontroller/kafkatypes/types.go b/internal/backupcontroller/kafkatypes/types.go index 9a7e36267..ad9f84633 100644 --- a/internal/backupcontroller/kafkatypes/types.go +++ b/internal/backupcontroller/kafkatypes/types.go @@ -1,6 +1,6 @@ // SPDX-License-Identifier: Apache-2.0 -// Package kafkatypes declares the minimum subset of the kafka.strimzi.io/v1beta2 +// Package kafkatypes declares the minimum subset of the kafka.strimzi.io/v1 // Kafka cluster CR the backupstrategy controller reads: the Ready condition, // used only as a readiness precondition before the Kafka metadata backup/restore // Job runs. Modelled as a partial (no spec) so we avoid pulling the full Strimzi @@ -8,7 +8,7 @@ // The driver never writes this CR. // // +groupName=kafka.strimzi.io -// +versionName=v1beta2 +// +versionName=v1 // +kubebuilder:object:generate=true package kafkatypes @@ -20,12 +20,11 @@ import ( const ( GroupName = "kafka.strimzi.io" - Version = "v1beta2" + Version = "v1" // ConditionTypeReady is the Strimzi cluster-operator condition that turns - // True once the Kafka cluster (brokers, and in ZooKeeper mode ZooKeeper) is - // up and serving. The Admin API the backup Job talks to is only reachable - // then. + // True once the Kafka cluster is up and serving. The Admin API the backup + // Job talks to is only reachable then. ConditionTypeReady = "Ready" ) From 3b65cebea341eea3f20cf394011b0ac2465b9223 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 13 Oct 2026 05:52:57 +0300 Subject: [PATCH 3/9] feat(kafka)!: run on the Strimzi v1 API and Kafka 4.x Strimzi 1.2 serves only the v1 API and runs Kafka 4.2 and 4.3. The node-pool hook no longer drives the ZooKeeper to KRaft migration or the drain that freed the shared "kafka" pool name: Strimzi 1.x has no ZooKeeper support, and the platform migration refuses to upgrade while a ZooKeeper cluster is left. A cluster migrated earlier keeps its "kafka" broker pool. Without the drain the hook takes seconds, so the seven-hour install timeout goes too. Kafka 4 rejects the topic configs it removed and a segment.bytes below 1 MiB, which Kafka 3.x accepted. A KafkaTopic that sets one stays NotReady while the release reports Ready, so the chart refuses to render such a topic. It reads segment.bytes the way Kafka does: trimmed, with an optional sign, leading zeros decimal. BREAKING CHANGE: the Kafka application accepts version v4.3 (default) and v4.2; v3.8 and v3.9 are gone, and the platform migration moves existing applications to v4.3. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- api/apps/v1alpha1/kafka/types.go | 8 +- docs/agents/e2e-testing.md | 4 +- examples/backups/kafka-metadata/00-helpers.sh | 2 +- .../backups/kafka-metadata/05-kafka-src.yaml | 4 - .../kafka-metadata/20-kafka-target.yaml | 4 - examples/backups/kafka/00-helpers.sh | 10 +- examples/backups/kafka/01-create-strategy.sh | 4 +- examples/backups/kafka/cleanup.sh | 2 +- .../kafka-metadata/chainsaw-test.yaml | 8 +- hack/e2e-chainsaw/kafka/chainsaw-test.yaml | 32 +- hack/e2e-chainsaw/kafka/kafka.yaml | 2 +- hack/kafka-migration-hook.bats | 387 ++--------------- hack/kafka-pre-delete-hook.bats | 4 +- hack/kafka-topic-backup-strategy.bats | 9 +- packages/apps/kafka/Chart.yaml | 2 +- packages/apps/kafka/README.md | 70 ++-- packages/apps/kafka/files/versions.yaml | 4 +- packages/apps/kafka/templates/_versions.tpl | 97 +---- packages/apps/kafka/templates/kafka.yaml | 12 +- .../apps/kafka/templates/migration-hook.yaml | 394 +----------------- packages/apps/kafka/templates/topics.yaml | 35 +- .../apps/kafka/tests/jvm_sizing_test.yaml | 16 - packages/apps/kafka/tests/kafka_test.yaml | 41 +- .../apps/kafka/tests/migration-hook_test.yaml | 367 +++++----------- packages/apps/kafka/tests/topics_test.yaml | 186 +++++++++ packages/apps/kafka/values.schema.json | 10 +- packages/apps/kafka/values.yaml | 10 +- packages/system/kafka-rd/cozyrds/kafka.yaml | 13 +- 28 files changed, 500 insertions(+), 1237 deletions(-) diff --git a/api/apps/v1alpha1/kafka/types.go b/api/apps/v1alpha1/kafka/types.go index a69cece1b..16e4ce04e 100644 --- a/api/apps/v1alpha1/kafka/types.go +++ b/api/apps/v1alpha1/kafka/types.go @@ -24,8 +24,8 @@ type ConfigSpec struct { // TLS configuration. Strimzi manages the cluster PKI automatically (no cert-manager is involved for this chart): the operator auto-creates `-cluster-ca-cert` and `-clients-ca-cert` secrets, both exposed for client trust setup. The internal TLS listener on 9093 is always on; this toggle only controls the external listener on 9094. // +kubebuilder:default:={} Tls TLS `json:"tls"` - // Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v3.9 back to v3.8) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it. - // +kubebuilder:default:="v3.9" + // Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v4.3 back to v4.2) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it. + // +kubebuilder:default:="v4.3" Version Version `json:"version"` // Topics configuration. // +kubebuilder:default:={} @@ -36,7 +36,7 @@ type ConfigSpec struct { } type Kafka struct { - // Persistent Volume size for KRaft controller metadata (used during ZK-to-KRaft migration). + // Persistent Volume size for KRaft controller metadata. // +kubebuilder:default:="5Gi" ControllerStorageSize resource.Quantity `json:"controllerStorageSize,omitempty"` // Number of Kafka replicas. @@ -83,5 +83,5 @@ type Topic struct { // +kubebuilder:validation:Enum="t1.nano";"t1.micro";"t1.small";"t1.medium";"t1.large";"t1.xlarge";"t1.2xlarge";"t1.4xlarge";"c1.nano";"c1.micro";"c1.small";"c1.medium";"c1.large";"c1.xlarge";"c1.2xlarge";"c1.4xlarge";"s1.nano";"s1.micro";"s1.small";"s1.medium";"s1.large";"s1.xlarge";"s1.2xlarge";"s1.4xlarge";"u1.nano";"u1.micro";"u1.small";"u1.medium";"u1.large";"u1.xlarge";"u1.2xlarge";"u1.4xlarge";"m1.nano";"m1.micro";"m1.small";"m1.medium";"m1.large";"m1.xlarge";"m1.2xlarge";"m1.4xlarge";"nano";"micro";"small";"medium";"large";"xlarge";"2xlarge" type ResourcesPreset string -// +kubebuilder:validation:Enum="v3.9";"v3.8" +// +kubebuilder:validation:Enum="v4.3";"v4.2" type Version string diff --git a/docs/agents/e2e-testing.md b/docs/agents/e2e-testing.md index c5bcd091e..f5c793125 100644 --- a/docs/agents/e2e-testing.md +++ b/docs/agents/e2e-testing.md @@ -205,7 +205,7 @@ The suite is pinned to Chainsaw **v0.2.15** (the latest release as of May 2026); 3. Imperative-only waits live in a `script` step; any bare `sleep` carries a `TODO(e2e-replace-fixed-timeouts):` justification, unless it is a measurement interval, which §2 carves out by name and which takes no marker because no event can replace it. 4. No test-level `EXIT`/`RETURN` trap — rely on Chainsaw cleanup; a self-contained trap is allowed inside a single `script` step (port-forward / temp dir), and inside an explicit subshell within a BATS `@test`, where it does not displace the runner's own handler. A BATS file that still carries test-level traps declares the count itself in `# EXIT-TRAP DEBT: N`; do not ask for one to be removed without checking that line, because the count is exact in both directions. 5. Controller-created artifacts the test cannot reclaim are pruned explicitly; nested tenants delete child → parent with a wait-for-deletion between. -6. An HR-Ready assert has a floor: the release's own install timeout, not only the app's boot time. Below it a failed install reports as a bare assert timeout, and above it by less than a start margin it can, so **a number under the floor buys a shorter red run at the price of the diagnosis**. Whether to leave a suite under it is a per-suite call, made against what that suite's red runs cost, unless no budget inside the E2E job's cap can clear the floor at all, as with kafka's 7h install timeout; either way, a Chainsaw assert left under it says why in-line. Derive the floor from [Sizing an HR-Ready budget](#sizing-an-hr-ready-budget) rather than copying a neighbour's number, and justify a raise in-line. The same goes for any other wait longer than the ones around it, HR-Ready or not: it carries its reason in-line. +6. An HR-Ready assert has a floor: the release's own install timeout, not only the app's boot time. Below it a failed install reports as a bare assert timeout, and above it by less than a start margin it can, so **a number under the floor buys a shorter red run at the price of the diagnosis**. Whether to leave a suite under it is a per-suite call, made against what that suite's red runs cost; either way, a Chainsaw assert left under it says why in-line. Derive the floor from [Sizing an HR-Ready budget](#sizing-an-hr-ready-budget) rather than copying a neighbour's number, and justify a raise in-line. The same goes for any other wait longer than the ones around it, HR-Ready or not: it carries its reason in-line. 7. Failure path attaches scoped diagnostics via a `catch:` block, never a silent pass. The node-join deadline is no exception, and no longer has one: with the soft-red marker gone it fails its suite like any other deadline and its diagnostics run from the same `catch:`. 8. If it touches HelmRelease install/upgrade behavior, add the `status.history` guard: fail the run on a `failed` or `uninstalled` Snapshot of a release whose install strategy is `RetryOnFailure`, fail it on a parent whose strategy is anything else, and report the footprint of every other release without failing. What is reusable today is the predicate and the jsonpath in `hack/e2e-chainsaw/_lib/remediation-guard.sh`; the policy that weighs the footprint against `.spec.install.strategy.name`, judges an empty history against the release's own readiness, and keeps each read's exit status apart from its output lives in `cozy_guard_helmrelease` in `hack/e2e-chainsaw/_lib/run-kubernetes.sh` — read it before rebuilding any of that for another suite. If the suite has a failure path that exits before the guard, give that path `cozy_report_helmrelease_remediation` as well, or the runs the guard exists for are the runs it does not see. 9. A test that creates a tenant `Kubernetes` cluster waits for its release to reach `Ready` **before the assertions it ends on**, with a `--timeout` the guard can read as positive and no longer than a day. Reads earlier in the run are expected and are not what this bounds. The mechanism, how to size the budget and what the guard does with it are under "Do not mask cleanup or teardown failures". One thing to carry from there rather than look up: the naming is part of the rule, not a detail of the check, so an end-assertion added under any name outside the `cozy_assert_*` / `cozy_switch_and_assert_*` convention is invisible to it and passes before the wait without reddening anything — as long as one recognised call site is left behind it. Rename every recognised call and the guard reds instead, on finding none at all, which is the one shape of this mistake it does catch. @@ -245,7 +245,7 @@ A HelmRelease install waits through kstatus unless the ApplicationDefinition car The first floor is the app's own first boot. For a chart that sets `startupProbe` it is `initialDelaySeconds + (failureThreshold - 1) * periodSeconds`; for a chart that does not, it is whatever its operator needs, and only a measurement will say. mariadb is the case in this tree where this floor is real rather than nominal: its chart grants a 310s startup budget, so a wait that fails earlier fails a boot the probe was still willing to wait for. -The second floor is the release's own install timeout, and it is usually the binding one. Where to read it depends on who created the release, and getting this wrong under-budgets rather than over-budgets. For a release cozystack-api generated from an ApplicationDefinition the default is 10m (`pkg/cmd/server/start.go`), overridden per definition by `release.cozystack.io/helm-install-timeout` — `kubernetes-rd` at 20m, `kubernetes-nodes-rd`, `tenant-rd` and `monitoring-rd` at 15m, `kafka-rd` at 7h, a floor no suite clears inside the E2E job's 215-minute cap; the first two also disable the wait, so for `kubernetes` the override bounds its blocking post-install hook rather than a kstatus poll, and `kubernetes-nodes`, which renders no post-install hook, has nothing in its install for the override to bound. For a HelmRelease an app chart renders itself there is no ApplicationDefinition to read and that default never applies: the floor is the `spec.install.timeout` on the template that renders it, which takes precedence over its `spec.timeout`, and where a template sets neither, helm-controller's own documented default of 5m applies instead. There is deliberately no count of that population here. Read the value off the template you are waiting on: they range from 5m to 30m, they are spread across `packages/apps`, `packages/extra` and `packages/system`, and a release carrying `spec.kubeConfig` installs into a tenant cluster and is not one of these at all. A number in this sentence would be an obligation to revisit it whenever a template is added, and its three qualifiers are each easy to drop from a recursive grep. The nested `-system` releases the app charts wrap their workloads in are all of this second kind. A HelmRelease that cozystack-operator builds for a `Package` is a third kind: its `spec.install.timeout` comes from the operator's `--helmrelease-install-timeout`, 10m unless `cozystackOperator.helmReleaseInstallTimeout` in the installer chart overrides it. At or below whichever floor applies, a failed install is reported as a bare assert timeout, because the release has not yet recorded why it failed. +The second floor is the release's own install timeout, and it is usually the binding one. Where to read it depends on who created the release, and getting this wrong under-budgets rather than over-budgets. For a release cozystack-api generated from an ApplicationDefinition the default is 10m (`pkg/cmd/server/start.go`), overridden per definition by `release.cozystack.io/helm-install-timeout` — `kubernetes-rd` at 20m, `kubernetes-nodes-rd`, `tenant-rd` and `monitoring-rd` at 15m; the first two also disable the wait, so for `kubernetes` the override bounds its blocking post-install hook rather than a kstatus poll, and `kubernetes-nodes`, which renders no post-install hook, has nothing in its install for the override to bound. For a HelmRelease an app chart renders itself there is no ApplicationDefinition to read and that default never applies: the floor is the `spec.install.timeout` on the template that renders it, which takes precedence over its `spec.timeout`, and where a template sets neither, helm-controller's own documented default of 5m applies instead. There is deliberately no count of that population here. Read the value off the template you are waiting on: they range from 5m to 30m, they are spread across `packages/apps`, `packages/extra` and `packages/system`, and a release carrying `spec.kubeConfig` installs into a tenant cluster and is not one of these at all. A number in this sentence would be an obligation to revisit it whenever a template is added, and its three qualifiers are each easy to drop from a recursive grep. The nested `-system` releases the app charts wrap their workloads in are all of this second kind. A HelmRelease that cozystack-operator builds for a `Package` is a third kind: its `spec.install.timeout` comes from the operator's `--helmrelease-install-timeout`, 10m unless `cozystackOperator.helmReleaseInstallTimeout` in the installer chart overrides it. At or below whichever floor applies, a failed install is reported as a bare assert timeout, because the release has not yet recorded why it failed. What holds the install open is anything in the rendered manifests kstatus reports InProgress, which is wider than a `Ready` condition. kstatus carries built-in rules for a list of kinds — the workloads, but also `Service`, `Pod`, `PersistentVolumeClaim` and `CustomResourceDefinition` — and falls through to reading `Ready` only for kinds it has no rule for. Read that list rather than recalling it: `legacyTypes` in `pkg/kstatus/status/core.go` of `fluxcd/cli-utils`, at whatever version the helm-controller image in `internal/fluxinstall/manifests/fluxcd.yaml` was built against — nothing in this repository vendors it, so there is no local copy to open and the version has to come from that image. It does not divide the way intuition suggests, because `Secret`, `ConfigMap` and `batch/CronJob` are in the table and map to always-ready, so being in it is not the same as gating on it. The practical consequence is that "renders no CR publishing `Ready`" does not exempt a release, and `tcp-balancer` and `vpn` show why: each renders a `WorkloadMonitor`, whose status type declares no `conditions` at all, so kstatus falls past the `Ready` fallback and reads that CR as Current — and then gates the install on the Deployment beside it through a built-in rule. The CR is present and contributes nothing; the workload is what holds the install open. diff --git a/examples/backups/kafka-metadata/00-helpers.sh b/examples/backups/kafka-metadata/00-helpers.sh index 8d77fba64..08a44b8bc 100755 --- a/examples/backups/kafka-metadata/00-helpers.sh +++ b/examples/backups/kafka-metadata/00-helpers.sh @@ -63,7 +63,7 @@ export CA_MOUNT_DIR="${CA_MOUNT_DIR:-/etc/ssl/kafka-backup-ca}" # the target broker's own image at reconcile time and renders it as the # strategy's .ClientImage. Override this only to match your operator's image if # the CLI it carries differs. -export KAFKA_IMAGE="${KAFKA_IMAGE:-quay.io/strimzi/kafka:0.45.1-rc1-kafka-3.9.1@sha256:ba52ed046b1dccdbd96f4e68057ce014d862a7c9c1fc670760c023b9aa09f23f}" +export KAFKA_IMAGE="${KAFKA_IMAGE:-quay.io/strimzi/kafka:1.2.0-kafka-4.3.1@sha256:fef34b5438e8556cc08c01f3e254e47346f061b53a4e38d4289853777e0ea7f1}" export KAFKA_BIN="${KAFKA_BIN:-/opt/kafka/bin}" # Name of that long-lived CLI Pod; cleanup.sh removes it. export KAFKA_CLI_POD="${KAFKA_CLI_POD:-kafka-cli}" diff --git a/examples/backups/kafka-metadata/05-kafka-src.yaml b/examples/backups/kafka-metadata/05-kafka-src.yaml index 041a8dc26..b905e12c5 100644 --- a/examples/backups/kafka-metadata/05-kafka-src.yaml +++ b/examples/backups/kafka-metadata/05-kafka-src.yaml @@ -14,8 +14,4 @@ spec: replicas: 1 size: 2Gi resourcesPreset: "c1.small" - zookeeper: - replicas: 1 - size: 1Gi - resourcesPreset: "c1.small" topics: [] diff --git a/examples/backups/kafka-metadata/20-kafka-target.yaml b/examples/backups/kafka-metadata/20-kafka-target.yaml index 7642f81bf..ff7e30b84 100644 --- a/examples/backups/kafka-metadata/20-kafka-target.yaml +++ b/examples/backups/kafka-metadata/20-kafka-target.yaml @@ -12,8 +12,4 @@ spec: replicas: 1 size: 2Gi resourcesPreset: "c1.small" - zookeeper: - replicas: 1 - size: 1Gi - resourcesPreset: "c1.small" topics: [] diff --git a/examples/backups/kafka/00-helpers.sh b/examples/backups/kafka/00-helpers.sh index 09b03fd6c..5ae81170a 100755 --- a/examples/backups/kafka/00-helpers.sh +++ b/examples/backups/kafka/00-helpers.sh @@ -92,8 +92,8 @@ resolve_kafka_image() { # is being assumed. On an air-gapped or operator-less cluster - the persona # the README's override note is for - that literal is unreachable, so the # run must set KAFKA_IMAGE to an image the nodes can actually pull. - echo "! could not read STRIMZI_KAFKA_IMAGES from deploy/strimzi-cluster-operator in cozy-kafka-operator; assuming quay.io/strimzi/kafka:0.45.1-rc1-kafka-3.9.1. Set KAFKA_IMAGE explicitly if that image is wrong or unreachable (e.g. air-gapped)." >&2 - echo "quay.io/strimzi/kafka:0.45.1-rc1-kafka-3.9.1" + echo "! could not read STRIMZI_KAFKA_IMAGES from deploy/strimzi-cluster-operator in cozy-kafka-operator; assuming quay.io/strimzi/kafka:1.2.0-kafka-4.3.1. Set KAFKA_IMAGE explicitly if that image is wrong or unreachable (e.g. air-gapped)." >&2 + echo "quay.io/strimzi/kafka:1.2.0-kafka-4.3.1" } export KAFKA_IMAGE="${KAFKA_IMAGE:-$(resolve_kafka_image)}" export KAFKA_BIN="${KAFKA_BIN:-/opt/kafka/bin}" @@ -215,7 +215,7 @@ seed_topic() { printf "k-%s\torder-%s\n" "$i" "$i" i=$((i + 1)) done | "$BIN"/kafka-console-producer.sh --bootstrap-server "$BOOT" \ - --topic "$TOPIC" --property parse.key=true + --topic "$TOPIC" --reader-property parse.key=true ' } @@ -231,7 +231,7 @@ seed_decoy_topic() { printf "d-%s\tdecoy-%s\n" "$i" "$i" i=$((i + 1)) done | "$BIN"/kafka-console-producer.sh --bootstrap-server "$BOOT" \ - --topic "$DECOY_TOPIC" --property parse.key=true + --topic "$DECOY_TOPIC" --reader-property parse.key=true ' } @@ -287,7 +287,7 @@ topic_dump() { [ "${end:-0}" -gt 0 ] || continue "$BIN"/kafka-console-consumer.sh --bootstrap-server "$BOOT" --topic "$TOPIC" \ --partition "$p" --offset 0 --max-messages "$end" --timeout-ms 60000 \ - --property print.key=true --property print.timestamp=false \ + --formatter-property print.key=true --formatter-property print.timestamp=false \ | sed "s/^/$p\t/" done ' diff --git a/examples/backups/kafka/01-create-strategy.sh b/examples/backups/kafka/01-create-strategy.sh index 26b8d95d9..7c57c1f65 100755 --- a/examples/backups/kafka/01-create-strategy.sh +++ b/examples/backups/kafka/01-create-strategy.sh @@ -324,7 +324,7 @@ spec: if [ "\${n}" -gt 0 ]; then "\${BIN}"/kafka-console-consumer.sh --bootstrap-server "\${BOOT}" \ --topic "\${t}" --partition "\${p}" --offset "\${begin}" --max-messages "\${n}" \ - --timeout-ms 120000 --property print.key=true --property print.timestamp=false \ + --timeout-ms 120000 --formatter-property print.key=true --formatter-property print.timestamp=false \ > "\${WORK}/data-\${t}-\${p}.tsv" # kafka-console-consumer exits 0 even on a short read: a # --timeout-ms expiry is swallowed inside ConsoleConsumer, @@ -509,7 +509,7 @@ spec: [ "\${mt}" = "\${t}" ] || continue [ "\${me}" -gt "\${mb}" ] || continue "\${BIN}"/kafka-console-producer.sh --bootstrap-server "\${BOOT}" \ - --topic "\${t}" --property parse.key=true < "\${WORK}/data-\${t}-\${mp}.tsv" + --topic "\${t}" --reader-property parse.key=true < "\${WORK}/data-\${t}-\${mp}.tsv" done < "\${WORK}/manifest.txt" echo " \${t}: replayed" done diff --git a/examples/backups/kafka/cleanup.sh b/examples/backups/kafka/cleanup.sh index 0e3340a48..7f06bab82 100755 --- a/examples/backups/kafka/cleanup.sh +++ b/examples/backups/kafka/cleanup.sh @@ -33,7 +33,7 @@ kubectl delete backupclass.backups.cozystack.io "$BACKUPCLASS_NAME" --ignore-not kubectl delete job.strategy.backups.cozystack.io "$STRATEGY_NAME" --ignore-not-found # `kubectl delete` returns as soon as deletionTimestamp is set, while the -# HelmRelease uninstall behind each Kafka app (two StatefulSets, ZooKeeper and +# HelmRelease uninstall behind each Kafka app (its broker and controller pods and # their PVCs) is still draining. Returning here lets the next suite start into a # namespace whose predecessor is still uninstalling - the neighbouring # chainsaw/kafka Test's PVC-reclaim step contends with exactly that. Wait for diff --git a/hack/e2e-chainsaw/kafka-metadata/chainsaw-test.yaml b/hack/e2e-chainsaw/kafka-metadata/chainsaw-test.yaml index c8b559e97..076015dc2 100644 --- a/hack/e2e-chainsaw/kafka-metadata/chainsaw-test.yaml +++ b/hack/e2e-chainsaw/kafka-metadata/chainsaw-test.yaml @@ -39,7 +39,7 @@ spec: kind: HelmRelease name: kafka-kafka-meta-src - describe: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: Kafka name: kafka-kafka-meta-src - describe: @@ -72,7 +72,7 @@ spec: check: ($error == null): true # 40m budget: two single-broker Strimzi clusters (source + to-copy target, - # each boots ZooKeeper then the broker and passes readiness), the metadata + # each boots its controller and broker and passes readiness), the metadata # backup, and the in-place + to-copy restores with their verifications. # run-all.sh fails fast on terminal BackupJob/RestoreJob phases and stalled # HelmReleases rather than polling out the budget. @@ -119,7 +119,7 @@ spec: - error: timeout: 12m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: @@ -127,7 +127,7 @@ spec: - error: timeout: 12m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: diff --git a/hack/e2e-chainsaw/kafka/chainsaw-test.yaml b/hack/e2e-chainsaw/kafka/chainsaw-test.yaml index dfeb9da2e..9f0edbbe7 100644 --- a/hack/e2e-chainsaw/kafka/chainsaw-test.yaml +++ b/hack/e2e-chainsaw/kafka/chainsaw-test.yaml @@ -3,7 +3,7 @@ # The chart deploys Kafka in KRaft mode (no ZooKeeper): a combined-free layout # of separate broker + controller KafkaNodePools. This test asserts the KRaft # topology renders and converges (both node pools present, the Strimzi Kafka CR -# reaches kafkaMetadataState=KRaft) and that no ZooKeeper resources remain. +# is Ready) and that no ZooKeeper resources remain. # The last step deletes the app itself and checks that no node pool or Strimzi # pod of this cluster survives the uninstall. apiVersion: chainsaw.kyverno.io/v1alpha1 @@ -46,9 +46,9 @@ spec: - name: wait-ready try: - # Deliberately under the floor: kafka-rd sets a 7h install timeout, which - # no budget inside the E2E job's cap can clear, so a failed install here - # reports as a bare assert timeout and the catch block's describe of the + # Deliberately under the floor, kafka-rd's default 10m install timeout: a + # healthy cluster is Ready in a few minutes, so a failed install ends the + # run early as a bare assert timeout, and the catch block's describe of the # HelmRelease is where its state is read. - assert: timeout: 6m @@ -73,7 +73,7 @@ spec: - assert: timeout: 5m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaTopic metadata: name: kafka-test-test-results @@ -91,7 +91,7 @@ spec: - assert: timeout: 2m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: @@ -102,7 +102,7 @@ spec: - assert: timeout: 2m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: @@ -113,15 +113,13 @@ spec: - assert: timeout: 5m resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: Kafka metadata: name: kafka-test - annotations: - strimzi.io/kraft: enabled - strimzi.io/node-pools: enabled status: - kafkaMetadataState: KRaft + (conditions[?type == 'Ready']): + - status: "True" # KRaft mode ships no ZooKeeper: the client Service must never exist. - error: resource: @@ -161,7 +159,7 @@ spec: # period, which the default error timeout covers. - error: resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: @@ -227,11 +225,11 @@ spec: kind: HelmRelease name: kafka-kafka-test - describe: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: Kafka name: kafka-kafka-test - describe: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: Kafka name: kafka-kafka-restore - describe: @@ -341,14 +339,14 @@ spec: try: - error: resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: strimzi.io/cluster: kafka-kafka-test - error: resource: - apiVersion: kafka.strimzi.io/v1beta2 + apiVersion: kafka.strimzi.io/v1 kind: KafkaNodePool metadata: labels: diff --git a/hack/e2e-chainsaw/kafka/kafka.yaml b/hack/e2e-chainsaw/kafka/kafka.yaml index 9f1e25f14..3c107f27b 100644 --- a/hack/e2e-chainsaw/kafka/kafka.yaml +++ b/hack/e2e-chainsaw/kafka/kafka.yaml @@ -3,7 +3,7 @@ kind: Kafka metadata: name: test spec: - version: "v3.9" + version: "v4.3" external: false kafka: size: 10Gi diff --git a/hack/kafka-migration-hook.bats b/hack/kafka-migration-hook.bats index 65925385f..8ba753e75 100644 --- a/hack/kafka-migration-hook.bats +++ b/hack/kafka-migration-hook.bats @@ -1,19 +1,10 @@ #!/usr/bin/env bats -# Behavioural tests for the ZooKeeper->KRaft migration hook of -# packages/apps/kafka. The shell script is extracted from the rendered Job and -# executed against a stub kubectl, so the fail-closed reads, the atomic -# namespace-collision guard, the node-pool creation, the finalize guard and the -# "free the kafka name" drain are exercised for real. The helm-unittest cases in +# Behavioural tests for the node-pool hook of packages/apps/kafka. The shell +# script is extracted from the rendered Job and executed against a stub +# kubectl, so what the hook actually applies, and whether a failed apply fails +# the Helm upgrade, are exercised for real. The helm-unittest cases in # packages/apps/kafka/tests/migration-hook_test.yaml only match the script's -# source TEXT; they cannot say whether a stuck migration refuses to stamp -# kraft=enabled, or whether the drain drives the rebalance to completion before -# deleting the pool. -# -# The hook manages the node pools itself (no regular KafkaNodePool manifest), so -# it runs on install AND upgrade. `helm template` (no live lookup) resolves the -# broker pool to the fresh "b-" name; tests that must exercise the -# migration "kafka" pool (collision guard, create-CAS, name-freeing) force the -# resolved value with force_kafka_pool. +# source TEXT. # # Run from the repo root with make bats-unit-tests. Direct assertions and # inline cleanup retain compatibility with cozytest. Its translator rewrites @@ -30,87 +21,21 @@ render_script() { --show-only templates/migration-hook.yaml 2>"$1.err" | yq 'select(.kind == "Job") | .spec.template.spec.containers[0].args[0]' - > "$1" if [ ! -s "$1" ]; then - echo "FAIL: the chart rendered no migration Job script" >&2 + echo "FAIL: the chart rendered no node-pool Job script" >&2 grep -v 'found symbolic link in path' "$1.err" >&2 || true return 1 fi } -# Force the render-resolved broker pool from "b-" to "kafka", so the -# collision / create-CAS / name-freeing branches (gated on brokerPool == "kafka") -# run. `helm template` cannot seed the live lookup that resolves it. -force_kafka_pool() { - sed 's/brokerPool="b-[0-9a-f]\{8\}"/brokerPool="kafka"/' "$1" > "$1.k" && mv "$1.k" "$1" - } - make_bin() { mkdir -p "$1" - printf '#!/bin/sh\nexit 0\n' > "$1/sleep" - chmod 0755 "$1/sleep" cat > "$1/kubectl" <<'STUB' #!/bin/sh -args="$*" -echo "$args" >> "${KUBECTL_LOG:-/dev/null}" - -# Order matters: match the more specific jsonpath reads before the generic -# owner/kraft/found reads that share substrings. -case "$args" in - *"create -f -"*) - cat >> "${KUBECTL_LOG:-/dev/null}" - [ -n "${STUB_CREATE_FAIL:-}" ] && { echo "AlreadyExists" >&2; exit 1; } - exit 0 - ;; +echo "$*" >> "${KUBECTL_LOG:-/dev/null}" +case "$*" in *"apply "*"-f -"*) cat >> "${KUBECTL_LOG:-/dev/null}" - exit 0 - ;; - *"range .items"*) - # kafkas list for the "other ZooKeeper clusters remain" count. - [ -n "${STUB_OTHERS_LIST_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } - printf '%s\n' "${STUB_OTHERS_LIST:-}" - exit 0 - ;; - *nodeIds*) - printf '%s' "${STUB_KAFKA_IDS:-}" - exit 0 - ;; - *].reason*) - # KafkaRebalance NotReady reason (retriable vs terminal). - printf '%s' "${STUB_REBALANCE_REASON:-}" - exit 0 - ;; - *conditions*) - # KafkaRebalance status.conditions[*].type, advanced via STUB_REBALANCE. - i=$(cat "${REBALANCE_COUNTER}" 2>/dev/null) - [ -n "$i" ] || i=0 - i=$((i + 1)) - echo "$i" > "${REBALANCE_COUNTER}" - printf '%s\n' "${STUB_REBALANCE:-}" | awk -v n="$i" '{a[NR]=$0} END{print (n<=NR)?a[n]:a[NR]}' - exit 0 - ;; - *kafkaMetadataState*) - [ -n "${STUB_STATE_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } - i=$(cat "${STATE_COUNTER}" 2>/dev/null) - [ -n "$i" ] || i=0 - i=$((i + 1)) - echo "$i" > "${STATE_COUNTER}" - printf '%s\n' "${STUB_STATES:-}" | awk -v n="$i" '{a[NR]=$0} END{print (n<=NR)?a[n]:a[NR]}' - exit 0 - ;; - *cluster}*) - [ -n "${STUB_OWNER_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } - printf '%s\n' "${STUB_OWNER:-}" - exit 0 - ;; - *kraft}*) - [ -n "${STUB_KRAFT_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } - printf '%s\n' "${STUB_KRAFT:-}" - exit 0 - ;; - *--ignore-not-found*) - [ -n "${STUB_FOUND_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } - printf '%s' "${STUB_FOUND:-}" - [ -n "${STUB_FOUND:-}" ] && echo + [ -n "${STUB_APPLY_FAIL:-}" ] && { echo "apiserver error" >&2; exit 1; } exit 0 ;; esac @@ -128,38 +53,26 @@ setup_case() { run_hook() { rc=0 : > "$tmp/kubectl.log" - : > "$tmp/state.counter" - : > "$tmp/rebalance.counter" KUBECONFIG=/dev/null \ KUBECTL_LOG="$tmp/kubectl.log" \ - STATE_COUNTER="$tmp/state.counter" \ - REBALANCE_COUNTER="$tmp/rebalance.counter" \ - STUB_FOUND="${STUB_FOUND:-}" \ - STUB_FOUND_FAIL="${STUB_FOUND_FAIL:-}" \ - STUB_KRAFT="${STUB_KRAFT:-}" \ - STUB_STATES="${STUB_STATES:-}" \ - STUB_OWNER="${STUB_OWNER:-}" \ - STUB_OWNER_FAIL="${STUB_OWNER_FAIL:-}" \ - STUB_KRAFT_FAIL="${STUB_KRAFT_FAIL:-}" \ - STUB_STATE_FAIL="${STUB_STATE_FAIL:-}" \ - STUB_CREATE_FAIL="${STUB_CREATE_FAIL:-}" \ - STUB_OTHERS_LIST="${STUB_OTHERS_LIST:-}" \ - STUB_OTHERS_LIST_FAIL="${STUB_OTHERS_LIST_FAIL:-}" \ - STUB_KAFKA_IDS="${STUB_KAFKA_IDS:-}" \ - STUB_REBALANCE="${STUB_REBALANCE:-}" \ - STUB_REBALANCE_REASON="${STUB_REBALANCE_REASON:-}" \ + STUB_APPLY_FAIL="${STUB_APPLY_FAIL:-}" \ PATH="$tmp/bin:$PATH" sh "$tmp/s.sh" > "$tmp/out" 2>&1 || rc=$? [ -s "$tmp/kubectl.log" ] || return 1 } -@test "a NotFound Kafka CR is a fresh install: pools ensured, no migration" { +@test "the hook applies the broker and the controller pool through the v1 API" { setup_case - STUB_FOUND="" run_hook + run_hook [ "$rc" = 0 ] - grep -qF 'fresh install' "$tmp/out" - if grep -qE 'kraft=(migration|enabled)|node-pools=enabled' "$tmp/kubectl.log"; then - echo "FAIL: the fresh-install path drove the migration state machine" + applies=$(grep -c 'apply --openapi-patch=false -f -' "$tmp/kubectl.log") + [ "$applies" = 2 ] + pools=$(grep -c '^apiVersion: kafka.strimzi.io/v1$' "$tmp/kubectl.log") + [ "$pools" = 2 ] + grep -qE '^ *- broker$' "$tmp/kubectl.log" + grep -qE '^ *- controller$' "$tmp/kubectl.log" + if grep -qE 'v1beta2|strimzi.io/kraft|KafkaRebalance|(^| )(annotate|delete|patch|create) ' "$tmp/kubectl.log"; then + echo "FAIL: the hook did more than apply the two pools" cat "$tmp/kubectl.log" return 1 fi @@ -167,267 +80,17 @@ run_hook() { rm -rf "$tmp" } -@test "a read error reading the Kafka CR fails closed, never mistaken for fresh" { +@test "a failed pool apply fails the hook" { setup_case - STUB_FOUND_FAIL=1 run_hook + STUB_APPLY_FAIL=1 run_hook [ "$rc" != 0 ] - grep -qF 'reading Kafka CR failed' "$tmp/out" - if grep -qF 'fresh install' "$tmp/out"; then - echo "FAIL: an apiserver error was read as a fresh install" - cat "$tmp/out" - return 1 - fi - - rm -rf "$tmp" -} - -@test "an already-KRaft cluster on b- is a no-op with nothing to free" { - setup_case - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" run_hook - - [ "$rc" = 0 ] - grep -qF 'already KRaft' "$tmp/out" - grep -qF 'nothing to free' "$tmp/out" - if grep -qE 'kraft=(migration|enabled)|node-pools=enabled' "$tmp/kubectl.log"; then - echo "FAIL: an already-KRaft cluster was driven through the state machine" + applies=$(grep -c 'apply --openapi-patch=false -f -' "$tmp/kubectl.log") + if [ "$applies" != 1 ]; then + echo "FAIL: the hook went on after the broker pool apply failed ($applies applies)" cat "$tmp/kubectl.log" return 1 fi rm -rf "$tmp" } - -@test "a kafka pool owned by another cluster fails fast before any mutation" { - setup_case - force_kafka_pool "$tmp/s.sh" - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="" \ - STUB_STATES="ZooKeeper" STUB_OWNER="other-kafka" run_hook - - [ "$rc" != 0 ] - grep -qF 'already belongs to cluster' "$tmp/out" - if grep -qE 'apply .*-f -|create -f -|kraft=|node-pools=' "$tmp/kubectl.log"; then - echo "FAIL: a foreign-owned kafka pool did not abort before mutating" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a failed read of the kafka pool owner fails closed" { - setup_case - force_kafka_pool "$tmp/s.sh" - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="" \ - STUB_STATES="ZooKeeper" STUB_OWNER_FAIL=1 run_hook - - [ "$rc" != 0 ] - grep -qF 'owner label failed' "$tmp/out" - if grep -qE 'apply .*-f -|create -f -|kraft=|node-pools=' "$tmp/kubectl.log"; then - echo "FAIL: proceeded to mutate the cluster after a failed owner read" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a failed read of the kraft annotation fails closed" { - setup_case - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT_FAIL=1 run_hook - - [ "$rc" != 0 ] - grep -qF 'strimzi.io/kraft annotation failed' "$tmp/out" - if grep -qE 'kraft=(migration|enabled)|node-pools=enabled' "$tmp/kubectl.log"; then - echo "FAIL: drove the migration after a failed kraft-annotation read" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a classic ZooKeeper cluster migrates and finalizes to KRaft" { - setup_case - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="" STUB_OWNER="" \ - STUB_STATES="ZooKeeper -KRaftPostMigration -KRaft" run_hook - - [ "$rc" = 0 ] - grep -qF 'Migration complete' "$tmp/out" - grep -qF 'node-pools=enabled' "$tmp/kubectl.log" - grep -qF 'kraft=migration' "$tmp/kubectl.log" - grep -qF 'kraft=enabled' "$tmp/kubectl.log" - - rm -rf "$tmp" -} - -@test "a migration stuck before KRaftPostMigration refuses to stamp kraft=enabled" { - setup_case - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="" STUB_OWNER="" \ - STUB_STATES="ZooKeeper" run_hook - - [ "$rc" != 0 ] - grep -qF 'Refusing to apply' "$tmp/out" - if grep -qF 'kraft=enabled' "$tmp/kubectl.log"; then - echo "FAIL: kraft=enabled was stamped while the cluster was still on ZooKeeper" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a retry already past the ZooKeeper phase does not re-issue kraft=migration" { - setup_case - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="" STUB_OWNER="" \ - STUB_STATES="KRaftPostMigration -KRaftPostMigration -KRaft" run_hook - - [ "$rc" = 0 ] - grep -qF 'not re-issuing kraft=migration' "$tmp/out" - grep -qF 'kraft=enabled' "$tmp/kubectl.log" - if grep -qF 'kraft=migration' "$tmp/kubectl.log"; then - echo "FAIL: kraft=migration was re-issued from a post-ZooKeeper state" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -# --- Name-freeing (multi Kafka per namespace) ------------------------------- - -@test "the sole migrated cluster keeps the kafka pool, no drain" { - setup_case - force_kafka_pool "$tmp/s.sh" - # Already KRaft on kafka; no other ZooKeeper clusters in the namespace. - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST="test-kafka=enabled" run_hook - - [ "$rc" = 0 ] - grep -qF 'keeping the "kafka" pool' "$tmp/out" - # No drain: no rebalance, no cruiseControl, no pool deletion. - if grep -qE 'KafkaRebalance|cruiseControl|delete .*kafkanodepools' "$tmp/kubectl.log"; then - echo "FAIL: a sole cluster ran the name-freeing drain" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a failed read of the other-clusters list fails closed, not mistaken for none" { - setup_case - force_kafka_pool "$tmp/s.sh" - # Already KRaft on kafka; the list read that decides whether others wait errors. - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST_FAIL=1 run_hook - - [ "$rc" != 0 ] - grep -qF 'could not list Kafka CRs' "$tmp/out" - if grep -qF 'keeping the "kafka" pool' "$tmp/out"; then - echo "FAIL: a failed list read was mistaken for 'no other clusters' and kept the pool" - cat "$tmp/out" - return 1 - fi - - rm -rf "$tmp" -} - -@test "a migrated cluster with others waiting drains and frees the kafka name" { - setup_case - force_kafka_pool "$tmp/s.sh" - # Already KRaft on kafka; one other cluster still on ZooKeeper. - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST="test-kafka=enabled -other-zk=" \ - STUB_KAFKA_IDS="[0,1,2]" \ - STUB_REBALANCE="PendingProposal -ProposalReady -Ready" run_hook - - [ "$rc" = 0 ] - grep -qF 'freeing the "kafka" pool name' "$tmp/out" - grep -qF 'Freed the "kafka" pool name' "$tmp/out" - # Full drain sequence, in order: target pool, Cruise Control, rebalance, - # approval, delete kafka pool + PVCs. - grep -qF '1000-1099' "$tmp/kubectl.log" # applied the b- target pool - grep -qF 'cruiseControl' "$tmp/kubectl.log" - grep -qF 'rebalance=approve' "$tmp/kubectl.log" - grep -qF 'delete kafkanodepools.kafka.strimzi.io kafka' "$tmp/kubectl.log" - grep -qF 'delete pvc data-0-test-kafka-kafka-0' "$tmp/kubectl.log" - grep -qF 'delete pvc data-0-test-kafka-kafka-2' "$tmp/kubectl.log" - - rm -rf "$tmp" -} - -@test "the drain refuses to delete the kafka pool if the rebalance never completes" { - setup_case - force_kafka_pool "$tmp/s.sh" - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST="test-kafka=enabled -other-zk=" \ - STUB_KAFKA_IDS="[0,1,2]" \ - STUB_REBALANCE="PendingProposal" run_hook - - [ "$rc" != 0 ] - grep -qF 'will resume on the next retry' "$tmp/out" - # Must NOT delete the pool (which still holds all the data) before the drain. - if grep -qF 'delete kafkanodepools.kafka.strimzi.io kafka' "$tmp/kubectl.log"; then - echo "FAIL: deleted the kafka pool before the rebalance finished — data loss" - cat "$tmp/kubectl.log" - return 1 - fi - - rm -rf "$tmp" -} - -# Cruise Control rejects proposals transiently while it settles — the pod/cert -# are still coming up (retriable connect), or the capacity model over-estimates -# load from noisy samples and trips a goal. The drain must refresh the proposal -# and keep polling, never treat a NotReady as terminal. Covers both reasons. -@test "the drain refreshes a transient-connection NotReady and proceeds" { - setup_case - force_kafka_pool "$tmp/s.sh" - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST="test-kafka=enabled -other-zk=" \ - STUB_KAFKA_IDS="[0,1,2]" \ - STUB_REBALANCE_REASON="CruiseControlRetriableConnectionException" \ - STUB_REBALANCE="PendingProposal -NotReady -ProposalReady -Ready" run_hook - - [ "$rc" = 0 ] - grep -qF 'refreshing the proposal' "$tmp/out" - grep -qF 'rebalance=refresh' "$tmp/kubectl.log" - grep -qF 'Freed the "kafka" pool name' "$tmp/out" - grep -qF 'delete kafkanodepools.kafka.strimzi.io kafka' "$tmp/kubectl.log" - - rm -rf "$tmp" -} - -@test "the drain refreshes and recovers from a capacity-goal NotReady" { - setup_case - force_kafka_pool "$tmp/s.sh" - STUB_FOUND="kafka.kafka.strimzi.io/test-kafka" STUB_KRAFT="enabled" \ - STUB_OTHERS_LIST="test-kafka=enabled -other-zk=" \ - STUB_KAFKA_IDS="[0,1,2]" \ - STUB_REBALANCE_REASON="OptimizationFailureException" \ - STUB_REBALANCE="PendingProposal -NotReady -ProposalReady -Ready" run_hook - - [ "$rc" = 0 ] - grep -qF 'refreshing the proposal' "$tmp/out" - grep -qF 'rebalance=refresh' "$tmp/kubectl.log" - grep -qF 'Freed the "kafka" pool name' "$tmp/out" - - rm -rf "$tmp" -} diff --git a/hack/kafka-pre-delete-hook.bats b/hack/kafka-pre-delete-hook.bats index 0ab408294..e8af58aec 100644 --- a/hack/kafka-pre-delete-hook.bats +++ b/hack/kafka-pre-delete-hook.bats @@ -135,7 +135,7 @@ KEOF # gone. Failing here would wedge the uninstall over nothing to clean up. Both # RESTMapper wordings are covered because kubectl picks between them. for msg in 'error: the server doesn'"'"'t have a resource type "kafkatopics"' \ - 'error: no matches for kind "KafkaTopic" in version "kafka.strimzi.io/v1beta2"'; do + 'error: no matches for kind "KafkaTopic" in version "kafka.strimzi.io/v1"'; do tmp=$(mktemp -d) write_fake_kubectl "$tmp" render_hook "$KAFKA_CHART" kafka-test tenant-test kafka-test-pre-delete "$tmp/hook.sh" @@ -289,7 +289,7 @@ KEOF @test "kafka pre-delete hook exits 0 when the KafkaNodePool CRD is absent" { for msg in 'error: the server doesn'"'"'t have a resource type "kafkanodepools"' \ - 'error: no matches for kind "KafkaNodePool" in version "kafka.strimzi.io/v1beta2"'; do + 'error: no matches for kind "KafkaNodePool" in version "kafka.strimzi.io/v1"'; do tmp=$(mktemp -d) write_fake_kubectl "$tmp" render_hook "$KAFKA_CHART" kafka-test tenant-test kafka-test-pre-delete "$tmp/hook.sh" diff --git a/hack/kafka-topic-backup-strategy.bats b/hack/kafka-topic-backup-strategy.bats index ce233e0d8..57830d8a7 100644 --- a/hack/kafka-topic-backup-strategy.bats +++ b/hack/kafka-topic-backup-strategy.bats @@ -116,7 +116,9 @@ topic=""; p=""; off=0; max=0 while [ $# -gt 0 ]; do case "$1" in --topic) topic="$2"; shift ;; --partition) p="$2"; shift ;; --offset) off="$2"; shift ;; - --max-messages) max="$2"; shift ;; --bootstrap-server|--timeout-ms|--property) shift ;; + --max-messages) max="$2"; shift ;; --bootstrap-server|--timeout-ms|--formatter-property) shift ;; + # Kafka 4 prints this on stdout, ahead of the records. + --property) echo "Option --property is deprecated and will be removed in a future version. Use --formatter-property instead."; shift ;; esac shift done @@ -131,7 +133,10 @@ STUB set -eu topic="" while [ $# -gt 0 ]; do - case "$1" in --topic) topic="$2"; shift ;; --bootstrap-server|--property) shift ;; esac + case "$1" in + --topic) topic="$2"; shift ;; --bootstrap-server|--reader-property) shift ;; + --property) echo "Warning: --property is deprecated and will be removed in a future version. Use --reader-property instead."; shift ;; + esac shift done [ -d "$BROKER/$topic" ] || { echo "no such topic $topic" >&2; exit 1; } diff --git a/packages/apps/kafka/Chart.yaml b/packages/apps/kafka/Chart.yaml index 7acada07e..9d88b84fe 100644 --- a/packages/apps/kafka/Chart.yaml +++ b/packages/apps/kafka/Chart.yaml @@ -4,4 +4,4 @@ description: Managed Kafka service icon: /logos/kafka.svg type: application version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process -appVersion: "3.9.0" +appVersion: "4.3.1" diff --git a/packages/apps/kafka/README.md b/packages/apps/kafka/README.md index 651e6e8d2..9872d554b 100644 --- a/packages/apps/kafka/README.md +++ b/packages/apps/kafka/README.md @@ -1,5 +1,27 @@ # Managed Kafka Service +## Node pools + +The chart runs Kafka in KRaft mode with two `KafkaNodePool` resources, one for brokers and one for controllers. The pools are not regular chart manifests: a `pre-install,pre-upgrade` Helm hook applies them on every install and upgrade, before the main release apply. Every pool that exists today was created by that hook and is absent from every release manifest. Follow a run with `kubectl logs -n job/-kafka-migration`. + +Each pool gets a short, release-hashed name (`b-` for brokers, `c-` for controllers). `KafkaNodePool` names are namespace-unique and each release hashes differently, so several Kafka instances can live in one namespace. The hash keeps the pool name at 10 characters, so the derived `--` pod hostname grows only with the release name and the node ID. At the 44-character `name` limit it fits the 63-character limit only while every node ID is a single digit; controllers take node IDs from `kafka.replicas` upward. + +A cluster migrated from ZooKeeper by an earlier chart keeps its broker pool named `kafka`: that pool adopted the cluster's original `-kafka-N` brokers and their data. The controller pool pins its node IDs above the broker range (`strimzi.io/next-node-ids`), so the broker pool keeps `[0..replicas-1]`. + +### Deletion and PVC retention + +Both node pools set `deleteClaim: false`, so deleting a Kafka release intentionally leaves its data PVCs (`data-0---N`) behind. The trade-off is that the PVCs keep consuming tenant quota until removed by hand, and recreating a same-named Kafka rebinds the stale volumes, whose on-disk cluster id will not match the new cluster (`InconsistentClusterIdException`, CrashLoop). To truly start over, delete the leftover PVCs before recreating: `kubectl -n delete pvc -l strimzi.io/cluster=`. + +### Upgrading from ZooKeeper and from Kafka 3.x + +The operator behind this chart (Strimzi 1.x) runs neither ZooKeeper nor Kafka 3.x. A ZooKeeper-based instance must reach KRaft on the Cozystack release that ships the ZooKeeper to KRaft migration (v1.7) before the platform moves past it. On every platform variant, the platform upgrade refuses to switch package sources while Strimzi still serves `v1beta2` and any Kafka lacks confirmed KRaft state. On the variants that run platform migrations, migration 61 checks the same evidence again, including on a repeat run after API conversion. Every Kafka must report a nonempty `kafkaMetadataVersion` and either `kafkaMetadataState: KRaft` or, without a state, `Ready=True` or the `cozystack.io/kraft-verified` annotation that migration 61 records on every Kafka it accepts before it stops the Strimzi operators. A `kafkaMetadataState` other than `KRaft`, a ZooKeeper section or a missing status blocks the upgrade, and the error names the reason for each Kafka. The API conversion deletes `kafkaMetadataState` and may change the spec, and neither invalidates that evidence on a retry. The migration also moves every instance on `v3.8` or `v3.9` to `v4.3`; Strimzi then rolls the brokers onto Kafka 4.3. Elsewhere those steps are manual, and the platform upgrade refuses to start while Strimzi still stores `v1beta2`: before upgrading the platform, run `/migrations/61` from this release's `platform-migrations` image with cluster-admin credentials. That run is needed on every such cluster that has the kafka-operator package installed, through the `paas` bundle or `bundles.enabledPackages`, even if it runs no Kafka: the Strimzi CRDs list `v1beta2` in their stored versions whether or not any object exists. The run stamps platform version 62 in the `cozystack-version` ConfigMap of `cozy-system`, and the migration runner starts from that stamp, so if the cluster later moves to an `isp-*` variant, the platform migrations numbered below 62 do not run there. Before it changes anything, the run checks the Kafkas, their topics and every Strimzi object, and refuses with the name and reason of each one it cannot move; it can be repeated after a failure. It then stops the Strimzi operators and converts with a Job from `quay.io/strimzi/operator:0.51.0`, pinned by digest, so a cluster that cannot reach quay.io needs that image mirrored under the same name and digest before the run. Until the platform upgrade the operators stay stopped, so a broker pod that is evicted or deleted is not recreated, and the HelmRelease of every application moved to `v4.3` reports a values error, because the old chart accepts only `v3.8` and `v3.9`: upgrade the platform right after the run. Outside that image the script, `packages/core/platform/images/migrations/migrations/61`, needs the `lib/` directory next to it and `kubectl`, `jq`, `gzip` and `base64`, plus `STRIMZI_CRD_FILE` pointing at `strimzi-crds-0.51.0.yaml` from the [Strimzi 0.51.0 release](https://github.com/strimzi/strimzi-kafka-operator/releases/tag/0.51.0). + +Kafka 4 no longer accepts clients older than Kafka 2.1: upgrade such clients before the platform upgrade. Kafka 4 also removed the `message.format.version`, `message.downconversion.enable` and `message.timestamp.difference.max.ms` topic settings, and it requires `segment.bytes` of at least 1048576 (1 MiB) where Kafka 3.x accepted much smaller values; a broker whose log holds a topic below that does not start. The platform upgrade refuses to start while a Kafka application sets one of them, migration 61 also refuses while a KafkaTopic does, and the chart refuses to render such a topic, so remove the removed settings and raise `segment.bytes` in the `config` of every topic before the upgrade. A topic created through the Kafka admin API without a KafkaTopic is not visible to these checks. + +### No downgrade + +Neither lowering the `version` field on a running cluster nor rolling the app back to an earlier chart is supported: Strimzi refuses an in-place downgrade of the KRaft metadata version, and an earlier chart renders resources for an operator API this platform no longer serves. Take a backup before upgrading. + ## Parameters ### Common parameters @@ -9,7 +31,7 @@ | `external` | Enable external access from outside the cluster. | `bool` | `false` | | `tls` | TLS configuration. Strimzi manages the cluster PKI automatically (no cert-manager is involved for this chart): the operator auto-creates `-cluster-ca-cert` and `-clients-ca-cert` secrets, both exposed for client trust setup. The internal TLS listener on 9093 is always on; this toggle only controls the external listener on 9094. | `object` | `{}` | | `tls.enabled` | Enable TLS on the external listener. When unset, inherits the value of `external` (TLS is on when external access is enabled). Warning: setting this to false while external is true exposes Kafka over plaintext on a public IP via LoadBalancer. Strimzi does not provide authentication on this listener unless SCRAM, mTLS, or OAuth is separately configured. Use only in controlled networks. | `*bool` | `null` | -| `version` | Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v3.9 back to v3.8) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it. | `string` | `v3.9` | +| `version` | Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v4.3 back to v4.2) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it. | `string` | `v4.3` | ### Application-specific parameters @@ -35,7 +57,7 @@ | `kafka.resourcesPreset` | Default sizing preset used when `resources` is omitted. | `string` | `c1.small` | | `kafka.size` | Persistent Volume size for Kafka. | `quantity` | `10Gi` | | `kafka.storageClass` | StorageClass used to store the Kafka data. | `string` | `""` | -| `kafka.controllerStorageSize` | Persistent Volume size for KRaft controller metadata (used during ZK-to-KRaft migration). | `quantity` | `5Gi` | +| `kafka.controllerStorageSize` | Persistent Volume size for KRaft controller metadata. | `quantity` | `5Gi` | ## Parameter examples and reference @@ -76,47 +98,3 @@ topics: partitions: 1 replicas: 3 ``` - -## ZooKeeper to KRaft Migration - -The chart is pure KRaft — it ships a Kafka CR annotated `strimzi.io/kraft: enabled` with no `spec.zookeeper` block. The broker and controller `KafkaNodePool` resources are **not** regular chart manifests: they are created and reconciled by the migration Job (a `pre-install,pre-upgrade` Helm hook) on every install and upgrade. This is deliberate. The hook has to create the pools and walk Strimzi's migration state machine *before* the main release apply, and a resource a hook creates cannot also be a normal chart manifest without Helm failing the migrating upgrade — its 3-way merge finds the pool in the new manifest and live in the cluster but absent from the previous (ZooKeeper) revision, and aborts with `original object KafkaNodePool ... not found`, leaving the release stuck `failed` even though the cluster migrated fine. Keeping the pools hook-owned avoids that whole class of conflict. - -Existing ZooKeeper-based instances are migrated automatically on the next chart upgrade. The Job is idempotent and safe to re-run. - -### How it works - -1. The Job runs on install and upgrade. It first ensures both `KafkaNodePool`s exist (creating or updating them), then decides what to do from the Kafka CR. -2. It inspects the Kafka CR: a genuine NotFound means a fresh install (the pools are ensured and the main apply creates the KRaft Kafka CR — nothing to migrate); a CR already annotated `strimzi.io/kraft=enabled` or in `status.kafkaMetadataState: KRaft` means done; any other read error aborts (fail closed) rather than risk driving a live ZooKeeper cluster. Otherwise (typically `ZooKeeper` state) it migrates. -3. For a migrating cluster the broker pool is named exactly `kafka` — Strimzi derives broker and PVC names as `--`, so only that name reuses the existing `-kafka-N` brokers and their data in place — while the controller pool pins its node IDs above the broker range (`strimzi.io/next-node-ids`) so the broker pool keeps `[0..replicas-1]` and Strimzi cannot hand the existing broker IDs to the fresh controllers (which would recreate the brokers empty and orphan the data). Fresh clusters use the release-hashed `b-` broker pool instead. -4. It annotates the Kafka CR with `strimzi.io/node-pools=enabled` and `strimzi.io/kraft=migration`, then polls `status.kafkaMetadataState` and waits for the migration to reach `KRaftPostMigration | PreKRaft | KRaft`. -5. Only once that safe state is reached does it flip the annotation to `strimzi.io/kraft=enabled` and wait until the state reaches `KRaft`; if the wait times out in an intermediate state the Job aborts (`exit 1`) so Helm fails the upgrade and the next reconcile retries from where the migration left off. -6. When the Job succeeds, the main release apply settles the KRaft Kafka CR, which already matches the migrated state. - -### Observability and escape hatches - -- Tail the Job logs to follow migration progress: `kubectl logs -n job/-kafka-migration` -- Monitor `status.kafkaMetadataState` on the Kafka CR directly. -- Until the cluster reaches `PreKRaft`, Strimzi's `rollback` annotation stays available as a manual escape hatch: `kubectl annotate kafka strimzi.io/kraft=rollback --overwrite` (back to `KRaftDualWriting`), then `strimzi.io/kraft=disabled` to return to `ZooKeeper`; then delete the failed Job and retry. Past `PreKRaft` the state machine accepts only `enabled`. - -### Multiple Kafka clusters per namespace - -Fresh KRaft clusters can coexist in one namespace: each one's node pools get short, release-hashed names (`b-` for brokers, `c-` for controllers), unique because `KafkaNodePool` object names are namespace-unique and each release hashes differently, while every pod/PVC/Service is already prefixed by the (unique) cluster name. The hash keeps the pool name a fixed length rather than embedding the release name — which Strimzi already prepends as the cluster name — so the derived `--` pod hostname stays within Kubernetes' 63-character limit even for long release names. So you can deploy any number of new Kafka instances side by side. - -The ZooKeeper→KRaft migration path is more constrained, but still fully automatic. To adopt an existing cluster's `-kafka-N` brokers and their data in place, the broker pool must be named exactly `kafka`, and that fixed name is namespace-unique — so co-namespaced ZooKeeper clusters cannot all migrate at once, they must serialize on the `kafka` name (Strimzi's own guidance is one Kafka per namespace for this reason — [strimzi/strimzi-kafka-operator discussions/11120](https://github.com/orgs/strimzi/discussions/11120)). The migration hook handles that serialization automatically: whichever cluster's upgrade wins the `kafka` name (claimed atomically via `kubectl create`) migrates in place; a cluster that loses the race fails its hook and its next reconcile retries. Once a migrated cluster sees other ZooKeeper clusters still waiting, it frees the name — it stands up its own `b-` broker pool, drains all data off the `kafka`-pool brokers with a Cruise Control `KafkaRebalance` (`mode: remove-brokers`, done online, no downtime), deletes the drained `kafka` pool, and the next waiting cluster's retry adopts the freed name. The last cluster to migrate keeps `kafka`. So N co-namespaced ZooKeeper clusters all reach KRaft with no manual steps — sequentially, and with one full data drain per cluster except the last. Cruise Control is enabled only for the drain and removed after. Fresh KRaft clusters are never involved in any of this. - -### Deletion and PVC retention - -Both node pools set `deleteClaim: false`, so deleting a Kafka release intentionally leaves its data PVCs (`data-0---N`) behind — this protects data across the migration and against accidental deletion. The trade-off is that the PVCs keep consuming tenant quota until removed by hand, and recreating a same-named Kafka rebinds the stale volumes, whose on-disk cluster id will not match the new cluster (`InconsistentClusterIdException`, CrashLoop). To truly start over, delete the leftover PVCs before recreating: `kubectl -n delete pvc -l strimzi.io/cluster=`. - -### No downgrade across the KRaft boundary - -Migration to KRaft is a point of no return once the cluster passes `PreKRaft` (before that, the escape hatch above still works). After it, Strimzi has no KRaft→ZooKeeper rollback, so neither lowering the `version` field on a running cluster nor rolling the whole app back to a pre-KRaft chart is supported: rolling the chart back would re-render the `spec.zookeeper` objects over live KRaft storage and drop the `strimzi.io/kraft` / `strimzi.io/node-pools` annotations — that does not restore ZooKeeper, it **damages** the Kafka cluster. Take a backup first, and never downgrade the kafka app or the platform once a cluster has reached KRaft (`strimzi.io/kraft=enabled`). - -**This chart release is a mandatory stop on the upgrade path.** A cluster that skips it arrives at a future Strimzi (0.46+) still on ZooKeeper, and 0.46 removed the ZooKeeper→KRaft migration entirely — there is then no supported path off ZooKeeper. Make sure every Kafka in the platform has reached KRaft on this release before upgrading past it. - -Budget a maintenance window of **up to a few hours per instance**: the in-place migration is tens of minutes (broker rolling restarts), and when several Kafkas share a namespace each one but the last also drains its full dataset off the shared `kafka` pool via a Cruise Control rebalance to free the name, which scales with data size. - -### Important notes - -- **Strimzi 0.45 is the last version supporting ZooKeeper.** Future Strimzi releases only support KRaft. -- The `kafka.controllerStorageSize` parameter controls PV size for the new KRaft controller nodes (default: `5Gi`). diff --git a/packages/apps/kafka/files/versions.yaml b/packages/apps/kafka/files/versions.yaml index caf329aee..64dedf4f9 100644 --- a/packages/apps/kafka/files/versions.yaml +++ b/packages/apps/kafka/files/versions.yaml @@ -1,2 +1,2 @@ -"v3.9": "3.9.0" -"v3.8": "3.8.1" +"v4.3": "4.3.1" +"v4.2": "4.2.1" diff --git a/packages/apps/kafka/templates/_versions.tpl b/packages/apps/kafka/templates/_versions.tpl index 30c63b3ef..dc8bbf91b 100644 --- a/packages/apps/kafka/templates/_versions.tpl +++ b/packages/apps/kafka/templates/_versions.tpl @@ -16,23 +16,17 @@ deterministic token — a role letter plus an 8-hex digest of the release — which is unique within the namespace (each release hashes differently) and adds a fixed ~10 characters regardless of release length. The broker pool of a - MIGRATED cluster is the one exception: it stays exactly "kafka" to adopt the - existing -kafka-N brokers and their data (see kafka.brokerPoolName). + cluster migrated from ZooKeeper is the one exception: it stays exactly "kafka", + the pool that adopted its -kafka-N brokers (see kafka.brokerPoolName). */ -}} {{- define "kafka.poolHash" -}} {{- substr 0 8 (sha256sum .Release.Name) -}} {{- end -}} {{- /* - The controller pool name is a pure function of the release — no adoption/ - self-ownership lookup like kafka.brokerPoolName has, on purpose. A KRaft - controller pool is ALWAYS created new: a fresh install makes one, and a - ZooKeeper cluster has no controller pool to adopt (KRaft controllers do not - exist under ZooKeeper). So there is never a pre-existing, differently-named - controller pool for this name to preserve — the broker's "kafka"-adoption - branch has no controller analogue because the underlying situation it handles - (live brokers to reuse in place) has no controller counterpart. No released - chart ships a controller pool under any other name, so no upgrade renames one. + The controller pool name is a pure function of the release, with no ownership + lookup like kafka.brokerPoolName has: no released chart ships a controller + pool under any other name, so no upgrade renames one. */ -}} {{- define "kafka.controllerPoolName" -}} {{- printf "c-%s" (include "kafka.poolHash" .) -}} @@ -41,7 +35,7 @@ {{- /* KRaft controller quorum — fixed at creation, never re-derived on a day-2 edit. - The quorum must be static: Strimzi 0.45 cannot scale a controller node pool, so + The quorum must be static: this chart does not resize a live KRaft quorum, so flipping it (e.g. because a tenant raised kafka.replicas from 1 to 3) would wedge the reconcile or lose the metadata quorum. So the size is pinned once: if the controller pool already exists, keep its current replica count; only on first @@ -57,81 +51,30 @@ applied — either way, computing the initial value is correct. */ -}} {{- define "kafka.controllerReplicas" -}} -{{- $existing := lookup "kafka.strimzi.io/v1beta2" "KafkaNodePool" .Release.Namespace (include "kafka.controllerPoolName" .) -}} +{{- $existing := lookup "kafka.strimzi.io/v1" "KafkaNodePool" .Release.Namespace (include "kafka.controllerPoolName" .) -}} {{- $pinned := dig "spec" "replicas" 0 ($existing | default dict) -}} {{- if $pinned -}}{{ int $pinned }}{{- else if le (int .Values.kafka.replicas) 1 -}}1{{- else -}}3{{- end -}} {{- end -}} {{- /* - Broker KafkaNodePool name — a short release-hashed token for fresh installs so - several Kafka clusters can coexist in one namespace, but exactly "kafka" for a - cluster being migrated from ZooKeeper. + Broker KafkaNodePool name: "kafka" for a cluster migrated from ZooKeeper, whose + pool of that name adopted its -kafka-N brokers and their PVCs, and + the release-hashed "b-" for every other cluster. - Strimzi derives node and PVC names as --. A fresh KRaft - cluster has no prior state to preserve, so its broker pool is named "b-" - — unique within the namespace (each release hashes differently), letting - multiple fresh clusters live side by side without doubling the release name - into every pod hostname. A pre-node-pool ZooKeeper cluster, however, has - brokers "-kafka-N", and only a pool named exactly "kafka" adopts those - brokers (and their PVCs) in place during migration. - - Resolution (lookup runs at render time, before the pre-upgrade migration Job): - - this cluster already owns a "kafka" pool -> "kafka" (migrated, steady state) - - this cluster already owns a "b-" pool -> "b-" (fresh, steady state) - - an existing non-KRaft CR with no release pool yet -> "kafka" (classic ZK about to migrate) - - nothing exists -> "b-" (fresh install) - - Safety of the lookup: Helm propagates any non-NotFound read error and fails the - whole render (pkg/engine/lookup_func.go returns an empty result only on - apierrors.IsNotFound), so a transient apiserver/RBAC miss makes the reconcile - retry — it never silently resolves a live migrated cluster to "b-" and - lets a later upgrade prune the "kafka" pool. An empty lookup means genuine - absence: a real fresh install, or a client-side render (helm template/unittest) - that is never applied. - - Because "kafka" is a fixed, namespace-unique name, co-namespaced ZooKeeper->KRaft - migrations run one at a time: while another cluster owns the "kafka" pool, this - cluster's render fails closed and its HelmRelease retries. The migration hook - drains the first cluster off the shared pool with a Cruise Control rebalance and - frees the name, then the waiting cluster retries into it — so every co-namespaced - Kafka reaches KRaft, not just the first (strimzi/strimzi-kafka-operator - discussions/11120). Fresh KRaft clusters are never affected. lookup returns - nothing during `helm template`, so unit tests resolve to the fresh "b-" - default. + An empty lookup means genuine absence: Helm propagates any non-NotFound read + error and fails the render (pkg/engine/lookup_func.go returns an empty result + only on apierrors.IsNotFound), so a transient apiserver/RBAC miss never + resolves a migrated cluster to "b-", which would make the hook create a + second, empty broker pool. lookup returns nothing during `helm template`, so + unit tests resolve to "b-". */ -}} {{- define "kafka.brokerPoolName" -}} -{{- $release := .Release.Name -}} -{{- $ns := .Release.Namespace -}} -{{- $brokerFresh := printf "b-%s" (include "kafka.poolHash" .) -}} -{{- $name := $brokerFresh -}} -{{- $kp := lookup "kafka.strimzi.io/v1beta2" "KafkaNodePool" $ns "kafka" -}} -{{- $kpOwner := "" -}} -{{- if $kp -}}{{- $kpOwner = dig "metadata" "labels" "strimzi.io/cluster" "" $kp -}}{{- end -}} -{{- if and $kp (eq $kpOwner $release) -}} - {{- /* Migrated, steady state: this cluster owns the adopted "kafka" pool. */ -}} - {{- $name = "kafka" -}} +{{- $kp := lookup "kafka.strimzi.io/v1" "KafkaNodePool" .Release.Namespace "kafka" -}} +{{- if and $kp (eq (dig "metadata" "labels" "strimzi.io/cluster" "" $kp) .Release.Name) -}} +kafka {{- else -}} - {{- $existingCR := lookup "kafka.strimzi.io/v1beta2" "Kafka" $ns $release -}} - {{- $ownBrokerPool := lookup "kafka.strimzi.io/v1beta2" "KafkaNodePool" $ns $brokerFresh -}} - {{- $kraftAnn := "" -}} - {{- if $existingCR -}}{{- $kraftAnn = dig "metadata" "annotations" "strimzi.io/kraft" "" $existingCR -}}{{- end -}} - {{- if $ownBrokerPool -}} - {{- /* Fresh KRaft cluster, steady state: it already owns its "b-" pool. */ -}} - {{- $name = $brokerFresh -}} - {{- else if and $existingCR (ne $kraftAnn "enabled") -}} - {{- /* Classic ZooKeeper cluster about to migrate: adopt its brokers as "kafka". */ -}} - {{- if and $kp (ne $kpOwner "") (ne $kpOwner $release) -}} - {{- fail (printf "Kafka %q is waiting to migrate: namespace %q still has the shared \"kafka\" broker pool owned by cluster %q, which is migrating first. Strimzi node pool names are namespace-unique, so co-namespaced ZooKeeper->KRaft migrations run one at a time (see strimzi/strimzi-kafka-operator discussions/11120): %q drains off the shared pool and frees the name, then this HelmRelease retries into it automatically — no manual action needed. Fresh KRaft clusters are unaffected." $release $ns $kpOwner $kpOwner) -}} - {{- end -}} - {{- $name = "kafka" -}} - {{- end -}} - {{- /* else: fresh install (no CR, no own pool) -> "b-". A transient read - error can't reach here silently: Helm propagates every non-NotFound error - and fails the render, so an empty lookup means genuine absence (a real - fresh install, or a client-side helm-template render that is never - applied), never a flaked read of a live migrated cluster's "kafka" pool. */ -}} +{{- printf "b-%s" (include "kafka.poolHash" .) -}} {{- end -}} -{{- $name -}} {{- end -}} {{- /* diff --git a/packages/apps/kafka/templates/kafka.yaml b/packages/apps/kafka/templates/kafka.yaml index 167a43b05..e1b5dc0ed 100644 --- a/packages/apps/kafka/templates/kafka.yaml +++ b/packages/apps/kafka/templates/kafka.yaml @@ -17,16 +17,13 @@ {{- $showExternal := .Values.external -}} {{- $kafkaVersion := include "kafka.versionMap" . -}} {{- $entityOperatorMemory := "512Mi" -}} -apiVersion: kafka.strimzi.io/v1beta2 +apiVersion: kafka.strimzi.io/v1 kind: Kafka metadata: name: {{ .Release.Name }} labels: app.kubernetes.io/instance: {{ .Release.Name }} app.kubernetes.io/managed-by: {{ .Release.Service }} - annotations: - strimzi.io/node-pools: enabled - strimzi.io/kraft: enabled spec: kafka: version: {{ $kafkaVersion }} @@ -65,6 +62,13 @@ spec: default.replication.factor: 3 min.insync.replicas: 2 {{- end }} + template: + pod: + metadata: + labels: + # The nodes read their certificates from Secrets through the + # Kubernetes API (Strimzi's strimzisecrets config provider). + policy.cozystack.io/allow-to-apiserver: "true" metricsConfig: type: jmxPrometheusExporter valueFrom: diff --git a/packages/apps/kafka/templates/migration-hook.yaml b/packages/apps/kafka/templates/migration-hook.yaml index 997b28ea1..646a6ae1d 100644 --- a/packages/apps/kafka/templates/migration-hook.yaml +++ b/packages/apps/kafka/templates/migration-hook.yaml @@ -1,32 +1,11 @@ {{- /* - Node pools are managed entirely by this hook, NOT by a regular chart manifest. - - Reason: an in-place ZooKeeper->KRaft migration must create the broker and - controller KafkaNodePools BEFORE the main release apply (Strimzi has to walk - the migration state machine while the old ZooKeeper Kafka CR is still live). - If the pools were also regular chart manifests, Helm's main apply would try to - reconcile a resource that exists in the cluster but was absent from the - previous (ZooKeeper) release revision, and fail the upgrade with - "original object KafkaNodePool ... not found" — leaving the HelmRelease stuck - in a failed state even though the Kafka cluster migrated fine. Keeping the - pools out of the main manifest and applying them here (idempotently, on every - install and upgrade) avoids that whole class of Helm 3-way-merge conflict. - - Multiple ZooKeeper Kafkas per namespace: only a pool named exactly "kafka" - adopts a ZooKeeper cluster's existing -kafka-N brokers in place, and - that name is namespace-unique, so migrations serialize on it. To let every - co-namespaced cluster reach KRaft, a cluster that took "kafka" and finds other - ZooKeeper clusters still waiting frees the name after migrating: it stands up - its own release-hashed "b-" broker pool, drains all data off the "kafka" - pool brokers with a Cruise Control KafkaRebalance, deletes the "kafka" pool, - and the next waiting cluster's retry grabs the freed name. The last one to - migrate keeps "kafka". Cruise Control is enabled only for the drain and removed - after (kafka.yaml never sets spec.cruiseControl, so Helm does not fight it). + Node pools are applied by this hook, not as regular chart manifests. Every + pool that exists today was created here and is absent from every release + manifest; applying them before the main release apply also keeps the Kafka + CR from being reconciled against pools that do not exist yet. */ -}} {{- $controllerReplicas := include "kafka.controllerReplicas" . }} {{- $brokerPool := include "kafka.brokerPoolName" . }} -{{- $freshBrokerPool := printf "b-%s" (include "kafka.poolHash" .) }} -{{- $cruiseControlMemory := "1Gi" }} --- apiVersion: v1 kind: ServiceAccount @@ -47,17 +26,8 @@ metadata: helm.sh/hook-delete-policy: before-hook-creation rules: - apiGroups: ["kafka.strimzi.io"] - # kafkanodepools: delete is needed to remove the drained "kafka" pool when - # freeing the name; kafkarebalances drive the Cruise Control drain. - resources: ["kafkas", "kafkanodepools", "kafkarebalances"] - verbs: ["get", "list", "watch", "create", "patch", "update", "delete"] -- apiGroups: [""] - # The drained "kafka" pool's PVCs (deleteClaim:false keeps them) are removed - # once empty, so freeing the name does not leak storage. "watch" is needed - # because `kubectl delete` waits on a watch by default — without it the delete - # still succeeds but logs noisy "cannot watch persistentvolumeclaims" errors. - resources: ["persistentvolumeclaims"] - verbs: ["get", "list", "watch", "delete"] + resources: ["kafkanodepools"] + verbs: ["get", "create", "patch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding @@ -88,13 +58,7 @@ metadata: helm.sh/hook-delete-policy: before-hook-creation spec: backoffLimit: 0 - # Freeing the "kafka" name drains the whole dataset off the old brokers with a - # Cruise Control rebalance, which can take hours on a large cluster. The Job - # deadline must cover that; if it is exceeded the Job is killed, Helm fails the - # upgrade, and the next reconcile re-runs this Job, which resumes from where it - # left off (every phase below is idempotent). The release's helm timeout must - # exceed this deadline or the migration is aborted before it can finish. - activeDeadlineSeconds: 21600 + activeDeadlineSeconds: 300 template: metadata: labels: @@ -113,14 +77,6 @@ spec: ns={{ .Release.Namespace | quote }} name={{ .Release.Name | quote }} brokerPool={{ $brokerPool | quote }} - freshBrokerPool={{ $freshBrokerPool | quote }} - rebalance="${name}-drain-kafka" - # Fully qualify the resources: the plural "kafkas" also exists in - # this chart's own apps.cozystack.io group, so a bare "kafka" could - # resolve to the tenant kind, which this hook's Role does not grant. - kafka=kafkas.kafka.strimzi.io - np=kafkanodepools.kafka.strimzi.io - kr=kafkarebalances.kafka.strimzi.io # Every apply here passes --openapi-patch=false. Without it, applying # an object that already exists makes kubectl download the cluster's @@ -129,17 +85,15 @@ spec: # while this container runs under a tenant LimitRange default of # 128Mi. With it kubectl builds the same merge patch from the # last-applied annotation. Not --server-side: fields that - # "kubectl create" set on the "kafka" pool stay co-owned by that - # manager, so a server-side apply would never remove them. - # ---- Node-pool manifests (broker pool reused for the create-CAS) ---- + # "kubectl create" set on a pool stay co-owned by that manager, so + # a server-side apply would never remove them. broker_manifest=$(cat <-kafka-N brokers and their data in - # place); "b-" for a fresh cluster. Resolved at render time - # by kafka.brokerPoolName. + # "kafka" for a cluster migrated from ZooKeeper (that pool adopted + # its -kafka-N brokers), "b-" otherwise. Resolved + # at render time by kafka.brokerPoolName. name: ${brokerPool} namespace: ${ns} labels: @@ -153,9 +107,7 @@ spec: roles: - broker resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | nindent 18 }} - # A pool template replaces spec.kafka.template whole, for every - # pool here: nothing set on the Kafka CR's template reaches these - # pods (KafkaPool.java at 0.45.1-rc1: https://github.com/strimzi/strimzi-kafka-operator/blob/0.45.1-rc1/cluster-operator/src/main/java/io/strimzi/operator/cluster/model/KafkaPool.java#L196-L218). + # JAVA_TOOL_OPTIONS: see kafka.javaToolOptions. template: kafkaContainer: env: @@ -172,17 +124,15 @@ spec: {{- with .Values.kafka.storageClass }} class: {{ . }} {{- end }} - # deleteClaim:false is load-bearing here. Strimzi's PvcReconciler - # deletes a pool's PVCs when the pool is removed, so with - # deleteClaim:true the drain's deletion of the "kafka" pool (and - # a plain pool teardown) would destroy the tenant's data instead - # of leaving the volumes to be adopted / cleaned up deliberately. + # Strimzi's PvcReconciler deletes a pool's PVCs when the pool + # is removed; deleteClaim:false leaves the tenant's data on a + # pool teardown. deleteClaim: false MANIFEST ) controller_manifest=$(cat <-kafka-N brokers and their PVCs in place. - # Without this Strimzi hands the controller pool the lowest free - # IDs — the broker IDs — recreating brokers empty and orphaning - # the tenant's data. + # Controller node IDs start above the broker range, leaving + # [0..replicas-1] to the broker pool. A cluster migrated from + # ZooKeeper relies on it: its brokers kept their original IDs. strimzi.io/next-node-ids: "[{{ .Values.kafka.replicas }}-999]" spec: replicas: {{ $controllerReplicas }} @@ -224,307 +171,8 @@ spec: MANIFEST ) - # ---- Ensure both node pools exist ---- - # Broker pool ownership FIRST, before any apply, so a "kafka" pool - # owned by another cluster aborts without mutating anything. For a - # fresh cluster the name is the release-unique "b-" and there - # is no collision — a plain apply is safe. For a cluster migrating - # from ZooKeeper the name is the namespace-unique "kafka"; enforce - # one migration per namespace ATOMICALLY. A plain read-then-apply - # would race: two ZooKeeper Kafkas in one namespace upgraded - # together could both observe no owner and both apply, the second - # silently re-parenting the first's brokers. Instead, when the pool - # is absent, "kubectl create" is the atomic arbiter: exactly one - # concurrent creator wins; the loser's create fails, re-reads the - # owner, and aborts (fail fast — it retries once the winner frees - # the name). - if [ "$brokerPool" = "kafka" ]; then - if ! owner=$(kubectl -n "$ns" get "$np" kafka --ignore-not-found -o jsonpath='{.metadata.labels.strimzi\.io/cluster}' 2>&1); then - echo "ERROR: reading the \"kafka\" node pool owner label failed: $owner" >&2 - exit 1 - fi - if [ -z "$owner" ]; then - echo "Creating broker pool \"kafka\"..." - if echo "$broker_manifest" | kubectl create -f - 2>/tmp/createerr; then - owner="$name" - else - if ! owner=$(kubectl -n "$ns" get "$np" kafka --ignore-not-found -o jsonpath='{.metadata.labels.strimzi\.io/cluster}' 2>&1); then - echo "ERROR: re-reading the \"kafka\" node pool owner after create failure failed: $owner" >&2 - exit 1 - fi - if [ -z "$owner" ]; then - echo "ERROR: creating broker pool \"kafka\" failed: $(cat /tmp/createerr)" >&2 - exit 1 - fi - fi - fi - if [ "$owner" != "$name" ]; then - echo "ERROR: KafkaNodePool 'kafka' in namespace '$ns' already belongs to cluster '$owner'." >&2 - echo "Its ZooKeeper->KRaft migration is in progress and will free the name when done; this cluster's upgrade will retry and proceed then. (Strimzi node pool names are namespace-unique, so migrations serialize on the \"kafka\" name.)" >&2 - exit 1 - fi - fi echo "Ensuring broker KafkaNodePool \"$brokerPool\" exists..." echo "$broker_manifest" | kubectl apply --openapi-patch=false -f - echo "Ensuring controller KafkaNodePool exists..." echo "$controller_manifest" | kubectl apply --openapi-patch=false -f - - - # ---- Decide what to do from the Kafka CR ---- - # Fail closed. --ignore-not-found makes kubectl exit 0 with empty - # output on a genuine NotFound (fresh install: the Kafka CR is - # created by the main release apply, after this hook) and exit - # non-zero only on a real error (apiserver/RBAC/timeout). - if ! found=$(kubectl -n "$ns" get "$kafka" "$name" --ignore-not-found -o name 2>&1); then - echo "ERROR: reading Kafka CR failed: $found" >&2 - exit 1 - fi - if [ -z "$found" ]; then - echo "No existing Kafka CR — fresh install. Node pools ensured; the main release apply will create the KRaft Kafka CR. Nothing to migrate." - exit 0 - fi - - # Read the current KRaft annotation / metadata state (fail closed). - if ! kraft=$(kubectl -n "$ns" get "$kafka" "$name" -o jsonpath='{.metadata.annotations.strimzi\.io/kraft}' 2>&1); then - echo "ERROR: reading the strimzi.io/kraft annotation failed: $kraft" >&2 - exit 1 - fi - if ! state=$(kubectl -n "$ns" get "$kafka" "$name" -o jsonpath='{.status.kafkaMetadataState}' 2>&1); then - echo "ERROR: reading kafkaMetadataState failed: $state" >&2 - exit 1 - fi - echo "Current strimzi.io/kraft='${kraft}', kafkaMetadataState='${state}'." - - # ---- Migrate a ZooKeeper cluster (skip if already KRaft) ---- - if [ "$kraft" != "enabled" ] && [ "$state" != "KRaft" ]; then - # Enabling node pools is idempotent. Only issue kraft=migration - # from a genuine pre-migration state: on a retry the cluster may - # already be at PreKRaft/KRaftPostMigration, where re-annotating - # "migration" is an invalid backward transition Strimzi rejects. - echo "Enabling node pools on the Kafka CR..." - kubectl -n "$ns" annotate "$kafka" "$name" strimzi.io/node-pools=enabled --overwrite - case "$state" in - KRaftMigration|KRaftDualWriting|KRaftPostMigration|PreKRaft|KRaft) - echo "Migration already past the ZooKeeper phase (state: $state); not re-issuing kraft=migration." ;; - *) - echo "Starting KRaft migration (kraft=migration)..." - kubectl -n "$ns" annotate "$kafka" "$name" strimzi.io/kraft=migration --overwrite ;; - esac - - echo "Waiting for migration to reach KRaftPostMigration..." - migration_ready=false - for i in $(seq 1 240); do - state=$(kubectl -n "$ns" get "$kafka" "$name" -o jsonpath='{.status.kafkaMetadataState}' || echo "") - case "$state" in - KRaftPostMigration|PreKRaft|KRaft) - echo "Reached state: $state" - migration_ready=true - break - ;; - *) - echo "[$i/240] State: ${state}, waiting..." - sleep 5 - ;; - esac - done - - # Strimzi only permits strimzi.io/kraft=enabled once the cluster - # has reached KRaftPostMigration. Applying it earlier — while - # brokers still depend on ZooKeeper — is unsafe and irreversible. - # If the wait timed out in an intermediate state, abort without - # finalising so Helm fails the upgrade and the next reconcile - # retries from where the migration left off. - if [ "$migration_ready" != "true" ]; then - echo "ERROR: migration did not reach KRaftPostMigration (last state: ${state:-unknown})." - echo "Refusing to apply strimzi.io/kraft=enabled in an unsafe state." - exit 1 - fi - - echo "Finalising migration to KRaft..." - kubectl -n "$ns" annotate "$kafka" "$name" strimzi.io/kraft=enabled --overwrite - - echo "Waiting for KRaft state..." - migrated=false - for i in $(seq 1 120); do - state=$(kubectl -n "$ns" get "$kafka" "$name" -o jsonpath='{.status.kafkaMetadataState}' || echo "") - if [ "$state" = "KRaft" ]; then - echo "Migration complete." - migrated=true - break - fi - echo "[$i/120] State: ${state}, waiting..." - sleep 5 - done - if [ "$migrated" != "true" ]; then - echo "ERROR: migration did not complete in time." - exit 1 - fi - else - echo "Kafka is already KRaft — no migration needed." - fi - - # ---- Free the "kafka" name for the next waiting cluster ---- - # Only relevant if THIS cluster ended up owning the shared "kafka" - # pool (a migrated ZooKeeper cluster). If other ZooKeeper Kafkas - # remain in the namespace, drain this cluster off "kafka" onto its - # release-hashed "b-" pool and delete "kafka" so the next one - # can adopt the name. Every step is idempotent so a killed Job - # resumes on retry. - if [ "$brokerPool" != "kafka" ]; then - echo "This cluster runs on \"$brokerPool\" (not the shared \"kafka\" pool); nothing to free." - exit 0 - fi - # Reading fails open if the pipeline swallows the kubectl error: an - # empty list is indistinguishable from a failed read, and both would - # count 0 and keep the pool, stranding any co-namespaced ZK cluster - # still waiting. Capture the read status separately, like the drain - # read below. - if ! others_raw=$(kubectl -n "$ns" get "$kafka" -o jsonpath='{range .items[*]}{.metadata.name}={.metadata.annotations.strimzi\.io/kraft}{"\n"}{end}' 2>&1); then - echo "ERROR: could not list Kafka CRs in $ns to decide whether to free the \"kafka\" pool: $others_raw" >&2 - exit 1 - fi - others=$(printf '%s\n' "$others_raw" | awk -F= -v me="$name" '$1!="" && $1!=me && $2!="enabled"{c++} END{print c+0}') - if [ "$others" -eq 0 ]; then - echo "No other ZooKeeper Kafka clusters in $ns; keeping the \"kafka\" pool. Done." - exit 0 - fi - echo "$others other ZooKeeper Kafka cluster(s) remain in $ns; freeing the \"kafka\" pool name..." - - # F1: stand up this cluster's own release-hashed broker pool with - # node IDs pinned high, clear of the existing broker/controller IDs. - echo "Creating target broker pool \"$freshBrokerPool\"..." - cat </dev/null || echo "") - if [ -z "$kafkaIds" ] || [ "$kafkaIds" = "[]" ]; then - echo "ERROR: could not read the \"kafka\" pool node IDs; refusing to drain blindly." >&2 - exit 1 - fi - echo "Draining brokers $kafkaIds via Cruise Control rebalance..." - cat </dev/null || echo "") - case " $st " in - *" Ready "*) - echo "Rebalance complete." - rebalanced=true - break ;; - *" ProposalReady "*) - echo "[$i] Proposal ready; approving..." - kubectl -n "$ns" annotate "$kr" "$rebalance" strimzi.io/rebalance=approve --overwrite || true - sleep 15 ;; - *" NotReady "*) - # A NotReady rebalance is not treated as terminal. Cruise - # Control rejects proposals transiently while it settles: a - # just-enabled CC is unreachable for a minute or two (its pod - # and cruise-control-certs Secret are still coming up — - # CruiseControlRetriableConnectionException), and its capacity - # model can briefly over-estimate load from noisy samples and - # trip an optimization goal even with skipHardGoalCheck. Both - # clear once CC's metric windows age out, so refresh the - # proposal (recompute against current metrics) and keep - # polling. This also covers resume: a re-run of the Job finds a - # stale NotReady rebalance and refreshes it instead of reading - # its old failure forever. Only the loop timeout below is - # terminal — then the Job fails and the next reconcile retries. - reason=$(kubectl -n "$ns" get "$kr" "$rebalance" -o jsonpath='{.status.conditions[*].reason}' 2>/dev/null || echo "") - echo "[$i] Rebalance NotReady (${reason}); refreshing the proposal and waiting for Cruise Control to settle..." - kubectl -n "$ns" annotate "$kr" "$rebalance" strimzi.io/rebalance=refresh --overwrite || true - sleep 15 ;; - *) - echo "[$i] Rebalance state: '${st}', waiting..." - sleep 15 ;; - esac - done - if [ "$rebalanced" != "true" ]; then - echo "ERROR: rebalance did not complete within the Job deadline; will resume on the next retry." - exit 1 - fi - - # F4/F5: the drained "kafka" brokers now hold no partitions — remove - # the pool and its (now empty) PVCs. After this the render-time - # kafka.brokerPoolName resolves to "$freshBrokerPool" (this cluster - # owns it), so the cluster stays there on every future reconcile. - echo "Deleting the drained \"kafka\" pool and its PVCs..." - kubectl -n "$ns" delete "$np" kafka --ignore-not-found - for id in $(echo "$kafkaIds" | tr -d '[] ' | tr ',' ' '); do - kubectl -n "$ns" delete pvc "data-0-${name}-kafka-${id}" --ignore-not-found - done - - # F6: tear Cruise Control back down and clean up the rebalance CR. - echo "Removing Cruise Control and the rebalance CR..." - kubectl -n "$ns" patch "$kafka" "$name" --type json -p '[{"op":"remove","path":"/spec/cruiseControl"}]' 2>/dev/null || true - kubectl -n "$ns" delete "$kr" "$rebalance" --ignore-not-found - - echo "Freed the \"kafka\" pool name; this cluster now runs on \"$freshBrokerPool\". The next waiting cluster's retry will adopt it." - exit 0 diff --git a/packages/apps/kafka/templates/topics.yaml b/packages/apps/kafka/templates/topics.yaml index 712b5d670..2456f5018 100644 --- a/packages/apps/kafka/templates/topics.yaml +++ b/packages/apps/kafka/templates/topics.yaml @@ -1,6 +1,39 @@ +{{- /* + Topic configs Kafka 3.x accepted and Kafka 4.2/4.3 reject. A KafkaTopic + carrying one stays NotReady while the release reports Ready, and a broker + whose log holds a segment.bytes below the floor does not start. The same + limits are in packages/core/platform/templates/_helpers.tpl, + packages/core/platform/images/migrations/migrations/61 and + packages/system/backupstrategy-controller/files/kafka-backup.sh. +*/}} +{{- $segmentBytesMin := 1048576 }} {{- range $topic := .Values.topics }} +{{- $config := $topic.config | default dict }} +{{- range $key := list "message.downconversion.enable" "message.format.version" "message.timestamp.difference.max.ms" }} +{{- if hasKey $config $key }} +{{- fail (printf "topic %q sets %s, which Kafka 4 removed: delete it from the topic's config" $topic.name $key) }} +{{- end }} +{{- end }} +{{- $segmentBytes := get $config "segment.bytes" }} +{{- if or (kindIs "float64" $segmentBytes) (kindIs "int64" $segmentBytes) (kindIs "int" $segmentBytes) }} +{{- $segmentBytes = printf "%d" (int64 $segmentBytes) }} +{{- end }} +{{- /* + Kafka trims characters up to U+0020 and reads the rest with + Integer.parseInt: a sign is allowed and leading zeros are decimal, where + int64 would read them as octal. The digits are compared as text, so a + string too long for int64 is not misread either. +*/}} +{{- $integer := `^[\x00-\x20]*(?:\+|(-))?0*([0-9]+)[\x00-\x20]*$` }} +{{- if and (kindIs "string" $segmentBytes) (regexMatch $integer $segmentBytes) }} +{{- $value := regexReplaceAll $integer $segmentBytes "${1}${2}" }} +{{- $min := toString $segmentBytesMin }} +{{- if or (hasPrefix "-" $value) (lt (len $value) (len $min)) (and (eq (len $value) (len $min)) (lt $value $min)) }} +{{- fail (printf "topic %q sets segment.bytes=%s, below the Kafka 4 minimum of %d: raise it" $topic.name $value $segmentBytesMin) }} +{{- end }} +{{- end }} --- -apiVersion: kafka.strimzi.io/v1beta2 +apiVersion: kafka.strimzi.io/v1 kind: KafkaTopic metadata: name: "{{ $.Release.Name }}-{{ kebabcase $topic.name }}" diff --git a/packages/apps/kafka/tests/jvm_sizing_test.yaml b/packages/apps/kafka/tests/jvm_sizing_test.yaml index f4569a1e2..223131814 100644 --- a/packages/apps/kafka/tests/jvm_sizing_test.yaml +++ b/packages/apps/kafka/tests/jvm_sizing_test.yaml @@ -19,9 +19,6 @@ tests: - matchRegex: path: spec.template.spec.containers[0].args[0] pattern: "roles:\\s+- controller\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=1073741824 -XX:ActiveProcessorCount=1\"\\s+storage:" - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "next-node-ids: \"\\[1000-1099\\]\"\\s+spec:\\s+replicas: \\d+\\s+roles:\\s+- broker\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=1073741824 -XX:ActiveProcessorCount=1\"\\s+storage:" # Strimzi moves jvmOptions -XX into KAFKA_JVM_PERFORMANCE_OPTS, and # kafka-run-class.sh then drops its G1 settings. @@ -64,19 +61,6 @@ tests: - matchRegex: path: spec.template.spec.containers[0].args[0] pattern: "roles:\\s+- controller\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=2147483648 -XX:ActiveProcessorCount=2\"\\s+storage:" - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "next-node-ids: \"\\[1000-1099\\]\"\\s+spec:\\s+replicas: \\d+\\s+roles:\\s+- broker\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=2147483648 -XX:ActiveProcessorCount=2\"\\s+storage:" - - # The whole patch, closing brace included: a merge patch kubectl cannot - # parse fails the hook before the drain starts. - - it: passes the drain-time Cruise Control memory limit to its JVM - template: templates/migration-hook.yaml - documentIndex: 3 - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "\\-\\-type\\ merge\\ \\-p\\ '\\{\"spec\":\\{\"cruiseControl\":\\{\"resources\":\\{\"limits\":\\{\"memory\":\"1Gi\"\\},\"requests\":\\{\"cpu\":\"200m\",\"memory\":\"768Mi\"\\}\\},\"template\":\\{\"cruiseControlContainer\":\\{\"env\":\\[\\{\"name\":\"JAVA_TOOL_OPTIONS\",\"value\":\"\\-XX:MaxRAM=1073741824\"\\}\\]\\}\\}\\}\\}\\}'\n" - it: passes the entity operator memory limit to its JVMs template: templates/kafka.yaml diff --git a/packages/apps/kafka/tests/kafka_test.yaml b/packages/apps/kafka/tests/kafka_test.yaml index dc3f406ab..0fd5a0488 100644 --- a/packages/apps/kafka/tests/kafka_test.yaml +++ b/packages/apps/kafka/tests/kafka_test.yaml @@ -8,7 +8,7 @@ tests: # KRaft mode (new inst.) # ########################## - - it: renders Kafka CR with KRaft annotations for new instance + - it: renders a v1 Kafka CR without the KRaft annotations Strimzi ignores release: name: test-kafka namespace: tenant-test @@ -17,12 +17,12 @@ tests: count: 1 - isKind: of: Kafka - - equal: + - isAPIVersion: + of: kafka.strimzi.io/v1 + - notExists: path: metadata.annotations["strimzi.io/kraft"] - value: enabled - - equal: + - notExists: path: metadata.annotations["strimzi.io/node-pools"] - value: enabled - it: sets correct name and labels release: @@ -40,25 +40,35 @@ tests: # Version # ########################## - - it: sets Kafka version from version map (default v3.9) + - it: sets Kafka version from version map (default v4.3) release: name: test-kafka namespace: tenant-test asserts: - equal: path: spec.kafka.version - value: "3.9.0" + value: "4.3.1" - - it: resolves v3.8 version + - it: resolves v4.2 version release: name: test-kafka namespace: tenant-test set: - version: "v3.8" + version: "v4.2" asserts: - equal: path: spec.kafka.version - value: "3.8.1" + value: "4.2.1" + + - it: refuses a Kafka 3.x version the operator no longer runs + release: + name: test-kafka + namespace: tenant-test + set: + version: "v3.9" + asserts: + - failedTemplate: + errorPattern: "value must be one of 'v4.3', 'v4.2'" ########################## # No ZooKeeper / node-pool managed fields # @@ -283,3 +293,14 @@ tests: - equal: path: spec.kafka.metricsConfig.valueFrom.configMapKeyRef.key value: kafka-metrics-config.yml + + # Strimzi 1.x Kafka nodes read their certificates from Secrets through the + # Kubernetes API, which tenant network policy opens only to labelled pods. + - it: lets the Kafka pods reach the Kubernetes API + release: + name: test-kafka + namespace: tenant-test + asserts: + - equal: + path: spec.kafka.template.pod.metadata.labels["policy.cozystack.io/allow-to-apiserver"] + value: "true" diff --git a/packages/apps/kafka/tests/migration-hook_test.yaml b/packages/apps/kafka/tests/migration-hook_test.yaml index 36255fab4..5b1daff11 100644 --- a/packages/apps/kafka/tests/migration-hook_test.yaml +++ b/packages/apps/kafka/tests/migration-hook_test.yaml @@ -63,85 +63,9 @@ tests: value: test-kafka-kafka-migration documentIndex: 3 - - it: grants permission to patch kafkas and kafkanodepools - release: - name: test-kafka - namespace: tenant-test - asserts: - - contains: - path: rules[0].apiGroups - content: kafka.strimzi.io - documentIndex: 1 - - contains: - path: rules[0].resources - content: kafkas - documentIndex: 1 - - contains: - path: rules[0].resources - content: kafkanodepools - documentIndex: 1 - - contains: - path: rules[0].verbs - content: patch - documentIndex: 1 - - contains: - path: rules[0].verbs - content: create - documentIndex: 1 - # Name-freeing needs to drive rebalances, delete the drained pool, and - # clean up its PVCs. - - contains: - path: rules[0].resources - content: kafkarebalances - documentIndex: 1 - - contains: - path: rules[0].verbs - content: delete - documentIndex: 1 - - contains: - path: rules[1].resources - content: persistentvolumeclaims - documentIndex: 1 - - # When it holds the shared "kafka" pool and other ZooKeeper clusters remain, - # the hook drains all data onto its own "b-" pool via a Cruise Control - # KafkaRebalance and only then deletes the "kafka" pool — so the name frees for - # the next cluster without losing data. - - it: frees the kafka pool name by draining via a rebalance - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'other ZooKeeper Kafka cluster' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'kind: KafkaRebalance' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'mode: remove-brokers' - documentIndex: 3 - # A mandatory broker evacuation must not be blocked by capacity hard goals. - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'skipHardGoalCheck: true' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'next-node-ids: "\[1000-1099\]"' - documentIndex: 3 - # The pool is deleted only after the rebalance is Ready — never before. - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'Rebalance complete[\s\S]*delete "\$np" kafka' - documentIndex: 3 - # The hook creates the broker pool under the render-resolved name - # (kafka.brokerPoolName): "kafka" for a migrating ZooKeeper cluster, "b-" - # for a fresh one. The manifest references the shell var so both resolve here. + # (kafka.brokerPoolName): "kafka" for a cluster migrated from ZooKeeper, + # "b-" otherwise. The manifest references the shell var so both resolve here. - it: creates the broker node pool under the resolved pool name release: name: test-kafka @@ -168,73 +92,6 @@ tests: pattern: 'brokerPool="b-[0-9a-f]{8}"' documentIndex: 3 - # CRITICAL adoption invariant: a cluster migrating from ZooKeeper (an existing - # Kafka CR without kraft=enabled) must resolve the broker pool to exactly - # "kafka" so Strimzi reuses the existing -kafka-N brokers and PVCs. - - it: resolves the broker pool to kafka for a migrating ZooKeeper cluster - release: - name: test-kafka - namespace: tenant-test - kubernetesProvider: - scheme: - "kafka.strimzi.io/v1beta2/Kafka": - gvr: - group: kafka.strimzi.io - version: v1beta2 - resource: kafkas - namespaced: true - objects: - - apiVersion: kafka.strimzi.io/v1beta2 - kind: Kafka - metadata: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'brokerPool="kafka"' - documentIndex: 3 - - # Render-time serialization of co-namespaced migrations: while a "kafka" pool is - # owned by a DIFFERENT cluster that is migrating first, kafka.brokerPoolName - # fails this ZooKeeper cluster's render so its HelmRelease retries until the - # first cluster drains off the shared pool and frees the name (the hook's - # runtime create-CAS is the second line of defense). - - it: fails the render when a foreign cluster still owns the kafka pool - release: - name: test-kafka - namespace: tenant-test - kubernetesProvider: - scheme: - "kafka.strimzi.io/v1beta2/Kafka": - gvr: - group: kafka.strimzi.io - version: v1beta2 - resource: kafkas - namespaced: true - "kafka.strimzi.io/v1beta2/KafkaNodePool": - gvr: - group: kafka.strimzi.io - version: v1beta2 - resource: kafkanodepools - namespaced: true - objects: - - apiVersion: kafka.strimzi.io/v1beta2 - kind: Kafka - metadata: - name: test-kafka - namespace: tenant-test - - apiVersion: kafka.strimzi.io/v1beta2 - kind: KafkaNodePool - metadata: - name: kafka - namespace: tenant-test - labels: - strimzi.io/cluster: other-cluster - asserts: - - failedTemplate: - errorPattern: "broker pool owned by cluster .* migrating first" - # The controller pool the hook creates must use the short hashed name # kafka.controllerPoolName, in lockstep with every selector helper. - it: creates the controller node pool with the release-hashed name @@ -268,133 +125,6 @@ tests: pattern: 'strimzi.io/next-node-ids: "\[3-999\]"' documentIndex: 3 - # The migration Job must never apply strimzi.io/kraft=enabled unless the - # cluster actually reached KRaftPostMigration — flipping it earlier is unsafe - # and irreversible. Guard the finalize step behind an explicit readiness flag - # that aborts (exit 1) on timeout, so Helm fails and the next reconcile retries. - - it: guards the KRaft finalize step behind a reached-safe-state check - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "migration_ready=false" - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'if \[ "\$migration_ready" != "true" \]; then' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "Refusing to apply strimzi.io/kraft=enabled in an unsafe state" - documentIndex: 3 - - # Fail closed: a real read error (non-zero from the --ignore-not-found get) - # must abort, never be mistaken for a fresh install. - - it: fails closed on non-NotFound errors reading the Kafka CR - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'if ! found=\$\(kubectl .* --ignore-not-found -o name' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "ERROR: reading Kafka CR failed" - documentIndex: 3 - - # Idempotent: on a retry the cluster may already be past ZooKeeper; do not - # re-issue kraft=migration (an invalid backward transition Strimzi rejects). - - it: does not re-issue kraft=migration once past the ZooKeeper phase - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "KRaftMigration\\|KRaftDualWriting\\|KRaftPostMigration\\|PreKRaft\\|KRaft\\)" - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "not re-issuing kraft=migration" - documentIndex: 3 - - # ATOMIC one-migration-per-namespace guard: an absent "kafka" pool is created - # via `kubectl create` (the atomic arbiter of a concurrent race), and a "kafka" - # pool owned by another cluster is refused. - - it: guards the shared kafka pool name atomically via create, refusing a foreign owner - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'if \[ "\$brokerPool" = "kafka" \]; then' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'kubectl create -f -' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "already belongs to cluster" - documentIndex: 3 - - # The bare plural "kafkas" is ambiguous (also apps.cozystack.io); the hook must - # fully qualify the Strimzi resource or it can hit the tenant kind and Forbidden. - - it: fully qualifies the Strimzi Kafka and KafkaNodePool resources - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "kafka=kafkas\\.kafka\\.strimzi\\.io" - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "np=kafkanodepools\\.kafka\\.strimzi\\.io" - documentIndex: 3 - - notMatchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'annotate kafka "' - documentIndex: 3 - - # NotFound must be structural (--ignore-not-found), never a stderr substring - # match that a stray "... not found" error could trip into skipping a live ZK. - - it: classifies NotFound structurally, not by stderr substring - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "--ignore-not-found" - documentIndex: 3 - - notMatchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "grep -qiE.*not.*found" - documentIndex: 3 - - # A KRaft-native / already-migrated CR (kraft=enabled) must be a no-op, never - # driven backwards into kraft=migration. - - it: treats a kraft=enabled cluster as done - release: - name: test-kafka - namespace: tenant-test - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: 'if \[ "\$kraft" != "enabled" \] && \[ "\$state" != "KRaft" \]' - documentIndex: 3 - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "Kafka is already KRaft" - documentIndex: 3 - # Preserve failed-Job logs for a post-mortem: only before-hook-creation, never # hook-succeeded (which would delete the previous attempt's pod logs). - it: keeps only before-hook-creation on the Job hook-delete-policy @@ -452,11 +182,100 @@ tests: release: name: test-kafka namespace: tenant-test - documentIndex: 3 + documentSelector: + path: kind + value: Job asserts: - notMatchRegex: path: spec.template.spec.containers[0].args[0] pattern: "kubectl[^|\\n]* apply (-f|--filename|--server-side)" - matchRegex: path: spec.template.spec.containers[0].args[0] - pattern: "echo \"\\$broker_manifest\" \\| kubectl apply --openapi-patch=false -f -\\n[\\s\\S]*echo \"\\$controller_manifest\" \\| kubectl apply --openapi-patch=false -f -\\n[\\s\\S]*<KRaft pre-upgrade migration Job (packages/apps/kafka/templates/migration-hook.yaml) - # can run for hours: the migration itself is tens of minutes (broker rolling - # restarts through the Strimzi state machine), and when several Kafkas share a - # namespace the Job also drains the whole dataset off the shared "kafka" pool - # via a Cruise Control rebalance to free the name for the next cluster. - # helm-controller waits on the pre-upgrade hook within the release timeout - # (default 10m — far below this). Raise it above the Job's - # activeDeadlineSeconds (21600s = 6h) so a real drain is not aborted and - # thrashed. Keep this above that Job deadline if either changes. - release.cozystack.io/helm-install-timeout: "7h" spec: application: kind: Kafka plural: kafkas singular: kafka openAPISchema: |- - {"title":"Chart Values","type":"object","x-cozystack-name":{"type":"string","description":"Kafka cluster name. The KRaft controller pods are named `kafka--c--` with an 8-character hash, and that pod hostname must fit the 63-character DNS-1123 label limit even for the smallest cluster.","maxLength":44},"properties":{"external":{"description":"Enable external access from outside the cluster.","type":"boolean","default":false},"tls":{"description":"TLS configuration. Strimzi manages the cluster PKI automatically (no cert-manager is involved for this chart): the operator auto-creates `-cluster-ca-cert` and `-clients-ca-cert` secrets, both exposed for client trust setup. The internal TLS listener on 9093 is always on; this toggle only controls the external listener on 9094.","type":"object","default":{},"properties":{"enabled":{"description":"Enable TLS on the external listener. When unset, inherits the value of `external` (TLS is on when external access is enabled). Warning: setting this to false while external is true exposes Kafka over plaintext on a public IP via LoadBalancer. Strimzi does not provide authentication on this listener unless SCRAM, mTLS, or OAuth is separately configured. Use only in controlled networks.","type":"boolean"}}},"version":{"description":"Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v3.9 back to v3.8) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it.","type":"string","default":"v3.9","enum":["v3.9","v3.8"]},"topics":{"description":"Topics configuration.","type":"array","default":[],"items":{"type":"object","required":["name","partitions","replicas"],"properties":{"config":{"description":"Topic configuration.","type":"object","x-kubernetes-preserve-unknown-fields":true},"name":{"description":"Topic name.","type":"string"},"partitions":{"description":"Number of partitions.","type":"integer"},"replicas":{"description":"Number of replicas.","type":"integer"}}}},"kafka":{"description":"Kafka configuration.","type":"object","default":{},"required":["replicas","resourcesPreset","size","storageClass"],"properties":{"controllerStorageSize":{"description":"Persistent Volume size for KRaft controller metadata (used during ZK-to-KRaft migration).","default":"5Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"replicas":{"description":"Number of Kafka replicas.","type":"integer","default":3},"resources":{"description":"Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.","type":"object","default":{},"properties":{"cpu":{"description":"CPU available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"memory":{"description":"Memory (RAM) available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}},"resourcesPreset":{"description":"Default sizing preset used when `resources` is omitted.","type":"string","default":"c1.small","enum":["t1.nano","t1.micro","t1.small","t1.medium","t1.large","t1.xlarge","t1.2xlarge","t1.4xlarge","c1.nano","c1.micro","c1.small","c1.medium","c1.large","c1.xlarge","c1.2xlarge","c1.4xlarge","s1.nano","s1.micro","s1.small","s1.medium","s1.large","s1.xlarge","s1.2xlarge","s1.4xlarge","u1.nano","u1.micro","u1.small","u1.medium","u1.large","u1.xlarge","u1.2xlarge","u1.4xlarge","m1.nano","m1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m1.2xlarge","m1.4xlarge","nano","micro","small","medium","large","xlarge","2xlarge"]},"size":{"description":"Persistent Volume size for Kafka.","default":"10Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"storageClass":{"description":"StorageClass used to store the Kafka data.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}}}}}} + {"title":"Chart Values","type":"object","x-cozystack-name":{"type":"string","description":"Kafka cluster name. The KRaft controller pods are named `kafka--c--` with an 8-character hash, and that pod hostname must fit the 63-character DNS-1123 label limit even for the smallest cluster.","maxLength":44},"properties":{"external":{"description":"Enable external access from outside the cluster.","type":"boolean","default":false},"tls":{"description":"TLS configuration. Strimzi manages the cluster PKI automatically (no cert-manager is involved for this chart): the operator auto-creates `-cluster-ca-cert` and `-clients-ca-cert` secrets, both exposed for client trust setup. The internal TLS listener on 9093 is always on; this toggle only controls the external listener on 9094.","type":"object","default":{},"properties":{"enabled":{"description":"Enable TLS on the external listener. When unset, inherits the value of `external` (TLS is on when external access is enabled). Warning: setting this to false while external is true exposes Kafka over plaintext on a public IP via LoadBalancer. Strimzi does not provide authentication on this listener unless SCRAM, mTLS, or OAuth is separately configured. Use only in controlled networks.","type":"boolean"}}},"version":{"description":"Kafka version to deploy. Upgrade-only: once a cluster's KRaft metadata is at a given version, Strimzi refuses an in-place downgrade, so lowering this on a running cluster (e.g. v4.3 back to v4.2) leaves the Kafka CR stuck in a reconcile error. Pick the target version at creation and only ever raise it.","type":"string","default":"v4.3","enum":["v4.3","v4.2"]},"topics":{"description":"Topics configuration.","type":"array","default":[],"items":{"type":"object","required":["name","partitions","replicas"],"properties":{"config":{"description":"Topic configuration.","type":"object","x-kubernetes-preserve-unknown-fields":true},"name":{"description":"Topic name.","type":"string"},"partitions":{"description":"Number of partitions.","type":"integer"},"replicas":{"description":"Number of replicas.","type":"integer"}}}},"kafka":{"description":"Kafka configuration.","type":"object","default":{},"required":["replicas","resourcesPreset","size","storageClass"],"properties":{"controllerStorageSize":{"description":"Persistent Volume size for KRaft controller metadata.","default":"5Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"replicas":{"description":"Number of Kafka replicas.","type":"integer","default":3},"resources":{"description":"Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.","type":"object","default":{},"properties":{"cpu":{"description":"CPU available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"memory":{"description":"Memory (RAM) available to each replica.","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true}}},"resourcesPreset":{"description":"Default sizing preset used when `resources` is omitted.","type":"string","default":"c1.small","enum":["t1.nano","t1.micro","t1.small","t1.medium","t1.large","t1.xlarge","t1.2xlarge","t1.4xlarge","c1.nano","c1.micro","c1.small","c1.medium","c1.large","c1.xlarge","c1.2xlarge","c1.4xlarge","s1.nano","s1.micro","s1.small","s1.medium","s1.large","s1.xlarge","s1.2xlarge","s1.4xlarge","u1.nano","u1.micro","u1.small","u1.medium","u1.large","u1.xlarge","u1.2xlarge","u1.4xlarge","m1.nano","m1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m1.2xlarge","m1.4xlarge","nano","micro","small","medium","large","xlarge","2xlarge"]},"size":{"description":"Persistent Volume size for Kafka.","default":"10Gi","pattern":"^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$","anyOf":[{"type":"integer"},{"type":"string"}],"x-kubernetes-int-or-string":true},"storageClass":{"description":"StorageClass used to store the Kafka data.","type":"string","default":"","x-kubernetes-validations":[{"rule":"self == oldSelf","message":"storageClass is immutable"}],"x-cozystack-options":{"source":"storageclass"}}}}}} release: prefix: kafka- labels: From 641faa7fde07b00a775ce42b3c7f8747b056ae2f Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 13 Oct 2026 05:52:37 +0300 Subject: [PATCH 4/9] build(kafka-operator): vendor Strimzi 1.2.0 Strimzi 0.45 is out of upstream support. Strimzi 1.x serves only the v1 API, so the release now replaces the CRDs on upgrade; helm-controller's default of skipping CRD upgrades would leave the v1beta2-only CRDs from the first install in place. Strimzi 1.x runs no ZooKeeper, so the ZooKeeper alert group goes too. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- .../core/platform/sources/kafka-operator.yaml | 6 + .../sources_kafka_operator_crds_test.yaml | 18 + packages/system/kafka-operator/Makefile | 2 +- .../alerts/prometheus-rules.yaml | 42 - .../charts/strimzi-kafka-operator/Chart.yaml | 16 +- .../charts/strimzi-kafka-operator/OWNERS | 20 - .../charts/strimzi-kafka-operator/README.md | 301 +- .../crds/040-Crd-kafka.yaml | 3541 ++--------------- .../crds/041-Crd-kafkaconnect.yaml | 600 ++- .../crds/042-Crd-strimzipodset.yaml | 4 +- .../crds/043-Crd-kafkatopic.yaml | 218 +- .../crds/044-Crd-kafkauser.yaml | 501 +-- .../crds/045-Crd-kafkamirrormaker.yaml | 1584 -------- ...depool.yaml => 045-Crd-kafkanodepool.yaml} | 167 +- .../crds/046-Crd-kafkabridge.yaml | 339 +- .../crds/047-Crd-kafkaconnector.yaml | 12 +- .../crds/048-Crd-kafkamirrormaker2.yaml | 891 ++--- .../crds/049-Crd-kafkarebalance.yaml | 11 +- .../strimzi-cruise-control.json | 12 +- .../strimzi-kafka-bridge.json | 9 +- .../strimzi-kafka-connect.json | 3 +- .../strimzi-kafka-exporter.json | 1 + .../strimzi-kafka-mirror-maker-2.json | 1492 +++---- .../strimzi-kafka-oauth.json | 13 +- .../grafana-dashboards/strimzi-kafka.json | 118 +- .../grafana-dashboards/strimzi-kraft.json | 1 + .../grafana-dashboards/strimzi-operators.json | 21 +- .../grafana-dashboards/strimzi-zookeeper.json | 1412 ------- ...terRole-strimzi-cluster-operator-role.yaml | 23 +- ...-RoleBinding-strimzi-cluster-operator.yaml | 32 +- ...terRole-strimzi-cluster-operator-role.yaml | 24 +- ...-RoleBinding-strimzi-cluster-operator.yaml | 30 +- ...1-ClusterRole-strimzi-entity-operator.yaml | 8 + ...r-operator-entity-operator-delegation.yaml | 28 +- ...50-ConfigMap-strimzi-cluster-operator.yaml | 4 - ...uptionBudget-strimzi-cluster-operator.yaml | 5 +- ...etworkPolicy-strimzi-cluster-operator.yaml | 21 + ...0-Deployment-strimzi-cluster-operator.yaml | 60 +- .../070-ClusterRole-strimzi-admin.yaml | 2 - .../080-ClusterRole-strimzi-view.yaml | 1 - .../templates/_helpers.tpl | 9 + .../templates/_kafka_image_map.tpl | 34 +- .../charts/strimzi-kafka-operator/values.yaml | 72 +- 43 files changed, 2738 insertions(+), 8970 deletions(-) create mode 100644 packages/core/platform/tests/sources_kafka_operator_crds_test.yaml delete mode 100644 packages/system/kafka-operator/charts/strimzi-kafka-operator/OWNERS delete mode 100644 packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkamirrormaker.yaml rename packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/{04A-Crd-kafkanodepool.yaml => 045-Crd-kafkanodepool.yaml} (88%) delete mode 100644 packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-zookeeper.json create mode 100644 packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/052-NetworkPolicy-strimzi-cluster-operator.yaml diff --git a/packages/core/platform/sources/kafka-operator.yaml b/packages/core/platform/sources/kafka-operator.yaml index 83d7d5023..a2fb60193 100644 --- a/packages/core/platform/sources/kafka-operator.yaml +++ b/packages/core/platform/sources/kafka-operator.yaml @@ -20,3 +20,9 @@ spec: install: namespace: cozy-kafka-operator releaseName: kafka-operator + # helm-controller defaults CRD upgrades to Skip, which would leave the + # Strimzi CRDs at whatever version first installed them. Replacing them + # with v1-only CRDs requires every stored object to be v1 already; the + # platform migration that converts them runs before this release on the + # variants that run platform migrations. + upgradeCRDs: CreateReplace diff --git a/packages/core/platform/tests/sources_kafka_operator_crds_test.yaml b/packages/core/platform/tests/sources_kafka_operator_crds_test.yaml new file mode 100644 index 000000000..b0ac30839 --- /dev/null +++ b/packages/core/platform/tests/sources_kafka_operator_crds_test.yaml @@ -0,0 +1,18 @@ +suite: kafka-operator applies CRD updates on upgrade +# Strimzi 1.x serves only the v1 API. Helm never upgrades CRDs shipped in +# crds/, so without CreateReplace an upgraded cluster keeps the v1beta2-only +# CRDs of its first install and the 1.x operator cannot work with them. +templates: + - templates/sources.yaml +release: + name: cozystack + namespace: cozy-system +tests: + - it: kafka-operator PackageSource sets upgradeCRDs to CreateReplace + documentSelector: + path: metadata.name + value: cozystack.kafka-operator + asserts: + - equal: + path: spec.variants[0].components[0].install.upgradeCRDs + value: CreateReplace diff --git a/packages/system/kafka-operator/Makefile b/packages/system/kafka-operator/Makefile index 30c35e912..fb8122538 100644 --- a/packages/system/kafka-operator/Makefile +++ b/packages/system/kafka-operator/Makefile @@ -8,4 +8,4 @@ test: update: rm -rf charts - helm pull oci://quay.io/strimzi-helm/strimzi-kafka-operator --untar --untardir charts --version 0.45.1-rc1 + helm pull oci://quay.io/strimzi-helm/strimzi-kafka-operator --untar --untardir charts --version 1.2.0 diff --git a/packages/system/kafka-operator/alerts/prometheus-rules.yaml b/packages/system/kafka-operator/alerts/prometheus-rules.yaml index 760ca5dad..2ef8226bd 100644 --- a/packages/system/kafka-operator/alerts/prometheus-rules.yaml +++ b/packages/system/kafka-operator/alerts/prometheus-rules.yaml @@ -89,48 +89,6 @@ spec: annotations: summary: 'One or more Kafka containers restarted too often' description: 'One or more Kafka containers were restarted too often within the last 5 minutes' - - name: zookeeper - rules: - - alert: AvgRequestLatency - expr: zookeeper_avgrequestlatency > 10 - for: 10s - labels: - severity: warning - annotations: - summary: 'Zookeeper average request latency' - description: 'The average request latency is {{ $value }} on {{ $labels.pod }}' - - alert: OutstandingRequests - expr: zookeeper_outstandingrequests > 10 - for: 10s - labels: - severity: warning - annotations: - summary: 'Zookeeper outstanding requests' - description: 'There are {{ $value }} outstanding requests on {{ $labels.pod }}' - - alert: ZookeeperRunningOutOfSpace - expr: kubelet_volume_stats_available_bytes{persistentvolumeclaim=~"data-(.+)-zookeeper-[0-9]+"} < 5368709120 - for: 10s - labels: - severity: warning - annotations: - summary: 'Zookeeper is running out of free disk space' - description: 'There are only {{ $value }} bytes available at {{ $labels.persistentvolumeclaim }} PVC' - - alert: ZookeeperContainerRestartedInTheLast5Minutes - expr: count(count_over_time(container_last_seen{container="zookeeper"}[5m])) > 2 * count(container_last_seen{container="zookeeper",pod=~".+-zookeeper-[0-9]+"}) - for: 5m - labels: - severity: warning - annotations: - summary: 'One or more Zookeeper containers were restarted too often' - description: 'One or more Zookeeper containers were restarted too often within the last 5 minutes. This alert can be ignored when the Zookeeper cluster is scaling up' - - alert: ZookeeperContainersDown - expr: absent(container_last_seen{container="zookeeper",pod=~".+-zookeeper-[0-9]+"}) - for: 3m - labels: - severity: major - annotations: - summary: 'All `zookeeper` containers in the Zookeeper pods down or in CrashLookBackOff status' - description: 'All `zookeeper` containers in the Zookeeper pods have been down or in CrashLookBackOff status for 3 minutes' - name: entityOperator rules: - alert: TopicOperatorContainerDown diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/Chart.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/Chart.yaml index 50ee9ff77..25b6afc8b 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/Chart.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/Chart.yaml @@ -1,5 +1,5 @@ apiVersion: v2 -appVersion: 0.45.1-rc1 +appVersion: 1.2.0 description: 'Strimzi: Apache Kafka running on Kubernetes' home: https://strimzi.io/ icon: https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/main/documentation/logo/strimzi_logo.png @@ -12,16 +12,10 @@ keywords: - datastore - topic maintainers: -- name: Frawless -- name: ppatierno -- name: samuel-hawker -- name: scholzj -- name: tombentley -- name: sknot-rh -- name: PaulRMellor -- name: im-konge -- name: see-quick +- email: cncf-strimzi-maintainers@lists.cncf.io + name: Strimzi Project Maintainers + url: https://github.com/strimzi/governance name: strimzi-kafka-operator sources: - https://github.com/strimzi/strimzi-kafka-operator -version: 0.45.1-rc1 +version: 1.2.0 diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/OWNERS b/packages/system/kafka-operator/charts/strimzi-kafka-operator/OWNERS deleted file mode 100644 index 1a5c3e14d..000000000 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/OWNERS +++ /dev/null @@ -1,20 +0,0 @@ -approvers: -- ppatierno -- scholzj -- tombentley -- samuel-hawker -- Frawless -- sknot-rh -- PaulRMellor -- im-konge -- see-quick -reviewers: -- ppatierno -- scholzj -- tombentley -- samuel-hawker -- Frawless -- sknot-rh -- PaulRMellor -- im-konge -- see-quick \ No newline at end of file diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/README.md b/packages/system/kafka-operator/charts/strimzi-kafka-operator/README.md index 7036760ee..2ef5cb42b 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/README.md +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/README.md @@ -6,14 +6,19 @@ See our [website](https://strimzi.io) for more details about the project. **!!! IMPORTANT !!!** -* **Strimzi 0.45 is the last Strimzi version with support for ZooKeeper-based Apache Kafka clusters and MirrorMaker 1 deployments.** - **Please make sure to [migrate to KRaft](https://strimzi.io/docs/operators/latest/full/deploying.html#assembly-kraft-mode-str) and MirrorMaker 2 before upgrading to Strimzi 0.46 or newer.** -* Strimzi 0.45 is the last Strimzi version to include the [Strimzi EnvVar Configuration Provider](https://github.com/strimzi/kafka-env-var-config-provider) (deprecated in Strimzi 0.38.0) and [Strimzi MirrorMaker 2 Extensions](https://github.com/strimzi/mirror-maker-2-extensions) (deprecated in Strimzi 0.28.0). - Please use the Apache Kafka [EnvVarConfigProvider](https://github.com/strimzi/kafka-env-var-config-provider?tab=readme-ov-file#deprecation-notice) and [Identity Replication Policy](https://github.com/strimzi/mirror-maker-2-extensions?tab=readme-ov-file#identity-replication-policy) instead. -* From Strimzi 0.44.0 on, we support only Kubernetes 1.25 and newer. - Kubernetes 1.23 and 1.24 are not supported anymore. -* Upgrading to Strimzi 0.32 and newer directly from Strimzi 0.22 and earlier is no longer possible. - Please follow the [documentation](https://strimzi.io/docs/operators/latest/full/deploying.html#assembly-upgrade-str) for more details. +**From Strimzi 1.2.0, the Helm Chart uses the default container security context that matches the [Restricted Kubernetes Pod Security Standard](https://kubernetes.io/docs/concepts/security/pod-security-standards/).** +```yaml +securityContext: + allowPrivilegeEscalation: false + runAsNonRoot: true + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL + seccompProfile: + type: RuntimeDefault +``` +**You can override the default security context by setting the `securityContext` parameter in the Helm values.** ## Introduction @@ -40,13 +45,18 @@ cluster using the [Helm](https://helm.sh) package manager. ### Upgrading your Clusters To upgrade the Strimzi operator, you can use the `helm upgrade` command. -The `helm upgrade` command does not upgrade the [Custom Resource Definitions](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/). -Install the new CRDs manually after upgrading the Cluster Operator. +**The `helm upgrade` command does not upgrade the [Custom Resource Definitions](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/).** +**Update the CRDs manually after upgrading the Cluster Operator.** You can access the CRDs from our [GitHub release page](https://github.com/strimzi/strimzi-kafka-operator/releases) or find them in the `crd` subdirectory inside the Helm Chart. +For example, when upgrading to Strimzi 0.50.0, you can do: + +```bash +kubectl apply -f https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.50.0/strimzi-crds-0.50.0.yaml +``` The Strimzi Operator understands how to run and upgrade between a set of Kafka versions. When specifying a new version in your config, check to make sure you aren't using any features that may have been removed. -See [the upgrade guide](https://strimzi.io/docs/operators/latest/deploying.html#assembly-upgrading-kafka-versions-str) for more information. +See [the upgrade guide](https://strimzi.io/docs/operators/latest/deploying.html#proc-upgrade-kafka-kraft-str) for more information. ### Documentation @@ -65,14 +75,14 @@ Strimzi is licensed under the [Apache License, Version 2.0](https://github.com/s ## Prerequisites -- Kubernetes 1.25+ +- Kubernetes 1.30+ ## Installing the Chart To install the chart with the release name `my-strimzi-cluster-operator`: ```bash -$ helm install my-strimzi-cluster-operator oci://quay.io/strimzi-helm/strimzi-kafka-operator +helm install my-strimzi-cluster-operator oci://quay.io/strimzi-helm/strimzi-kafka-operator ``` The command deploys the Strimzi Cluster Operator on the Kubernetes cluster with the default configuration. @@ -83,144 +93,151 @@ The [configuration](#configuration) section lists the parameters that can be con To uninstall/delete the `my-strimzi-cluster-operator` deployment: ```bash -$ helm delete my-strimzi-cluster-operator +helm delete my-strimzi-cluster-operator ``` The command removes all the Kubernetes components associated with the operator and deletes the release. ## Configuration -The following table lists the configurable parameters of the Strimzi chart and their default values. Runtime -configuration of Kafka and other components are defined within their respective Custom Resource Definitions. See -the documentation for more details. +The following table lists the configurable parameters of the Strimzi chart and their default values. +Runtime configuration of Kafka and other components are defined within their respective Custom Resource Definitions. +See the documentation for more details. -| Parameter | Description | Default | -|---------------------------------------------|---------------------------------------------------------------------------------|------------------------------| -| `replicas` | Number of replicas of the cluster operator | 1 | -| `revisionHistoryLimit` | Number of replicaSet to keep of the operator deployment | 10 | -| `watchNamespaces` | Comma separated list of additional namespaces for the strimzi-operator to watch | [] | -| `watchAnyNamespace` | Watch the whole Kubernetes cluster (all namespaces) | `false` | -| `defaultImageRegistry` | Default image registry for all the images | `quay.io` | -| `defaultImageRepository` | Default image registry for all the images | `strimzi` | -| `defaultImageTag` | Default image tag for all the images except Kafka Bridge | `0.45.1-rc1` | -| `image.registry` | Override default Cluster Operator image registry | `nil` | -| `image.repository` | Override default Cluster Operator image repository | `nil` | -| `image.name` | Cluster Operator image name | `cluster-operator` | -| `image.tag` | Override default Cluster Operator image tag | `nil` | -| `image.digest` | Override Cluster Operator image tag with digest | `nil` | -| `image.imagePullPolicy` | Image pull policy for all pods deployed by Cluster Operator | `IfNotPresent` | -| `image.imagePullSecrets` | List of Docker registry pull secrets | `[]` | -| `fullReconciliationIntervalMs` | Full reconciliation interval in milliseconds | 120000 | -| `leaderElection.enable` | Whether to enable leader election | `true` | -| `operationTimeoutMs` | Operation timeout in milliseconds | 300000 | -| `operatorNamespaceLabels` | Labels of the namespace where the operator runs | `nil` | -| `podSecurityContext` | Cluster Operator pod's security context | `nil` | -| `priorityClassName` | Cluster Operator pod's priority class name | `nil` | -| `securityContext` | Cluster Operator container's security context | `nil` | -| `rbac.create` | Whether to create RBAC related resources | `yes` | -| `serviceAccountCreate` | Whether to create a service account | `yes` | -| `serviceAccount` | Cluster Operator's service account | `strimzi-cluster-operator` | -| `podDisruptionBudget.enabled` | Whether to enable the podDisruptionBudget feature | `false` | -| `podDisruptionBudget.minAvailable` | Default value for how many pods must be running in a cluster | `1` | -| `podDisruptionBudget.maxUnavailable` | Default value for how many pods can be down | `nil` | -| `extraEnvs` | Extra environment variables for the Cluster operator container | `[]` | -| `kafka.image.registry` | Override default Kafka image registry | `nil` | -| `kafka.image.repository` | Override default Kafka image repository | `nil` | -| `kafka.image.name` | Kafka image name | `kafka` | -| `kafka.image.tagPrefix` | Override default Kafka image tag prefix | `nil` | -| `kafka.image.tag` | Override default Kafka image tag and ignore suffix | `nil` | -| `kafka.image.digest` | Override Kafka image tag with digest | `nil` | -| `kafkaConnect.image.registry` | Override default Kafka Connect image registry | `nil` | -| `kafkaConnect.image.repository` | Override default Kafka Connect image repository | `nil` | -| `kafkaConnect.image.name` | Kafka Connect image name | `kafka` | -| `kafkaConnect.image.tagPrefix` | Override default Kafka Connect image tag prefix | `nil` | -| `kafkaConnect.image.tag` | Override default Kafka Connect image tag and ignore suffix | `nil` | -| `kafkaConnect.image.digest` | Override Kafka Connect image tag with digest | `nil` | -| `kafkaMirrorMaker.image.registry` | Override default Kafka Mirror Maker image registry | `nil` | -| `kafkaMirrorMaker.image.repository` | Override default Kafka Mirror Maker image repository | `nil` | -| `kafkaMirrorMaker.image.name` | Kafka Mirror Maker image name | `kafka` | -| `kafkaMirrorMaker.image.tagPrefix` | Override default Kafka Mirror Maker image tag prefix | `nil` | -| `kafkaMirrorMaker.image.tag` | Override default Kafka Mirror Maker image tag and ignore suffix | `nil` | -| `kafkaMirrorMaker.image.digest` | Override Kafka Mirror Maker image tag with digest | `nil` | -| `cruiseControl.image.registry` | Override default Cruise Control image registry | `nil` | -| `cruiseControl.image.repository` | Override default Cruise Control image repository | `nil` | -| `cruiseControl.image.name` | Cruise Control image name | `kafka` | -| `cruiseControl.image.tagPrefix` | Override default Cruise Control image tag prefix | `nil` | -| `cruiseControl.image.tag` | Override default Cruise Control image tag and ignore suffix | `nil` | -| `cruiseControl.image.digest` | Override Cruise Control image tag with digest | `nil` | -| `topicOperator.image.registry` | Override default Topic Operator image registry | `nil` | -| `topicOperator.image.repository` | Override default Topic Operator image repository | `nil` | -| `topicOperator.image.name` | Topic Operator image name | `operator` | -| `topicOperator.image.tag` | Override default Topic Operator image tag | `nil` | -| `topicOperator.image.digest` | Override Topic Operator image tag with digest | `nil` | -| `userOperator.image.registry` | Override default User Operator image registry | `nil` | -| `userOperator.image.repository` | Override default User Operator image repository | `nil` | -| `userOperator.image.name` | User Operator image name | `operator` | -| `userOperator.image.tag` | Override default User Operator image tag | `nil` | -| `userOperator.image.digest` | Override User Operator image tag with digest | `nil` | -| `kafkaInit.image.registry` | Override default Init Kafka image registry | `nil` | -| `kafkaInit.image.repository` | Override default Init Kafka image repository | `nil` | -| `kafkaInit.image.name` | Init Kafka image name | `operator` | -| `kafkaInit.image.tag` | Override default Init Kafka image tag | `nil` | -| `kafkaInit.image.digest` | Override Init Kafka image tag with digest | `nil` | -| `kafkaBridge.image.registry` | Override default Kafka Bridge image registry | `quay.io` | -| `kafkaBridge.image.repository` | Override default Kafka Bridge image repository | `strimzi` | -| `kafkaBridge.image.name` | Kafka Bridge image name | `kafka-bridge` | -| `kafkaBridge.image.tag` | Override default Kafka Bridge image tag | `0.31.2` | -| `kafkaBridge.image.digest` | Override Kafka Bridge image tag with digest | `nil` | -| `kafkaExporter.image.registry` | Override default Kafka Exporter image registry | `nil` | -| `kafkaExporter.image.repository` | Override default Kafka Exporter image repository | `nil` | -| `kafkaExporter.image.name` | Kafka Exporter image name | `kafka` | -| `kafkaExporter.image.tagPrefix` | Override default Kafka Exporter image tag prefix | `nil` | -| `kafkaExporter.image.tag` | Override default Kafka Exporter image tag and ignore suffix | `nil` | -| `kafkaExporter.image.digest` | Override Kafka Exporter image tag with digest | `nil` | -| `kafkaMirrorMaker2.image.registry` | Override default Kafka Mirror Maker 2 image registry | `nil` | -| `kafkaMirrorMaker2.image.repository` | Override default Kafka Mirror Maker 2 image repository | `nil` | -| `kafkaMirrorMaker2.image.name` | Kafka Mirror Maker 2 image name | `kafka` | -| `kafkaMirrorMaker2.image.tagPrefix` | Override default Kafka Mirror Maker 2 image tag prefix | `nil` | -| `kafkaMirrorMaker2.image.tag` | Override default Kafka Mirror Maker 2 image tag and ignore suffix | `nil` | -| `kafkaMirrorMaker2.image.digest` | Override Kafka Mirror Maker 2 image tag with digest | `nil` | -| `kanikoExecutor.image.registry` | Override default Kaniko Executor image registry | `nil` | -| `kanikoExecutor.image.repository` | Override default Kaniko Executor image repository | `nil` | -| `kanikoExecutor.image.name` | Kaniko Executor image name | `kaniko-executor` | -| `kanikoExecutor.image.tag` | Override default Kaniko Executor image tag | `nil` | -| `kanikoExecutor.image.digest` | Override Kaniko Executor image tag with digest | `nil` | -| `resources.limits.memory` | Memory constraint for limits | `384Mi` | -| `resources.limits.cpu` | CPU constraint for limits | `1000m` | -| `resources.requests.memory` | Memory constraint for requests | `384Mi` | -| `livenessProbe.initialDelaySeconds` | Liveness probe initial delay in seconds | 10 | -| `livenessProbe.periodSeconds` | Liveness probe period in seconds | 30 | -| `readinessProbe.initialDelaySeconds` | Readiness probe initial delay in seconds | 10 | -| `readinessProbe.periodSeconds` | Readiness probe period in seconds | 30 | -| `imageTagOverride` | Override all image tag config | `nil` | -| `createGlobalResources` | Allow creation of cluster-scoped resources | `true` | -| `createAggregateRoles` | Create cluster roles that extend aggregated roles to use Strimzi CRDs | `false` | -| `tolerations` | Add tolerations to Operator Pod | `[]` | -| `affinity` | Add affinities to Operator Pod | `{}` | -| `annotations` | Add annotations to Operator Pod | `{}` | -| `labels` | Add labels to Operator Pod | `{}` | -| `nodeSelector` | Add a node selector to Operator Pod | `{}` | -| `featureGates` | Feature Gates configuration | `nil` | -| `tmpDirSizeLimit` | Set the `sizeLimit` for the tmp dir volume used by the operator | `1Mi` | -| `labelsExclusionPattern` | Override the exclude pattern for exclude some labels | `""` | -| `generateNetworkPolicy` | Controls whether Strimzi generates network policy resources | `true` | -| `connectBuildTimeoutMs` | Overrides the default timeout value for building new Kafka Connect | `300000` | -| `mavenBuilder.image.registry` | Override default Maven Builder image registry | `nil` | -| `mavenBuilder.image.repository` | Maven Builder image repository | `nil` | -| `mavenBuilder.image.name` | Override default Maven Builder image name | `maven-builder` | -| `mavenBuilder.image.tag` | Override default Maven Builder image tag | `nil` | -| `mavenBuilder.image.digest` | Override Maven Builder image tag with digest | `nil` | -| `logConfiguration` | Override default `log4j.properties` content | `nil` | -| `logLevel` | Override default logging level | `INFO` | -| `dashboards.enable` | Generate configmaps containing the dashboards | `false` | -| `dashboards.label` | How should the dashboards be labeled for the sidecar | `grafana_dashboard` | -| `dashboards.labelValue` | What should the dashboards label value be for the sidecar | `"1"` | -| `dashboards.extraLabels` | Any additional labels you would like on the dashboards | `{}` | -| `dashboards.namespace` | What namespace should the dashboards be loaded into | `Follows toplevel Namespace` | -| `dashboards.annotations` | Any custom annotations (such as folder for the sidecar) | `{}` | +| Parameter | Description | Default | +|--------------------------------------------------|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------| +| `replicas` | Number of replicas of the cluster operator | 1 | +| `revisionHistoryLimit` | Number of replicaSet to keep of the operator deployment | 10 | +| `watchNamespaces` | Comma separated list of additional namespaces for the strimzi-operator to watch | [] | +| `watchAnyNamespace` | Watch the whole Kubernetes cluster (all namespaces) | `false` | +| `defaultImageRegistry` | Default image registry for all the images | `quay.io` | +| `defaultImageRepository` | Default image registry for all the images | `strimzi` | +| `defaultImageTag` | Default image tag for all the images except Kafka Bridge | `1.2.0` | +| `deploymentAnnotations` | Annotations for the operator deployment | `{}` | +| `deploymentLabels` | Labels for the operator deployment | `{}` | +| `image.registry` | Override default Cluster Operator image registry | `nil` | +| `image.repository` | Override default Cluster Operator image repository | `nil` | +| `image.name` | Cluster Operator image name | `operator` | +| `image.tag` | Override default Cluster Operator image tag | `nil` | +| `image.digest` | Override Cluster Operator image tag with digest | `nil` | +| `image.imagePullPolicy` | Image pull policy for all pods deployed by Cluster Operator | `IfNotPresent` | +| `image.imagePullSecrets` | List of Docker registry pull secrets | `[]` | +| `fullReconciliationIntervalMs` | Full reconciliation interval in milliseconds | 120000 | +| `leaderElection.enable` | Whether to enable leader election | `true` | +| `operationTimeoutMs` | Operation timeout in milliseconds | 300000 | +| `operatorNamespaceLabels` | Labels of the namespace where the operator runs | `nil` | +| `podSecurityContext` | Cluster Operator pod's security context | `nil` | +| `deploymentStrategy` | Adjust the Kubernetes rollout strategy of the Cluster Operator Deployment | `nil` | +| `priorityClassName` | Cluster Operator pod's priority class name | `nil` | +| `hostUsers` | Set hostUsers value on the Cluster Operator container | `nil` | +| `enableServiceLinks` | Set enableServiceLinks value on the Cluster Operator pod | `nil` | +| `securityContext` | Cluster Operator container's security context | [Restricted Kubernetes Pod Security Standard](https://kubernetes.io/docs/concepts/security/pod-security-standards/) | +| `rbac.create` | Whether to create RBAC related resources | `yes` | +| `serviceAccountCreate` | Whether to create a service account | `yes` | +| `serviceAccount` | Cluster Operator's service account | `strimzi-cluster-operator` | +| `podDisruptionBudget.enabled` | Whether to enable the podDisruptionBudget feature | `false` | +| `podDisruptionBudget.minAvailable` | Default value for how many pods must be running in a cluster | `1` | +| `podDisruptionBudget.maxUnavailable` | Default value for how many pods can be down | `nil` | +| `podDisruptionBudget.unhealthyPodEvictionPolicy` | Default value for how to respond to unheathly pods | `IfHealthyBudget` | +| `extraEnvs` | Extra environment variables for the Cluster operator container | `[]` | +| `kafka.image.registry` | Override default Kafka image registry | `nil` | +| `kafka.image.repository` | Override default Kafka image repository | `nil` | +| `kafka.image.name` | Kafka image name | `kafka` | +| `kafka.image.tagPrefix` | Override default Kafka image tag prefix | `nil` | +| `kafka.image.tag` | Override default Kafka image tag and ignore suffix | `nil` | +| `kafka.image.digest` | Override Kafka image tag with digest | `nil` | +| `kafkaConnect.image.registry` | Override default Kafka Connect image registry | `nil` | +| `kafkaConnect.image.repository` | Override default Kafka Connect image repository | `nil` | +| `kafkaConnect.image.name` | Kafka Connect image name | `kafka` | +| `kafkaConnect.image.tagPrefix` | Override default Kafka Connect image tag prefix | `nil` | +| `kafkaConnect.image.tag` | Override default Kafka Connect image tag and ignore suffix | `nil` | +| `kafkaConnect.image.digest` | Override Kafka Connect image tag with digest | `nil` | +| `cruiseControl.image.registry` | Override default Cruise Control image registry | `nil` | +| `cruiseControl.image.repository` | Override default Cruise Control image repository | `nil` | +| `cruiseControl.image.name` | Cruise Control image name | `kafka` | +| `cruiseControl.image.tagPrefix` | Override default Cruise Control image tag prefix | `nil` | +| `cruiseControl.image.tag` | Override default Cruise Control image tag and ignore suffix | `nil` | +| `cruiseControl.image.digest` | Override Cruise Control image tag with digest | `nil` | +| `topicOperator.image.registry` | Override default Topic Operator image registry | `nil` | +| `topicOperator.image.repository` | Override default Topic Operator image repository | `nil` | +| `topicOperator.image.name` | Topic Operator image name | `operator` | +| `topicOperator.image.tag` | Override default Topic Operator image tag | `nil` | +| `topicOperator.image.digest` | Override Topic Operator image tag with digest | `nil` | +| `userOperator.image.registry` | Override default User Operator image registry | `nil` | +| `userOperator.image.repository` | Override default User Operator image repository | `nil` | +| `userOperator.image.name` | User Operator image name | `operator` | +| `userOperator.image.tag` | Override default User Operator image tag | `nil` | +| `userOperator.image.digest` | Override User Operator image tag with digest | `nil` | +| `kafkaInit.image.registry` | Override default Init Kafka image registry | `nil` | +| `kafkaInit.image.repository` | Override default Init Kafka image repository | `nil` | +| `kafkaInit.image.name` | Init Kafka image name | `operator` | +| `kafkaInit.image.tag` | Override default Init Kafka image tag | `nil` | +| `kafkaInit.image.digest` | Override Init Kafka image tag with digest | `nil` | +| `kafkaBridge.image.registry` | Override default Kafka Bridge image registry | `quay.io` | +| `kafkaBridge.image.repository` | Override default Kafka Bridge image repository | `strimzi` | +| `kafkaBridge.image.name` | Kafka Bridge image name | `kafka-bridge` | +| `kafkaBridge.image.tag` | Override default Kafka Bridge image tag | `1.1.0` | +| `kafkaBridge.image.digest` | Override Kafka Bridge image tag with digest | `nil` | +| `kafkaExporter.image.registry` | Override default Kafka Exporter image registry | `nil` | +| `kafkaExporter.image.repository` | Override default Kafka Exporter image repository | `nil` | +| `kafkaExporter.image.name` | Kafka Exporter image name | `kafka` | +| `kafkaExporter.image.tagPrefix` | Override default Kafka Exporter image tag prefix | `nil` | +| `kafkaExporter.image.tag` | Override default Kafka Exporter image tag and ignore suffix | `nil` | +| `kafkaExporter.image.digest` | Override Kafka Exporter image tag with digest | `nil` | +| `kafkaMirrorMaker2.image.registry` | Override default Kafka Mirror Maker 2 image registry | `nil` | +| `kafkaMirrorMaker2.image.repository` | Override default Kafka Mirror Maker 2 image repository | `nil` | +| `kafkaMirrorMaker2.image.name` | Kafka Mirror Maker 2 image name | `kafka` | +| `kafkaMirrorMaker2.image.tagPrefix` | Override default Kafka Mirror Maker 2 image tag prefix | `nil` | +| `kafkaMirrorMaker2.image.tag` | Override default Kafka Mirror Maker 2 image tag and ignore suffix | `nil` | +| `kafkaMirrorMaker2.image.digest` | Override Kafka Mirror Maker 2 image tag with digest | `nil` | +| `kanikoExecutor.image.registry` | Override default Kaniko Executor image registry | `nil` | +| `kanikoExecutor.image.repository` | Override default Kaniko Executor image repository | `nil` | +| `kanikoExecutor.image.name` | Kaniko Executor image name | `kaniko-executor` | +| `kanikoExecutor.image.tag` | Override default Kaniko Executor image tag | `nil` | +| `kanikoExecutor.image.digest` | Override Kaniko Executor image tag with digest | `nil` | +| `buildah.image.registry` | Override default Buildah image registry | `nil` | +| `buildah.image.repository` | Override default Buildah image repository | `nil` | +| `buildah.image.name` | Buildah image name | `buildah` | +| `buildah.image.tag` | Override default Buildah image tag | `nil` | +| `buildah.image.digest` | Override Buildah image tag with digest | `nil` | +| `resources.limits.memory` | Memory constraint for limits | `384Mi` | +| `resources.limits.cpu` | CPU constraint for limits | `1000m` | +| `resources.requests.memory` | Memory constraint for requests | `384Mi` | +| `livenessProbe.initialDelaySeconds` | Liveness probe initial delay in seconds | 10 | +| `livenessProbe.periodSeconds` | Liveness probe period in seconds | 30 | +| `readinessProbe.initialDelaySeconds` | Readiness probe initial delay in seconds | 10 | +| `readinessProbe.periodSeconds` | Readiness probe period in seconds | 30 | +| `imageTagOverride` | Override all image tag config | `nil` | +| `createGlobalResources` | Allow creation of cluster-scoped resources | `true` | +| `createAggregateRoles` | Create cluster roles that extend aggregated roles to use Strimzi CRDs | `false` | +| `tolerations` | Add tolerations to Operator Pod | `[]` | +| `topologySpreadConstraints` | Add topology spread constraints to Operator Pod | `[]` | +| `affinity` | Add affinities to Operator Pod | `{}` | +| `annotations` | Add annotations to Operator Pod | `{}` | +| `labels` | Add labels to Operator Pod | `{}` | +| `nodeSelector` | Add a node selector to Operator Pod | `{}` | +| `featureGates` | Feature Gates configuration | `nil` | +| `tmpDirSizeLimit` | Set the `sizeLimit` for the tmp dir volume used by the operator | `1Mi` | +| `labelsExclusionPattern` | Override the exclude pattern for exclude some labels | `""` | +| `operatorNetworkPolicy.enabled` | Generate a networkPolicy for the strimzi-operator (operator not Kafka) | `false` | +| `generateNetworkPolicy` | Controls whether Strimzi generates network policy resources | `true` | +| `connectBuildTimeoutMs` | Overrides the default timeout value for building new Kafka Connect | `300000` | +| `mavenBuilder.image.registry` | Override default Maven Builder image registry | `nil` | +| `mavenBuilder.image.repository` | Maven Builder image repository | `nil` | +| `mavenBuilder.image.name` | Override default Maven Builder image name | `maven-builder` | +| `mavenBuilder.image.tag` | Override default Maven Builder image tag | `nil` | +| `mavenBuilder.image.digest` | Override Maven Builder image tag with digest | `nil` | +| `logConfiguration` | Override default `log4j.properties` content | `nil` | +| `logLevel` | Override default logging level | `INFO` | +| `dashboards.enabled` | Generate configmaps containing the dashboards | `false` | +| `dashboards.label` | How should the dashboards be labeled for the sidecar | `grafana_dashboard` | +| `dashboards.labelValue` | What should the dashboards label value be for the sidecar | `"1"` | +| `dashboards.extraLabels` | Any additional labels you would like on the dashboards | `{}` | +| `dashboards.namespace` | What namespace should the dashboards be loaded into | `Follows toplevel Namespace` | +| `dashboards.annotations` | Any custom annotations (such as folder for the sidecar) | `{}` | Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example, ```bash -$ helm install my-strimzi-cluster-operator --set replicas=2 oci://quay.io/strimzi-helm/strimzi-kafka-operator +helm install my-strimzi-cluster-operator --set replicas=2 oci://quay.io/strimzi-helm/strimzi-kafka-operator ``` diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/040-Crd-kafka.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/040-Crd-kafka.yaml index 8e5ca8570..c2edb2f9a 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/040-Crd-kafka.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/040-Crd-kafka.yaml @@ -21,32 +21,28 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: status: {} additionalPrinterColumns: - - name: Desired Kafka replicas - description: The desired number of Kafka replicas in the cluster - jsonPath: .spec.kafka.replicas - type: integer - - name: Desired ZK replicas - description: The desired number of ZooKeeper replicas in the cluster - jsonPath: .spec.zookeeper.replicas - type: integer - name: Ready description: The state of the custom resource jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" type: string - - name: Metadata State - description: The state of the cluster metadata - jsonPath: .status.kafkaMetadataState - type: string - name: Warnings description: Warnings related to the custom resource jsonPath: ".status.conditions[?(@.type==\"Warning\")].status" type: string + - name: Kafka version + description: The Kafka version used by the cluster + jsonPath: .status.kafkaVersion + type: string + - name: Metadata version + description: The Kafka metadata version used by the cluster + jsonPath: .status.kafkaMetadataVersion + type: string schema: openAPIV3Schema: type: object @@ -71,10 +67,6 @@ spec: metadataVersion: type: string description: "The KRaft metadata version used by the Kafka cluster. This property is ignored when running in ZooKeeper mode. If the property is not set, it defaults to the metadata version that corresponds to the `version` property." - replicas: - type: integer - minimum: 1 - description: The number of pods in the cluster. This property is required when node pools are not used. image: type: string description: "The container image used for Kafka pods. If the property is not set, the default Kafka image version is determined based on the `version` configuration. The image names are specifically mapped to corresponding versions in the Cluster Operator configuration. Changing the Kafka image version does not automatically update the image versions for other components, such as Kafka Exporter. " @@ -97,200 +89,32 @@ spec: enum: - internal - route + - tlsroute - loadbalancer - nodeport - ingress - cluster-ip - description: "Type of the listener. The supported types are as follows: \n\n* `internal` type exposes Kafka internally only within the Kubernetes cluster.\n* `route` type uses OpenShift Routes to expose Kafka.\n* `loadbalancer` type uses LoadBalancer type services to expose Kafka.\n* `nodeport` type uses NodePort type services to expose Kafka.\n* `ingress` type uses Kubernetes Nginx Ingress to expose Kafka with TLS passthrough.\n* `cluster-ip` type uses a per-broker `ClusterIP` service.\n" + description: "Type of the listener. The supported types are as follows: \n\n* `internal` type exposes Kafka internally only within the Kubernetes cluster.\n* `route` type uses OpenShift Routes to expose Kafka.\n* `loadbalancer` type uses LoadBalancer type services to expose Kafka.\n* `nodeport` type uses NodePort type services to expose Kafka.\n* `ingress` (deprecated) type uses Kubernetes Nginx Ingress to expose Kafka with TLS passthrough.\n* `cluster-ip` type uses a per-broker `ClusterIP` service.\n" tls: type: boolean description: "Enables TLS encryption on the listener. This is a required property. For `route` and `ingress` type listeners, TLS encryption must be always enabled." authentication: type: object properties: - accessTokenIsJwt: - type: boolean - description: Configure whether the access token is treated as JWT. This must be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - checkAccessTokenType: - type: boolean - description: Configure whether the access token type check is performed or not. This should be set to `false` if the authorization server does not include 'typ' claim in JWT token. Defaults to `true`. - checkAudience: - type: boolean - description: "Enable or disable audience checking. Audience checks identify the recipients of tokens. If audience checking is enabled, the OAuth Client ID also has to be configured using the `clientId` property. The Kafka broker will reject tokens that do not have its `clientId` in their `aud` (audience) claim.Default value is `false`." - checkIssuer: - type: boolean - description: Enable or disable issuer checking. By default issuer is checked using the value configured by `validIssuerUri`. Default value is `true`. - clientAudience: - type: string - description: The audience to use when making requests to the authorization server's token endpoint. Used for inter-broker authentication and for configuring OAuth 2.0 over PLAIN using the `clientId` and `secret` method. - clientId: - type: string - description: OAuth Client ID which the Kafka broker can use to authenticate against the authorization server and use the introspect endpoint URI. - clientScope: - type: string - description: The scope to use when making requests to the authorization server's token endpoint. Used for inter-broker authentication and for configuring OAuth 2.0 over PLAIN using the `clientId` and `secret` method. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka broker can use to authenticate against the authorization server and use the introspect endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - customClaimCheck: - type: string - description: JsonPath filter query to be applied to the JWT token or to the response of the introspection endpoint for additional token validation. Not set by default. - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableECDSA: - type: boolean - description: Enable or disable ECDSA support by installing BouncyCastle crypto provider. ECDSA support is always enabled. The BouncyCastle libraries are no longer packaged with Strimzi. Value is ignored. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - enableOauthBearer: - type: boolean - description: Enable or disable OAuth authentication over SASL_OAUTHBEARER. Default value is `true`. - enablePlain: - type: boolean - description: Enable or disable OAuth authentication over SASL_PLAIN. There is no re-authentication support when this mechanism is used. Default value is `false`. - failFast: - type: boolean - description: Enable or disable termination of Kafka broker processes due to potentially recoverable runtime errors during startup. Default value is `true`. - fallbackUserNameClaim: - type: string - description: The fallback username claim to be used for the user ID if the claim specified by `userNameClaim` is not present. This is useful when `client_credentials` authentication only results in the client ID being provided in another claim. It only takes effect if `userNameClaim` is set. - fallbackUserNamePrefix: - type: string - description: "The prefix to use with the value of `fallbackUserNameClaim` to construct the user id. This only takes effect if `fallbackUserNameClaim` is true, and the value is present for the claim. Mapping usernames and client ids into the same user id space is useful in preventing name collisions." - groupsClaim: - type: string - description: JsonPath query used to extract groups for the user during authentication. Extracted groups can be used by a custom authorizer. By default no groups are extracted. - groupsClaimDelimiter: - type: string - description: "A delimiter used to parse groups when they are extracted as a single String value rather than a JSON array. Default value is ',' (comma)." - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - introspectionEndpointUri: - type: string - description: URI of the token introspection endpoint which can be used to validate opaque non-JWT tokens. - jwksEndpointUri: - type: string - description: "URI of the JWKS certificate endpoint, which can be used for local JWT validation." - jwksExpirySeconds: - type: integer - minimum: 1 - description: Configures how often are the JWKS certificates considered valid. The expiry interval has to be at least 60 seconds longer then the refresh interval specified in `jwksRefreshSeconds`. Defaults to 360 seconds. - jwksIgnoreKeyUse: - type: boolean - description: Flag to ignore the 'use' attribute of `key` declarations in a JWKS endpoint response. Default value is `false`. - jwksMinRefreshPauseSeconds: - type: integer - minimum: 0 - description: "The minimum pause between two consecutive refreshes. When an unknown signing key is encountered the refresh is scheduled immediately, but will always wait for this minimum pause. Defaults to 1 second." - jwksRefreshSeconds: - type: integer - minimum: 1 - description: Configures how often are the JWKS certificates refreshed. The refresh interval has to be at least 60 seconds shorter then the expiry interval specified in `jwksExpirySeconds`. Defaults to 300 seconds. listenerConfig: x-kubernetes-preserve-unknown-fields: true type: object description: Configuration to be used for a specific listener. All values are prefixed with `listener.name.`. - maxSecondsWithoutReauthentication: - type: integer - description: "Maximum number of seconds the authenticated session remains valid without re-authentication. This enables Apache Kafka re-authentication feature, and causes sessions to expire when the access token expires. If the access token expires before max time or if max time is reached, the client has to re-authenticate, otherwise the server will drop the connection. Not set by default - the authenticated session does not expire when the access token expires. This option only applies to SASL_OAUTHBEARER authentication mechanism (when `enableOauthBearer` is `true`)." - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." sasl: type: boolean description: Enable or disable SASL on this listener. - secrets: - type: array - items: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Secrets to be mounted to `/opt/kafka/custom-authn-secrets/custom-listener--/`. - serverBearerTokenLocation: - type: string - description: Path to the file on the local filesystem that contains a bearer token to be used instead of client ID and secret when authenticating to authorization server. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: "URI of the Token Endpoint to use with SASL_PLAIN mechanism when the client authenticates with `clientId` and a `secret`. If set, the client can authenticate over SASL_PLAIN by either setting `username` to `clientId`, and setting `password` to client `secret`, or by setting `username` to account username, and `password` to access token prefixed with `$accessToken:`. If this option is not set, the `password` is always interpreted as an access token (without a prefix), and `username` as the account username (a so called 'no-client-credentials' mode)." type: type: string enum: - tls - scram-sha-512 - - oauth - custom - description: Authentication type. `oauth` type uses SASL OAUTHBEARER Authentication. `scram-sha-512` type uses SASL SCRAM-SHA-512 Authentication. `tls` type uses TLS Client Authentication. `tls` type is supported only on TLS listeners.`custom` type allows for any authentication type to be used. - userInfoEndpointUri: - type: string - description: 'URI of the User Info Endpoint to use as a fallback to obtaining the user id when the Introspection Endpoint does not return information that can be used for the user id. ' - userNameClaim: - type: string - description: "Name of the claim from the JWT authentication token, Introspection Endpoint response or User Info Endpoint response which will be used to extract the user id. Defaults to `sub`." - userNamePrefix: - type: string - description: "The prefix to use with the value of `userNameClaim` to construct the user ID. This only takes effect if `userNameClaim` is specified and the value is present for the claim. When used in combination with `fallbackUserNameClaims`, it ensures consistent mapping of usernames and client IDs into the same user ID space and prevents name collisions." - validIssuerUri: - type: string - description: URI of the token issuer used for authentication. - validTokenType: - type: string - description: "Valid value for the `token_type` attribute returned by the Introspection Endpoint. No default value, and not checked by default." + description: Authentication type. `scram-sha-512` type uses SASL SCRAM-SHA-512 Authentication. `tls` type uses TLS Client Authentication. `tls` type is supported only on TLS listeners. `custom` type allows for any authentication type to be used. required: - type description: Authentication configuration for this listener. @@ -308,7 +132,7 @@ spec: description: The name of the file certificate in the Secret. key: type: string - description: The name of the private key in the Secret. + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." required: - secretName - certificate @@ -476,22 +300,56 @@ spec: * `InternalIP` * `Hostname` - This property is used to select the preferred address type, which is checked first. If no address is found for this address type, the other types are checked in the default order.For `nodeport` listeners only. + This property is used to select the preferred address type, which is checked first. If no address is found for this address type, the other types are checked in the default order. For `nodeport` listeners only. publishNotReadyAddresses: type: boolean description: Configures whether the service endpoints are considered "ready" even if the Pods themselves are not. Defaults to `false`. This field can not be used with `internal` listeners. + perBrokerAnnotationsTemplate: + additionalProperties: + type: string + type: object + description: "Configures the template used to generate annotations for individual brokers. You can use the `{nodeId}` and `{nodePodName}` placeholders in annotation values. Templating applies only to annotation values." + perBrokerLabelsTemplate: + additionalProperties: + type: string + type: object + description: "Configures the template used to generate labels for individual brokers. You can use the `{nodeId}` and `{nodePodName}` placeholders in label values. Templating applies only to label values." hostTemplate: type: string description: "Configures the template for generating the hostnames of the individual brokers. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." advertisedHostTemplate: type: string description: "Configures the template for generating the advertised hostnames of the individual brokers. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + advertisedPortTemplate: + type: string + description: "Configures the template for generating the advertised ports of the individual brokers. It allows to specify a simple mathematics formula that will be used to calculate the port. The only valid placeholder that you can use in the template is `{nodeId}`. Supported operations are `+`, `-`, and `*`. For example, `9000 + {nodeId}` will generate ports `9000`, `9001`, `9002`, and so on for the individual brokers. You can also use a fixed port number in the template, for example `9000`, which will generate the same port for all brokers." allocateLoadBalancerNodePorts: type: boolean description: |- Configures whether to allocate NodePort automatically for the `Service` with type `LoadBalancer`. This is a one to one with the `spec.allocateLoadBalancerNodePorts` configuration in the `Service` type For `loadbalancer` listeners only. + parentRefs: + type: array + items: + type: object + properties: + kind: + type: string + group: + type: string + name: + type: string + namespace: + type: string + port: + type: integer + sectionName: + type: string + description: |- + For `tlsroute` listeners only. + Configures the Gateway API resource or resources that generated `TLSRoute` resources attach to. + Typically, this is a `Gateway` resource. description: Additional listener configuration. networkPolicyPeers: type: array @@ -557,176 +415,13 @@ spec: config: x-kubernetes-preserve-unknown-fields: true type: object - description: "Kafka broker config properties with the following prefixes cannot be set: listeners, advertised., broker., listener., host.name, port, inter.broker.listener.name, sasl., ssl., security., password., log.dir, zookeeper.connect, zookeeper.set.acl, zookeeper.ssl, zookeeper.clientCnxnSocket, authorizer., super.user, cruise.control.metrics.topic, cruise.control.metrics.reporter.bootstrap.servers, node.id, process.roles, controller., metadata.log.dir, zookeeper.metadata.migration.enable, client.quota.callback.static.kafka.admin., client.quota.callback.static.produce, client.quota.callback.static.fetch, client.quota.callback.static.storage.per.volume.limit.min.available., client.quota.callback.static.excluded.principal.name.list (with the exception of: zookeeper.connection.timeout.ms, sasl.server.max.receive.size, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols, ssl.secure.random.implementation, cruise.control.metrics.topic.num.partitions, cruise.control.metrics.topic.replication.factor, cruise.control.metrics.topic.retention.ms, cruise.control.metrics.topic.auto.create.retries, cruise.control.metrics.topic.auto.create.timeout.ms, cruise.control.metrics.topic.min.insync.replicas, controller.quorum.election.backoff.max.ms, controller.quorum.election.timeout.ms, controller.quorum.fetch.timeout.ms)." - storage: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation. - deleteClaim: - type: boolean - description: Specifies if the persistent volume claim has to be deleted when the cluster is un-deployed. - id: - type: integer - minimum: 0 - description: Storage identification number. It is mandatory only for storage volumes defined in a storage of type 'jbod'. - kraftMetadata: - type: string - enum: - - shared - description: "Specifies whether this volume should be used for storing KRaft metadata. This property is optional. When set, the only currently supported value is `shared`. At most one volume can have this property set." - overrides: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation for this broker. - broker: - type: integer - description: Id of the kafka broker (broker identifier). - description: Overrides for individual brokers. The `overrides` field allows you to specify a different configuration for different brokers. - selector: - additionalProperties: - type: string - type: object - description: Specifies a specific persistent volume to use. It contains key:value pairs representing labels for selecting such a volume. - size: - type: string - description: "When `type=persistent-claim`, defines the size of the persistent volume claim, such as 100Gi. Mandatory when `type=persistent-claim`." - sizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "When type=ephemeral, defines the total amount of local storage required for this EmptyDir volume (for example 1Gi)." - type: - type: string - enum: - - ephemeral - - persistent-claim - - jbod - description: "Storage type, must be either 'ephemeral', 'persistent-claim', or 'jbod'." - volumes: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation. - deleteClaim: - type: boolean - description: Specifies if the persistent volume claim has to be deleted when the cluster is un-deployed. - id: - type: integer - minimum: 0 - description: Storage identification number. Mandatory for storage volumes defined with a `jbod` storage type configuration. - kraftMetadata: - type: string - enum: - - shared - description: "Specifies whether this volume should be used for storing KRaft metadata. This property is optional. When set, the only currently supported value is `shared`. At most one volume can have this property set." - overrides: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation for this broker. - broker: - type: integer - description: Id of the kafka broker (broker identifier). - description: Overrides for individual brokers. The `overrides` field allows you to specify a different configuration for different brokers. - selector: - additionalProperties: - type: string - type: object - description: Specifies a specific persistent volume to use. It contains key:value pairs representing labels for selecting such a volume. - size: - type: string - description: "When `type=persistent-claim`, defines the size of the persistent volume claim, such as 100Gi. Mandatory when `type=persistent-claim`." - sizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "When type=ephemeral, defines the total amount of local storage required for this EmptyDir volume (for example 1Gi)." - type: - type: string - enum: - - ephemeral - - persistent-claim - description: "Storage type, must be either 'ephemeral' or 'persistent-claim'." - required: - - type - description: List of volumes as Storage objects representing the JBOD disks array. - required: - - type - description: Storage configuration (disk). Cannot be updated. This property is required when node pools are not used. + description: Kafka broker config properties with certain prefixes cannot be set unless it is in the exception list. Consult the documentation for the list of forbidden prefixes and exceptions. authorization: type: object properties: - allowOnError: - type: boolean - description: "Defines whether a Kafka client should be allowed or denied by default when the authorizer fails to query the Open Policy Agent, for example, when it is temporarily unavailable). Defaults to `false` - all actions will be denied." authorizerClass: type: string description: "Authorization implementation class, which must be available in classpath." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - minimum: 1 - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - delegateToKafkaAcls: - type: boolean - description: Whether authorization decision should be delegated to the 'Simple' authorizer if DENIED by Keycloak Authorization Services policies. Default value is `false`. - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. The default value is `false`. - expireAfterMs: - type: integer - description: The expiration of the records kept in the local cache to avoid querying the Open Policy Agent for every request. Defines how often the cached authorization decisions are reloaded from the Open Policy Agent server. In milliseconds. Defaults to `3600000`. - grantsAlwaysLatest: - type: boolean - description: "Controls whether the latest grants are fetched for a new session. When enabled, grants are retrieved from Keycloak and cached for the user. The default value is `false`." - grantsGcPeriodSeconds: - type: integer - minimum: 1 - description: "The time, in seconds, between consecutive runs of a job that cleans stale grants from the cache. The default value is 300." - grantsMaxIdleTimeSeconds: - type: integer - minimum: 1 - description: "The time, in seconds, after which an idle grant can be evicted from the cache. The default value is 300." - grantsRefreshPeriodSeconds: - type: integer - minimum: 0 - description: The time between two consecutive grants refresh runs in seconds. The default value is 60. - grantsRefreshPoolSize: - type: integer - minimum: 1 - description: "The number of threads to use to refresh grants for active sessions. The more threads, the more parallelism, so the sooner the job completes. However, using more threads places a heavier load on the authorization server. The default value is 5." - httpRetries: - type: integer - minimum: 0 - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - initialCacheCapacity: - type: integer - description: Initial capacity of the local cache used by the authorizer to avoid querying the Open Policy Agent for every request Defaults to `5000`. - maximumCacheSize: - type: integer - description: Maximum capacity of the local cache used by the authorizer to avoid querying the Open Policy Agent for every request. Defaults to `50000`. - readTimeoutSeconds: - type: integer - minimum: 1 - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." superUsers: type: array items: @@ -735,60 +430,37 @@ spec: supportsAdminApi: type: boolean description: Indicates whether the custom authorizer supports the APIs for managing ACLs using the Kafka Admin API. Defaults to `false`. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. type: type: string enum: - simple - - opa - - keycloak - custom - description: "Authorization type. Currently, the supported types are `simple`, `keycloak`, `opa` and `custom`. `simple` authorization type uses Kafka's built-in authorizer for authorization. `keycloak` authorization type uses Keycloak Authorization Services for authorization. `opa` authorization type uses Open Policy Agent based authorization.`custom` authorization type uses user-provided implementation for authorization." - url: - type: string - example: http://opa:8181/v1/data/kafka/authz/allow - description: The URL used to connect to the Open Policy Agent server. The URL has to include the policy which will be queried by the authorizer. This option is required. + description: "Authorization type. Currently, the supported types are `simple` and `custom`. `simple` authorization type uses Kafka's built-in authorizer for authorization. `custom` authorization type uses user-provided implementation for authorization." required: - type description: Authorization configuration for Kafka brokers. rack: type: object properties: + envVarName: + type: string + description: The name of the environment variable that defines the rack ID. Its value sets the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect or MirrorMaker 2. topologyKey: type: string example: topology.kubernetes.io/zone description: "A key that matches labels assigned to the Kubernetes cluster nodes. The value of the label is used to set a broker's `broker.rack` config, and the `client.rack` config for Kafka Connect or MirrorMaker 2." - required: - - topologyKey + type: + type: string + enum: + - topology-label + - environment-variable + description: "Specifies the rack awareness type. Supported types are `topology-label` and `environment-variable`. `topology-label` uses a Kubernetes worker node label to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. `environment-variable` uses an environment variable to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. When not specified, `topology-label` type is used by default." description: Configuration of the `broker.rack` broker config. + x-kubernetes-validations: + - rule: (has(self.type) && self.type != "topology-label") || self.topologyKey != "" + message: topologyKey property is required + - rule: has(self.type) == false || self.type != "environment-variable" || self.envVarName != "" + message: envVarName property is required brokerRackInitImage: type: string description: The image of the init container used for initializing the `broker.rack`. @@ -887,35 +559,6 @@ spec: - type description: Authentication configuration for connecting to the JMX port. description: JMX Options for Kafka brokers. - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. metricsConfig: type: object properties: @@ -923,7 +566,8 @@ spec: type: string enum: - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. + - strimziMetricsReporter + description: Metrics type. The supported types are `jmxPrometheusExporter` and `strimziMetricsReporter`. Type `jmxPrometheusExporter` uses the Prometheus JMX Exporter to expose Kafka JMX metrics in Prometheus format through an HTTP endpoint. Type `strimziMetricsReporter` uses the Strimzi Metrics Reporter to directly expose Kafka metrics in Prometheus format through an HTTP endpoint. valueFrom: type: object properties: @@ -937,11 +581,22 @@ spec: optional: type: boolean description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' + description: ConfigMap entry where the Prometheus JMX Exporter configuration is stored. + values: + type: object + properties: + allowList: + type: array + items: + type: string + description: A list of regex patterns to filter the metrics to collect. Should contain at least one element. + description: Configuration values for the Strimzi Metrics Reporter. required: - type - - valueFrom description: Metrics configuration. + x-kubernetes-validations: + - rule: self.type != 'jmxPrometheusExporter' || has(self.valueFrom) + message: valueFrom property is required logging: type: object properties: @@ -976,30 +631,6 @@ spec: template: type: object properties: - statefulset: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - podManagementPolicy: - type: string - enum: - - OrderedReady - - Parallel - description: PodManagementPolicy which will be used for this StatefulSet. Valid values are `Parallel` and `OrderedReady`. Defaults to `Parallel`. - description: Template for Kafka `StatefulSet`. pod: type: object properties: @@ -1501,6 +1132,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -1536,7 +1196,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -1557,20 +1217,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -1578,7 +1238,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -1597,7 +1257,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -1605,7 +1273,75 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + templatedVolumes: + type: array + items: + type: object + properties: + name: + type: string + description: Name to use for the volume. Required. + secret: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + optional: + type: boolean + secretName: + type: string + description: "`Secret` to use to populate the volume. The name of the Secret and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + configMap: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + name: + type: string + optional: + type: boolean + description: "`ConfigMap` to use to populate the volume. The name of the ConfigMap and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + persistentVolumeClaim: + type: object + properties: + claimName: + type: string + readOnly: + type: boolean + description: "`PersistentVolumeClaim` object to use to populate the volume. The name of the Persistent Volume Claim can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + oneOf: + - properties: + secret: {} + configMap: {} + persistentVolumeClaim: {} + description: Additional volumes that can be mounted to the pod. These volumes can use templates to mount different volumes into individual Pods. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka `Pods`. bootstrapService: type: object @@ -2269,1194 +2005,6 @@ spec: required: - listeners description: Configuration of the Kafka cluster. - zookeeper: - type: object - properties: - replicas: - type: integer - minimum: 1 - description: The number of pods in the cluster. - image: - type: string - description: "The container image used for ZooKeeper pods. If no image name is explicitly specified, it is determined based on the Kafka version set in `spec.kafka.version`. The image names are specifically mapped to corresponding versions in the Cluster Operator configuration." - storage: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation. - deleteClaim: - type: boolean - description: Specifies if the persistent volume claim has to be deleted when the cluster is un-deployed. - id: - type: integer - minimum: 0 - description: Storage identification number. Mandatory for storage volumes defined with a `jbod` storage type configuration. - kraftMetadata: - type: string - enum: - - shared - description: "Specifies whether this volume should be used for storing KRaft metadata. This property is optional. When set, the only currently supported value is `shared`. At most one volume can have this property set." - overrides: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation for this broker. - broker: - type: integer - description: Id of the kafka broker (broker identifier). - description: Overrides for individual brokers. The `overrides` field allows you to specify a different configuration for different brokers. - selector: - additionalProperties: - type: string - type: object - description: Specifies a specific persistent volume to use. It contains key:value pairs representing labels for selecting such a volume. - size: - type: string - description: "When `type=persistent-claim`, defines the size of the persistent volume claim, such as 100Gi. Mandatory when `type=persistent-claim`." - sizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "When type=ephemeral, defines the total amount of local storage required for this EmptyDir volume (for example 1Gi)." - type: - type: string - enum: - - ephemeral - - persistent-claim - description: "Storage type, must be either 'ephemeral' or 'persistent-claim'." - required: - - type - description: Storage configuration (disk). Cannot be updated. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: "The ZooKeeper broker config. Properties with the following prefixes cannot be set: server., dataDir, dataLogDir, clientPort, authProvider, quorum.auth, requireClientAuthScheme, snapshot.trust.empty, standaloneEnabled, reconfigEnabled, 4lw.commands.whitelist, secureClientPort, ssl., serverCnxnFactory, sslQuorum (with the exception of: ssl.protocol, ssl.quorum.protocol, ssl.enabledProtocols, ssl.quorum.enabledProtocols, ssl.ciphersuites, ssl.quorum.ciphersuites, ssl.hostnameVerification, ssl.quorum.hostnameVerification)." - livenessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod liveness checking. - readinessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod readiness checking. - jvmOptions: - type: object - properties: - "-XX": - additionalProperties: - type: string - type: object - description: A map of -XX options to the JVM. - "-Xmx": - type: string - pattern: "^[0-9]+[mMgG]?$" - description: -Xmx option to to the JVM. - "-Xms": - type: string - pattern: "^[0-9]+[mMgG]?$" - description: -Xms option to to the JVM. - gcLoggingEnabled: - type: boolean - description: Specifies whether the Garbage Collection logging is enabled. The default is false. - javaSystemProperties: - type: array - items: - type: object - properties: - name: - type: string - description: The system property name. - value: - type: string - description: The system property value. - description: A map of additional system properties which will be passed using the `-D` option to the JVM. - description: JVM Options for pods. - jmxOptions: - type: object - properties: - authentication: - type: object - properties: - type: - type: string - enum: - - password - description: Authentication type. Currently the only supported types are `password`.`password` type creates a username and protected port with no TLS. - required: - - type - description: Authentication configuration for connecting to the JMX port. - description: JMX Options for Zookeeper nodes. - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. - metricsConfig: - type: object - properties: - type: - type: string - enum: - - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. - valueFrom: - type: object - properties: - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' - required: - - type - - valueFrom - description: Metrics configuration. - logging: - type: object - properties: - loggers: - additionalProperties: - type: string - type: object - description: A Map from logger name to logger level. - type: - type: string - enum: - - inline - - external - description: "Logging type, must be either 'inline' or 'external'." - valueFrom: - type: object - properties: - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to the key in the ConfigMap containing the configuration. - description: '`ConfigMap` entry where the logging configuration is stored. ' - required: - - type - description: Logging configuration for ZooKeeper. - template: - type: object - properties: - statefulset: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - podManagementPolicy: - type: string - enum: - - OrderedReady - - Parallel - description: PodManagementPolicy which will be used for this StatefulSet. Valid values are `Parallel` and `OrderedReady`. Defaults to `Parallel`. - description: Template for ZooKeeper `StatefulSet`. - podSet: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for ZooKeeper `StrimziPodSet` resource. - pod: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - imagePullSecrets: - type: array - items: - type: object - properties: - name: - type: string - description: "List of references to secrets in the same namespace to use for pulling any of the images used by this Pod. When the `STRIMZI_IMAGE_PULL_SECRETS` environment variable in Cluster Operator and the `imagePullSecrets` option are specified, only the `imagePullSecrets` variable is used and the `STRIMZI_IMAGE_PULL_SECRETS` variable is ignored." - securityContext: - type: object - properties: - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - fsGroup: - type: integer - fsGroupChangePolicy: - type: string - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxChangePolicy: - type: string - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - supplementalGroups: - type: array - items: - type: integer - supplementalGroupsPolicy: - type: string - sysctls: - type: array - items: - type: object - properties: - name: - type: string - value: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Configures pod-level security attributes and common container settings. - terminationGracePeriodSeconds: - type: integer - minimum: 0 - description: "The grace period is the duration in seconds after the processes running in the pod are sent a termination signal, and the time when the processes are forcibly halted with a kill signal. Set this value to longer than the expected cleanup time for your process. Value must be a non-negative integer. A zero value indicates delete immediately. You might need to increase the grace period for very large Kafka clusters, so that the Kafka brokers have enough time to transfer their work to another broker before they are terminated. Defaults to 30 seconds." - affinity: - type: object - properties: - nodeAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - preference: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: object - properties: - nodeSelectorTerms: - type: array - items: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - podAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - podAntiAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - description: The pod's affinity rules. - tolerations: - type: array - items: - type: object - properties: - effect: - type: string - key: - type: string - operator: - type: string - tolerationSeconds: - type: integer - value: - type: string - description: The pod's tolerations. - topologySpreadConstraints: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - maxSkew: - type: integer - minDomains: - type: integer - nodeAffinityPolicy: - type: string - nodeTaintsPolicy: - type: string - topologyKey: - type: string - whenUnsatisfiable: - type: string - description: The pod's topology spread constraints. - priorityClassName: - type: string - description: 'The name of the priority class used to assign priority to the pods. ' - schedulerName: - type: string - description: "The name of the scheduler used to dispatch this `Pod`. If not specified, the default scheduler will be used." - hostAliases: - type: array - items: - type: object - properties: - hostnames: - type: array - items: - type: string - ip: - type: string - description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. - enableServiceLinks: - type: boolean - description: Indicates whether information about services should be injected into Pod's environment variables. - tmpDirSizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "Defines the total amount of pod memory allocated for the temporary `EmptyDir` volume `/tmp`. Specify the allocation in memory units, for example, `100Mi` for 100 mebibytes. Default value is `5Mi`. The `/tmp` volume is backed by pod memory, not disk storage, so avoid setting a high value as it consumes pod memory resources." - volumes: - type: array - items: - type: object - properties: - name: - type: string - description: Name to use for the volume. Required. - secret: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - optional: - type: boolean - secretName: - type: string - description: Secret to use populate the volume. - configMap: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - name: - type: string - optional: - type: boolean - description: ConfigMap to use to populate the volume. - emptyDir: - type: object - properties: - medium: - type: string - sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. - persistentVolumeClaim: - type: object - properties: - claimName: - type: string - readOnly: - type: boolean - description: PersistentVolumeClaim object to use to populate the volume. - csi: - type: object - properties: - driver: - type: string - fsType: - type: string - nodePublishSecretRef: - type: object - properties: - name: - type: string - readOnly: - type: boolean - volumeAttributes: - additionalProperties: - type: string - type: object - description: CSIVolumeSource object to use to populate the volume. - oneOf: - - properties: - secret: {} - configMap: {} - emptyDir: {} - persistentVolumeClaim: {} - csi: {} - description: Additional volumes that can be mounted to the pod. - description: Template for ZooKeeper `Pods`. - clientService: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - ipFamilyPolicy: - type: string - enum: - - SingleStack - - PreferDualStack - - RequireDualStack - description: "Specifies the IP Family Policy used by the service. Available options are `SingleStack`, `PreferDualStack` and `RequireDualStack`. `SingleStack` is for a single IP family. `PreferDualStack` is for two IP families on dual-stack configured clusters or a single IP family on single-stack clusters. `RequireDualStack` fails unless there are two IP families on dual-stack configured clusters. If unspecified, Kubernetes will choose the default value based on the service type." - ipFamilies: - type: array - items: - type: string - enum: - - IPv4 - - IPv6 - description: "Specifies the IP Families used by the service. Available options are `IPv4` and `IPv6`. If unspecified, Kubernetes will choose the default value based on the `ipFamilyPolicy` setting." - description: Template for ZooKeeper client `Service`. - nodesService: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - ipFamilyPolicy: - type: string - enum: - - SingleStack - - PreferDualStack - - RequireDualStack - description: "Specifies the IP Family Policy used by the service. Available options are `SingleStack`, `PreferDualStack` and `RequireDualStack`. `SingleStack` is for a single IP family. `PreferDualStack` is for two IP families on dual-stack configured clusters or a single IP family on single-stack clusters. `RequireDualStack` fails unless there are two IP families on dual-stack configured clusters. If unspecified, Kubernetes will choose the default value based on the service type." - ipFamilies: - type: array - items: - type: string - enum: - - IPv4 - - IPv6 - description: "Specifies the IP Families used by the service. Available options are `IPv4` and `IPv6`. If unspecified, Kubernetes will choose the default value based on the `ipFamilyPolicy` setting." - description: Template for ZooKeeper nodes `Service`. - persistentVolumeClaim: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for all ZooKeeper `PersistentVolumeClaims`. - podDisruptionBudget: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata to apply to the `PodDisruptionBudgetTemplate` resource. - maxUnavailable: - type: integer - minimum: 0 - description: "Maximum number of unavailable pods to allow automatic Pod eviction. A Pod eviction is allowed when the `maxUnavailable` number of pods or fewer are unavailable after the eviction. Setting this value to 0 prevents all voluntary evictions, so the pods must be evicted manually. Defaults to 1." - description: Template for ZooKeeper `PodDisruptionBudget`. - zookeeperContainer: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: The environment variable key. - value: - type: string - description: The environment variable value. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a config map. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Reference to the secret or config map property to which the environment variable is set. - oneOf: - - properties: - value: {} - required: - - value - - properties: - valueFrom: {} - required: - - valueFrom - description: Environment variables which should be applied to the container. - securityContext: - type: object - properties: - allowPrivilegeEscalation: - type: boolean - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - capabilities: - type: object - properties: - add: - type: array - items: - type: string - drop: - type: array - items: - type: string - privileged: - type: boolean - procMount: - type: string - readOnlyRootFilesystem: - type: boolean - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Security context for the container. - volumeMounts: - type: array - items: - type: object - properties: - mountPath: - type: string - mountPropagation: - type: string - name: - type: string - readOnly: - type: boolean - recursiveReadOnly: - type: string - subPath: - type: string - subPathExpr: - type: string - description: Additional volume mounts which should be applied to the container. - description: Template for the ZooKeeper container. - serviceAccount: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for the ZooKeeper service account. - jmxSecret: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for Secret of the Zookeeper Cluster JMX authentication. - description: Template for ZooKeeper cluster resources. The template allows users to specify how the Kubernetes resources are generated. - required: - - replicas - - storage - description: Configuration of the ZooKeeper cluster. This section is required when running a ZooKeeper-based Apache Kafka cluster. entityOperator: type: object properties: @@ -3469,18 +2017,10 @@ spec: image: type: string description: The image to use for the Topic Operator. - reconciliationIntervalSeconds: - type: integer - minimum: 0 - description: Interval between periodic reconciliations in seconds. Ignored if reconciliationIntervalMs is set. reconciliationIntervalMs: type: integer minimum: 0 description: Interval between periodic reconciliations in milliseconds. - zookeeperSessionTimeoutSeconds: - type: integer - minimum: 0 - description: Timeout for the ZooKeeper session. startupProbe: type: object properties: @@ -3582,10 +2122,6 @@ spec: x-kubernetes-int-or-string: true type: object description: CPU and memory resources to reserve. - topicMetadataMaxAttempts: - type: integer - minimum: 0 - description: The number of attempts at getting topic metadata. logging: type: object properties: @@ -3659,18 +2195,10 @@ spec: image: type: string description: The image to use for the User Operator. - reconciliationIntervalSeconds: - type: integer - minimum: 0 - description: Interval between periodic reconciliations in seconds. Ignored if reconciliationIntervalMs is set. reconciliationIntervalMs: type: integer minimum: 0 description: Interval between periodic reconciliations in milliseconds. - zookeeperSessionTimeoutSeconds: - type: integer - minimum: 0 - description: Timeout for the ZooKeeper session. secretPrefix: type: string description: The prefix that will be added to the KafkaUser name to be used as the Secret name. @@ -3815,102 +2343,6 @@ spec: description: A map of additional system properties which will be passed using the `-D` option to the JVM. description: JVM Options for pods. description: Configuration of the User Operator. - tlsSidecar: - type: object - properties: - image: - type: string - description: The docker image for the container. - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. - livenessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod liveness checking. - readinessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod readiness checking. - logLevel: - type: string - enum: - - emerg - - alert - - crit - - err - - warning - - notice - - info - - debug - description: The log level for the TLS sidecar. Default value is `notice`. - description: TLS sidecar configuration. template: type: object properties: @@ -4439,6 +2871,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -4474,7 +2935,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -4495,20 +2956,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -4516,7 +2977,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -4535,7 +2996,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -4543,7 +3012,11 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Entity Operator `Pods`. topicOperatorContainer: type: object @@ -4831,149 +3304,6 @@ spec: type: string description: Additional volume mounts which should be applied to the container. description: Template for the Entity User Operator container. - tlsSidecarContainer: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: The environment variable key. - value: - type: string - description: The environment variable value. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a config map. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Reference to the secret or config map property to which the environment variable is set. - oneOf: - - properties: - value: {} - required: - - value - - properties: - valueFrom: {} - required: - - valueFrom - description: Environment variables which should be applied to the container. - securityContext: - type: object - properties: - allowPrivilegeEscalation: - type: boolean - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - capabilities: - type: object - properties: - add: - type: array - items: - type: string - drop: - type: array - items: - type: string - privileged: - type: boolean - procMount: - type: string - readOnlyRootFilesystem: - type: boolean - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Security context for the container. - volumeMounts: - type: array - items: - type: object - properties: - mountPath: - type: string - mountPropagation: - type: string - name: - type: string - readOnly: - type: boolean - recursiveReadOnly: - type: string - subPath: - type: string - subPathExpr: - type: string - description: Additional volume mounts which should be applied to the container. - description: Template for the Entity Operator TLS sidecar container. serviceAccount: type: object properties: @@ -4992,6 +3322,28 @@ spec: description: Annotations added to the Kubernetes resource. description: Metadata applied to the resource. description: Template for the Entity Operator service account. + podDisruptionBudget: + type: object + properties: + metadata: + type: object + properties: + labels: + additionalProperties: + type: string + type: object + description: Labels added to the Kubernetes resource. + annotations: + additionalProperties: + type: string + type: object + description: Annotations added to the Kubernetes resource. + description: Metadata to apply to the `PodDisruptionBudgetTemplate` resource. + maxUnavailable: + type: integer + minimum: 0 + description: "Maximum number of unavailable pods to allow automatic Pod eviction. A Pod eviction is allowed when the `maxUnavailable` number of pods or fewer are unavailable after the eviction. Setting this value to 0 prevents all voluntary evictions, so the pods must be evicted manually. Defaults to 1." + description: Template for the Entity Operator Pod Disruption Budget. entityOperatorRole: type: object properties: @@ -5102,102 +3454,6 @@ spec: image: type: string description: "The container image used for Cruise Control pods. If no image name is explicitly specified, the image name corresponds to the name specified in the Cluster Operator configuration. If an image name is not defined in the Cluster Operator configuration, a default value is used." - tlsSidecar: - type: object - properties: - image: - type: string - description: The docker image for the container. - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. - livenessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod liveness checking. - readinessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod readiness checking. - logLevel: - type: string - enum: - - emerg - - alert - - crit - - err - - warning - - notice - - info - - debug - description: The log level for the TLS sidecar. Default value is `notice`. - description: TLS sidecar configuration. resources: type: object properties: @@ -5866,6 +4122,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -5901,7 +4186,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -5922,20 +4207,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -5943,7 +4228,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -5962,7 +4247,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -5970,7 +4263,11 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Cruise Control `Pods`. apiService: type: object @@ -6170,149 +4467,6 @@ spec: type: string description: Additional volume mounts which should be applied to the container. description: Template for the Cruise Control container. - tlsSidecarContainer: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: The environment variable key. - value: - type: string - description: The environment variable value. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a config map. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Reference to the secret or config map property to which the environment variable is set. - oneOf: - - properties: - value: {} - required: - - value - - properties: - valueFrom: {} - required: - - valueFrom - description: Environment variables which should be applied to the container. - securityContext: - type: object - properties: - allowPrivilegeEscalation: - type: boolean - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - capabilities: - type: object - properties: - add: - type: array - items: - type: string - drop: - type: array - items: - type: string - privileged: - type: boolean - procMount: - type: string - readOnlyRootFilesystem: - type: boolean - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Security context for the container. - volumeMounts: - type: array - items: - type: object - properties: - mountPath: - type: string - mountPropagation: - type: string - name: - type: string - readOnly: - type: boolean - recursiveReadOnly: - type: string - subPath: - type: string - subPathExpr: - type: string - description: Additional volume mounts which should be applied to the container. - description: Template for the Cruise Control TLS sidecar container. serviceAccount: type: object properties: @@ -6335,15 +4489,6 @@ spec: brokerCapacity: type: object properties: - disk: - type: string - pattern: "^[0-9]+([.][0-9]*)?([KMGTPE]i?|e[0-9]+)?$" - description: "Broker capacity for disk in bytes. Use a number value with either standard Kubernetes byte units (K, M, G, or T), their bibyte (power of two) equivalents (Ki, Mi, Gi, or Ti), or a byte value with or without E notation. For example, 100000M, 100000Mi, 104857600000, or 1e+11." - cpuUtilization: - type: integer - minimum: 0 - maximum: 100 - description: Broker capacity for CPU resource utilization as a percentage (0 - 100). cpu: type: string pattern: "^[0-9]+([.][0-9]{0,3}|[m]?)$" @@ -6393,7 +4538,8 @@ spec: type: string enum: - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. + - strimziMetricsReporter + description: Metrics type. The supported types are `jmxPrometheusExporter` and `strimziMetricsReporter`. Type `jmxPrometheusExporter` uses the Prometheus JMX Exporter to expose Kafka JMX metrics in Prometheus format through an HTTP endpoint. Type `strimziMetricsReporter` uses the Strimzi Metrics Reporter to directly expose Kafka metrics in Prometheus format through an HTTP endpoint. valueFrom: type: object properties: @@ -6407,11 +4553,24 @@ spec: optional: type: boolean description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' + description: ConfigMap entry where the Prometheus JMX Exporter configuration is stored. + values: + type: object + properties: + allowList: + type: array + items: + type: string + description: A list of regex patterns to filter the metrics to collect. Should contain at least one element. + description: Configuration values for the Strimzi Metrics Reporter. required: - type - - valueFrom - description: Metrics configuration. + description: "Metrics configuration. Only `jmxPrometheusExporter` can be configured, as this component does not support `strimziMetricsReporter`." + x-kubernetes-validations: + - rule: self.type != 'jmxPrometheusExporter' || has(self.valueFrom) + message: valueFrom property is required + - rule: self.type != 'strimziMetricsReporter' + message: value type not supported apiUsers: type: object properties: @@ -6460,896 +4619,6 @@ spec: - mode description: "Auto-rebalancing on scaling related configuration listing the modes, when brokers are added or removed, with the corresponding rebalance template configurations.If this field is set, at least one mode has to be defined." description: Configuration for Cruise Control deployment. Deploys a Cruise Control instance when specified. - jmxTrans: - type: object - properties: - image: - type: string - description: The image to use for the JmxTrans. - outputDefinitions: - type: array - items: - type: object - properties: - outputType: - type: string - description: "Template for setting the format of the data that will be pushed.For more information see https://github.com/jmxtrans/jmxtrans/wiki/OutputWriters[JmxTrans OutputWriters]." - host: - type: string - description: The DNS/hostname of the remote host that the data is pushed to. - port: - type: integer - description: The port of the remote host that the data is pushed to. - flushDelayInSeconds: - type: integer - description: How many seconds the JmxTrans waits before pushing a new set of data out. - typeNames: - type: array - items: - type: string - description: "Template for filtering data to be included in response to a wildcard query. For more information see https://github.com/jmxtrans/jmxtrans/wiki/Queries[JmxTrans queries]." - name: - type: string - description: Template for setting the name of the output definition. This is used to identify where to send the results of queries should be sent. - required: - - outputType - - name - description: "Defines the output hosts that will be referenced later on. For more information on these properties see, xref:type-JmxTransOutputDefinitionTemplate-reference[`JmxTransOutputDefinitionTemplate` schema reference]." - logLevel: - type: string - description: "Sets the logging level of the JmxTrans deployment.For more information see, https://github.com/jmxtrans/jmxtrans-agent/wiki/Troubleshooting[JmxTrans Logging Level]." - kafkaQueries: - type: array - items: - type: object - properties: - targetMBean: - type: string - description: If using wildcards instead of a specific MBean then the data is gathered from multiple MBeans. Otherwise if specifying an MBean then data is gathered from that specified MBean. - attributes: - type: array - items: - type: string - description: Determine which attributes of the targeted MBean should be included. - outputs: - type: array - items: - type: string - description: "List of the names of output definitions specified in the spec.kafka.jmxTrans.outputDefinitions that have defined where JMX metrics are pushed to, and in which data format." - required: - - targetMBean - - attributes - - outputs - description: "Queries to send to the Kafka brokers to define what data should be read from each broker. For more information on these properties see, xref:type-JmxTransQueryTemplate-reference[`JmxTransQueryTemplate` schema reference]." - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. - template: - type: object - properties: - deployment: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - deploymentStrategy: - type: string - enum: - - RollingUpdate - - Recreate - description: Pod replacement strategy for deployment configuration changes. Valid values are `RollingUpdate` and `Recreate`. Defaults to `RollingUpdate`. - description: Template for JmxTrans `Deployment`. - pod: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - imagePullSecrets: - type: array - items: - type: object - properties: - name: - type: string - description: "List of references to secrets in the same namespace to use for pulling any of the images used by this Pod. When the `STRIMZI_IMAGE_PULL_SECRETS` environment variable in Cluster Operator and the `imagePullSecrets` option are specified, only the `imagePullSecrets` variable is used and the `STRIMZI_IMAGE_PULL_SECRETS` variable is ignored." - securityContext: - type: object - properties: - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - fsGroup: - type: integer - fsGroupChangePolicy: - type: string - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxChangePolicy: - type: string - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - supplementalGroups: - type: array - items: - type: integer - supplementalGroupsPolicy: - type: string - sysctls: - type: array - items: - type: object - properties: - name: - type: string - value: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Configures pod-level security attributes and common container settings. - terminationGracePeriodSeconds: - type: integer - minimum: 0 - description: "The grace period is the duration in seconds after the processes running in the pod are sent a termination signal, and the time when the processes are forcibly halted with a kill signal. Set this value to longer than the expected cleanup time for your process. Value must be a non-negative integer. A zero value indicates delete immediately. You might need to increase the grace period for very large Kafka clusters, so that the Kafka brokers have enough time to transfer their work to another broker before they are terminated. Defaults to 30 seconds." - affinity: - type: object - properties: - nodeAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - preference: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: object - properties: - nodeSelectorTerms: - type: array - items: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - podAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - podAntiAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - description: The pod's affinity rules. - tolerations: - type: array - items: - type: object - properties: - effect: - type: string - key: - type: string - operator: - type: string - tolerationSeconds: - type: integer - value: - type: string - description: The pod's tolerations. - topologySpreadConstraints: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - maxSkew: - type: integer - minDomains: - type: integer - nodeAffinityPolicy: - type: string - nodeTaintsPolicy: - type: string - topologyKey: - type: string - whenUnsatisfiable: - type: string - description: The pod's topology spread constraints. - priorityClassName: - type: string - description: 'The name of the priority class used to assign priority to the pods. ' - schedulerName: - type: string - description: "The name of the scheduler used to dispatch this `Pod`. If not specified, the default scheduler will be used." - hostAliases: - type: array - items: - type: object - properties: - hostnames: - type: array - items: - type: string - ip: - type: string - description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. - enableServiceLinks: - type: boolean - description: Indicates whether information about services should be injected into Pod's environment variables. - tmpDirSizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "Defines the total amount of pod memory allocated for the temporary `EmptyDir` volume `/tmp`. Specify the allocation in memory units, for example, `100Mi` for 100 mebibytes. Default value is `5Mi`. The `/tmp` volume is backed by pod memory, not disk storage, so avoid setting a high value as it consumes pod memory resources." - volumes: - type: array - items: - type: object - properties: - name: - type: string - description: Name to use for the volume. Required. - secret: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - optional: - type: boolean - secretName: - type: string - description: Secret to use populate the volume. - configMap: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - name: - type: string - optional: - type: boolean - description: ConfigMap to use to populate the volume. - emptyDir: - type: object - properties: - medium: - type: string - sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. - persistentVolumeClaim: - type: object - properties: - claimName: - type: string - readOnly: - type: boolean - description: PersistentVolumeClaim object to use to populate the volume. - csi: - type: object - properties: - driver: - type: string - fsType: - type: string - nodePublishSecretRef: - type: object - properties: - name: - type: string - readOnly: - type: boolean - volumeAttributes: - additionalProperties: - type: string - type: object - description: CSIVolumeSource object to use to populate the volume. - oneOf: - - properties: - secret: {} - configMap: {} - emptyDir: {} - persistentVolumeClaim: {} - csi: {} - description: Additional volumes that can be mounted to the pod. - description: Template for JmxTrans `Pods`. - container: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: The environment variable key. - value: - type: string - description: The environment variable value. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a config map. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Reference to the secret or config map property to which the environment variable is set. - oneOf: - - properties: - value: {} - required: - - value - - properties: - valueFrom: {} - required: - - valueFrom - description: Environment variables which should be applied to the container. - securityContext: - type: object - properties: - allowPrivilegeEscalation: - type: boolean - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - capabilities: - type: object - properties: - add: - type: array - items: - type: string - drop: - type: array - items: - type: string - privileged: - type: boolean - procMount: - type: string - readOnlyRootFilesystem: - type: boolean - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Security context for the container. - volumeMounts: - type: array - items: - type: object - properties: - mountPath: - type: string - mountPropagation: - type: string - name: - type: string - readOnly: - type: boolean - recursiveReadOnly: - type: string - subPath: - type: string - subPathExpr: - type: string - description: Additional volume mounts which should be applied to the container. - description: Template for JmxTrans container. - serviceAccount: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for the JmxTrans service account. - description: Template for JmxTrans resources. - required: - - outputDefinitions - - kafkaQueries - description: "As of Strimzi 0.35.0, JMXTrans is not supported anymore and this option is ignored." kafkaExporter: type: object properties: @@ -7982,6 +5251,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -8017,7 +5315,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -8038,20 +5336,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -8059,7 +5357,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -8078,7 +5376,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -8086,26 +5392,12 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka Exporter `Pods`. - service: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for Kafka Exporter `Service`. container: type: object properties: @@ -8267,6 +5559,28 @@ spec: description: Annotations added to the Kubernetes resource. description: Metadata applied to the resource. description: Template for the Kafka Exporter service account. + podDisruptionBudget: + type: object + properties: + metadata: + type: object + properties: + labels: + additionalProperties: + type: string + type: object + description: Labels added to the Kubernetes resource. + annotations: + additionalProperties: + type: string + type: object + description: Annotations added to the Kubernetes resource. + description: Metadata to apply to the `PodDisruptionBudgetTemplate` resource. + maxUnavailable: + type: integer + minimum: 0 + description: "Maximum number of unavailable pods to allow automatic Pod eviction. A Pod eviction is allowed when the `maxUnavailable` number of pods or fewer are unavailable after the eviction. Setting this value to 0 prevents all voluntary evictions, so the pods must be evicted manually. Defaults to 1." + description: Template for the Pod Disruption Budget for Kafka Exporter pods. description: Customization of deployment templates and pods. description: "Configuration of the Kafka Exporter. Kafka Exporter can provide additional metrics, for example lag of consumer group at topic/partition." maintenanceTimeWindows: @@ -8276,7 +5590,7 @@ spec: description: "A list of time windows for maintenance tasks (that is, certificates renewal). Each time window is defined by a cron expression." required: - kafka - description: "The specification of the Kafka and ZooKeeper clusters, and Topic Operator." + description: The specification of the Kafka cluster. status: type: object properties: @@ -8309,9 +5623,6 @@ spec: items: type: object properties: - type: - type: string - description: The name of the listener. name: type: string description: The name of the listener. @@ -8345,11 +5656,6 @@ spec: type: string description: The name of the KafkaNodePool used by this Kafka resource. description: List of the KafkaNodePools used by this Kafka cluster. - registeredNodeIds: - type: array - items: - type: integer - description: Registered node IDs used by this Kafka cluster. This field is used for internal purposes only and will be removed in the future. clusterId: type: string description: Kafka cluster Id. @@ -8362,16 +5668,6 @@ spec: kafkaMetadataVersion: type: string description: The KRaft metadata.version currently used by the Kafka cluster. - kafkaMetadataState: - type: string - enum: - - ZooKeeper - - KRaftMigration - - KRaftDualWriting - - KRaftPostMigration - - PreKRaft - - KRaft - description: "Defines where cluster metadata are stored. Possible values are: ZooKeeper if the metadata are stored in ZooKeeper; KRaftMigration if the controllers are connected to ZooKeeper, brokers are being rolled with Zookeeper migration enabled and connection information to controllers, and the metadata migration process is running; KRaftDualWriting if the metadata migration process finished and the cluster is in dual-write mode; KRaftPostMigration if the brokers are fully KRaft-based but controllers being rolled to disconnect from ZooKeeper; PreKRaft if brokers and controller are fully KRaft-based, metadata are stored in KRaft, but ZooKeeper must be deleted; KRaft if the metadata are stored in KRaft." autoRebalance: type: object properties: @@ -8403,4 +5699,11 @@ spec: description: "List of broker IDs involved in an auto-rebalancing operation related to the current mode. \nThe list contains one of the following: \n\n* Broker IDs for a current auto-rebalance. \n* Broker IDs for a queued auto-rebalance (if a previous auto-rebalance is still in progress). \n" description: "List of modes where an auto-rebalancing operation is either running or queued. \nEach mode entry (`add-brokers` or `remove-brokers`) includes one of the following: \n\n* Broker IDs for a current auto-rebalance. \n* Broker IDs for a queued auto-rebalance (if a previous rebalance is still in progress)." description: The status of an auto-rebalancing triggered by a cluster scaling request. - description: "The status of the Kafka and ZooKeeper clusters, and Topic Operator." + clusterSecurity: + type: object + properties: {} + x-kubernetes-preserve-unknown-fields: true + description: The current security configuration of the Kafka cluster. + description: The status of the Kafka cluster. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/041-Crd-kafkaconnect.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/041-Crd-kafkaconnect.yaml index 2aa3fcf32..2f546f518 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/041-Crd-kafkaconnect.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/041-Crd-kafkaconnect.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -59,13 +59,25 @@ spec: description: The Kafka Connect version. Defaults to the latest version. Consult the user documentation to understand the process required to upgrade or downgrade the version. replicas: type: integer - description: The number of pods in the Kafka Connect group. Defaults to `3`. + description: The number of pods in the Kafka Connect group. Required in the `v1` version of the Strimzi API. image: type: string description: "The container image used for Kafka Connect pods. If no image name is explicitly specified, it is determined based on the `spec.version` configuration. The image names are specifically mapped to corresponding versions in the Cluster Operator configuration." bootstrapServers: type: string description: Bootstrap servers to connect to. This should be given as a comma separated list of __:__ pairs. + groupId: + type: string + description: A unique ID that identifies the Connect cluster group. + configStorageTopic: + type: string + description: The name of the Kafka topic where connector configurations are stored. + statusStorageTopic: + type: string + description: The name of the Kafka topic where connector and task status are stored. + offsetStorageTopic: + type: string + description: The name of the Kafka topic where source connector offsets are stored. tls: type: object properties: @@ -99,28 +111,6 @@ spec: authentication: type: object properties: - accessToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the access token which was obtained from the authorization server. - accessTokenIsJwt: - type: boolean - description: Configure whether access token should be treated as JWT. This should be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - accessTokenLocation: - type: string - description: Path to the token file containing an access token to be used for authentication. - audience: - type: string - description: "OAuth audience to use when authenticating against the authorization server. Some authorization servers require the audience to be explicitly set. The possible values depend on how the authorization server is configured. By default, `audience` is not specified when performing the token endpoint request." certificateAndKey: type: object properties: @@ -132,68 +122,16 @@ spec: description: The name of the file certificate in the Secret. key: type: string - description: The name of the private key in the Secret. + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." required: - secretName - certificate - key description: Reference to the `Secret` which holds the certificate and private key pair. - clientAssertion: + config: + x-kubernetes-preserve-unknown-fields: true type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes secret containing the client assertion which was manually configured for the client. - clientAssertionLocation: - type: string - description: Path to the file containing the client assertion to be used for authentication. - clientAssertionType: - type: string - description: "The client assertion type. If not set, and either `clientAssertion` or `clientAssertionLocation` is configured, this value defaults to `urn:ietf:params:oauth:client-assertion-type:jwt-bearer`." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - maxTokenExpirySeconds: - type: integer - description: Set or limit time-to-live of the access tokens to the specified number of seconds. This should be set if the authorization server returns opaque tokens. + description: Configuration for the custom authentication mechanism. Only properties with the `sasl.` and `ssl.keystore.` prefixes are allowed. Specify other options in the regular configuration section of the custom resource. passwordSecret: type: object properties: @@ -207,59 +145,9 @@ spec: - secretName - password description: Reference to the `Secret` which holds the password. - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." - refreshToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the refresh token which can be used to obtain access token from the authorization server. - saslExtensions: - additionalProperties: - type: string - type: object - description: SASL extensions parameters. - scope: - type: string - description: OAuth scope to use when authenticating against the authorization server. Some authorization servers require this to be set. The possible values depend on how authorization server is configured. By default `scope` is not specified when doing the token endpoint request. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. + sasl: + type: boolean + description: Enable or disable SASL on this authentication mechanism. type: type: string enum: @@ -267,8 +155,8 @@ spec: - scram-sha-256 - scram-sha-512 - plain - - oauth - description: "Authentication type. Currently the supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and 'oauth'. `scram-sha-256` and `scram-sha-512` types use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 Authentication, respectively. `plain` type uses SASL PLAIN Authentication. `oauth` type uses SASL OAUTHBEARER Authentication. The `tls` type uses TLS Client Authentication. The `tls` type is supported only over TLS connections." + - custom + description: "Specifies the authentication type. Supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and `custom`. `tls` uses TLS client authentication and is supported only over TLS connections. `scram-sha-256` and `scram-sha-512` use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 authentication, respectively. `plain` uses SASL PLAIN authentication. `custom` allows you to configure a custom authentication mechanism." username: type: string description: Username used for the authentication. @@ -278,7 +166,7 @@ spec: config: x-kubernetes-preserve-unknown-fields: true type: object - description: "The Kafka Connect configuration. Properties with the following prefixes cannot be set: ssl., sasl., security., listeners, plugin.path, rest., bootstrap.servers, consumer.interceptor.classes, producer.interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." + description: "The Kafka Connect configuration. Properties with the following prefixes cannot be set: group.id, config.storage.topic, offset.storage.topic, status.storage.topic, ssl., sasl., security., listeners, plugin.path, rest., bootstrap.servers, consumer.interceptor.classes, producer.interceptor.classes, prometheus.metrics.reporter. (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." resources: type: object properties: @@ -440,13 +328,25 @@ spec: rack: type: object properties: + envVarName: + type: string + description: The name of the environment variable that defines the rack ID. Its value sets the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect or MirrorMaker 2. topologyKey: type: string example: topology.kubernetes.io/zone description: "A key that matches labels assigned to the Kubernetes cluster nodes. The value of the label is used to set a broker's `broker.rack` config, and the `client.rack` config for Kafka Connect or MirrorMaker 2." - required: - - topologyKey + type: + type: string + enum: + - topology-label + - environment-variable + description: "Specifies the rack awareness type. Supported types are `topology-label` and `environment-variable`. `topology-label` uses a Kubernetes worker node label to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. `environment-variable` uses an environment variable to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. When not specified, `topology-label` type is used by default." description: Configuration of the node label which will be used as the `client.rack` consumer configuration. + x-kubernetes-validations: + - rule: (has(self.type) && self.type != "topology-label") || self.topologyKey != "" + message: topologyKey property is required + - rule: has(self.type) == false || self.type != "environment-variable" || self.envVarName != "" + message: envVarName property is required metricsConfig: type: object properties: @@ -454,7 +354,8 @@ spec: type: string enum: - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. + - strimziMetricsReporter + description: Metrics type. The supported types are `jmxPrometheusExporter` and `strimziMetricsReporter`. Type `jmxPrometheusExporter` uses the Prometheus JMX Exporter to expose Kafka JMX metrics in Prometheus format through an HTTP endpoint. Type `strimziMetricsReporter` uses the Strimzi Metrics Reporter to directly expose Kafka metrics in Prometheus format through an HTTP endpoint. valueFrom: type: object properties: @@ -468,50 +369,36 @@ spec: optional: type: boolean description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' + description: ConfigMap entry where the Prometheus JMX Exporter configuration is stored. + values: + type: object + properties: + allowList: + type: array + items: + type: string + description: A list of regex patterns to filter the metrics to collect. Should contain at least one element. + description: Configuration values for the Strimzi Metrics Reporter. required: - type - - valueFrom description: Metrics configuration. + x-kubernetes-validations: + - rule: self.type != 'jmxPrometheusExporter' || has(self.valueFrom) + message: valueFrom property is required tracing: type: object properties: type: type: string enum: - - jaeger - opentelemetry - description: "Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. As of Strimzi 0.37.0, `jaeger` type is not supported anymore and this option is ignored." + description: Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. required: - type description: The configuration of tracing in Kafka Connect. template: type: object properties: - deployment: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - deploymentStrategy: - type: string - enum: - - RollingUpdate - - Recreate - description: Pod replacement strategy for deployment configuration changes. Valid values are `RollingUpdate` and `Recreate`. Defaults to `RollingUpdate`. - description: Template for Kafka Connect `Deployment`. podSet: type: object properties: @@ -1031,6 +918,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -1066,7 +982,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -1087,20 +1003,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -1108,7 +1024,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -1127,7 +1043,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -1135,7 +1059,75 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + templatedVolumes: + type: array + items: + type: object + properties: + name: + type: string + description: Name to use for the volume. Required. + secret: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + optional: + type: boolean + secretName: + type: string + description: "`Secret` to use to populate the volume. The name of the Secret and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + configMap: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + name: + type: string + optional: + type: boolean + description: "`ConfigMap` to use to populate the volume. The name of the ConfigMap and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + persistentVolumeClaim: + type: object + properties: + claimName: + type: string + readOnly: + type: boolean + description: "`PersistentVolumeClaim` object to use to populate the volume. The name of the Persistent Volume Claim can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + oneOf: + - properties: + secret: {} + configMap: {} + persistentVolumeClaim: {} + description: Additional volumes that can be mounted to the pod. These volumes can use templates to mount different volumes into individual Pods. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka Connect `Pods`. apiService: type: object @@ -2048,6 +2040,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -2083,7 +2104,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -2104,20 +2125,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -2125,7 +2146,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -2144,7 +2165,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -2152,7 +2181,11 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka Connect Build `Pods`. The build pod is used only on Kubernetes. buildContainer: type: object @@ -2355,128 +2388,22 @@ spec: description: Metadata applied to the resource. description: Template for Secret of the Kafka Connect Cluster JMX authentication. description: "Template for Kafka Connect and Kafka MirrorMaker 2 resources. The template allows users to specify how the `Pods`, `Service`, and other services are generated." - externalConfiguration: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: Name of the environment variable which will be passed to the Kafka Connect pods. The name of the environment variable cannot start with `KAFKA_` or `STRIMZI_`. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a Secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a ConfigMap. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Value of the environment variable which will be passed to the Kafka Connect pods. It can be passed either as a reference to Secret or ConfigMap field. The field has to specify exactly one Secret or ConfigMap. - required: - - name - - valueFrom - description: Makes data from a Secret or ConfigMap available in the Kafka Connect pods as environment variables. - volumes: - type: array - items: - type: object - properties: - name: - type: string - description: Name of the volume which will be added to the Kafka Connect pods. - secret: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - optional: - type: boolean - secretName: - type: string - description: Reference to a key in a Secret. Exactly one Secret or ConfigMap has to be specified. - configMap: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a ConfigMap. Exactly one Secret or ConfigMap has to be specified. - oneOf: - - properties: - secret: {} - required: - - secret - - properties: - configMap: {} - required: - - configMap - required: - - name - description: Makes data from a Secret or ConfigMap available in the Kafka Connect pods as volumes. - description: Pass data from Secrets or ConfigMaps to the Kafka Connect pods and use them to configure connectors. build: type: object properties: output: type: object properties: - additionalKanikoOptions: + additionalBuildOptions: type: array items: type: string - description: "Configures additional options which will be passed to the Kaniko executor when building the new Connect image. Allowed options are: --customPlatform, --custom-platform, --insecure, --insecure-pull, --insecure-registry, --log-format, --log-timestamp, --registry-mirror, --reproducible, --single-snapshot, --skip-tls-verify, --skip-tls-verify-pull, --skip-tls-verify-registry, --verbosity, --snapshotMode, --use-new-run, --registry-certificate, --registry-client-cert, --ignore-path. These options will be used only on Kubernetes where the Kaniko executor is used. They will be ignored on OpenShift. The options are described in the link:https://github.com/GoogleContainerTools/kaniko[Kaniko GitHub repository^]. Changing this field does not trigger new build of the Kafka Connect image." + description: "Configures additional options to pass to the `build` command of either Kaniko or Buildah (depending on the feature gate setting) when building a new Kafka Connect image. Allowed Kaniko options: --customPlatform, --custom-platform, --insecure, --insecure-pull, --insecure-registry, --log-format, --log-timestamp, --registry-mirror, --reproducible, --single-snapshot, --skip-tls-verify, --skip-tls-verify-pull, --skip-tls-verify-registry, --verbosity, --snapshotMode, --use-new-run, --registry-certificate, --registry-client-cert, --ignore-path. Allowed Buildah `build` options: --authfile, --cert-dir, --creds, --decryption-key, --retry, --retry-delay, --tls-verify. Those options are used only on Kubernetes, where Kaniko and Buildah are available. They are ignored on OpenShift. For more information, see the link:https://github.com/GoogleContainerTools/kaniko[Kaniko GitHub repository^] or the link:https://github.com/containers/buildah/blob/main/docs/buildah-build.1.md[Buildah build document^]. Changing this field does not trigger a rebuild of the Kafka Connect image." + additionalPushOptions: + type: array + items: + type: string + description: "Configures additional options to pass to the Buildah `push` command when pushing a new Connect image. Allowed options: --authfile, --cert-dir, --creds, --quiet, --retry, --retry-delay, --tls-verify. Those options are used only on Kubernetes, where Buildah is available. They are ignored on OpenShift. For more information, see the link:https://github.com/containers/buildah/blob/main/docs/buildah-push.1.md[Buildah push document^]. Changing this field does not trigger a rebuild of the Kafka Connect image." image: type: string description: The name of the image which will be built. Required. @@ -2516,9 +2443,29 @@ spec: group: type: string description: Maven group id. Applicable to the `maven` artifact type only. + includeScope: + type: string + enum: + - compile + - provided + - runtime + - test + - system + description: "Maven scope for including dependencies for the artifact. Valid values are `compile`, `provided`, `runtime`, `test`, and `system`. When not configured, no scope is set, and all dependencies are included. Applicable to the `maven` artifact type only." insecure: type: boolean description: "By default, connections using TLS are verified to check they are secure. The server certificate used must be valid, trusted, and contain the server name. By setting this option to `true`, all TLS verification is disabled and the artifact will be downloaded, even when the server is considered insecure." + mirrors: + type: array + items: + type: object + properties: + url: + type: string + description: "The URL of the Maven mirror. All repository requests during the build, including plugin repositories and Maven Central, are redirected to this mirror. Applicable to the `maven` artifact type only." + required: + - url + description: "List of Maven mirrors used to download the artifact and its dependencies. All repository requests during the build, including plugin repositories and Maven Central, are redirected to the configured mirror(s). Applicable to the `maven` artifact type only." repository: type: string description: Maven repository to download the artifact from. Applicable to the `maven` artifact type only. @@ -2581,8 +2528,55 @@ spec: - output - plugins description: Configures how the Connect container image should be built. Optional. + plugins: + type: array + items: + type: object + properties: + name: + type: string + pattern: "^[a-z0-9][-_a-z0-9]*[a-z0-9]$" + description: "A unique name for the connector plugin. This name is used to generate the mount path for the connector artifacts. The name has to be unique within the KafkaConnect resource. The name must be unique within the `KafkaConnect` resource and match the pattern: `^[a-z][-_a-z0-9]*[a-z]$`. Required." + artifacts: + type: array + items: + type: object + properties: + pullPolicy: + type: string + description: |- + Policy that determines when the container image (OCI artifact) is pulled. + + Possible values are: + + * `Always`: Always pull the image. If the pull fails, container creation fails. + * `Never`: Never pull the image. Use only a locally available image. Container creation fails if the image isn’t present. + * `IfNotPresent`: Pull the image only if it’s not already available locally. Container creation fails if the image isn’t present and the pull fails. + + Defaults to `Always` if `:latest` tag is specified, or `IfNotPresent` otherwise. + reference: + type: string + description: Reference to the container image (OCI artifact) containing the Kafka Connect plugin. The image is mounted as a volume and provides the plugin binary. Required. + type: + type: string + enum: + - image + description: "Artifact type. Currently, the only supported artifact type is `image`." + required: + - reference + - type + description: List of artifacts associated with this connector plugin. Required. + required: + - name + - artifacts + description: List of connector plugins to mount into the `KafkaConnect` pod. required: + - replicas - bootstrapServers + - groupId + - configStorageTopic + - statusStorageTopic + - offsetStorageTopic description: The specification of the Kafka Connect cluster. status: type: object @@ -2636,3 +2630,5 @@ spec: type: string description: Label selector for pods providing this resource. description: The status of the Kafka Connect cluster. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/042-Crd-strimzipodset.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/042-Crd-strimzipodset.yaml index 4832407aa..4e4547738 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/042-Crd-strimzipodset.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/042-Crd-strimzipodset.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -126,3 +126,5 @@ spec: type: integer description: Number of pods managed by this `StrimziPodSet` resource that have the current revision. description: The status of the StrimziPodSet. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/043-Crd-kafkatopic.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/043-Crd-kafkatopic.yaml index 2f84c462a..7957e1cf4 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/043-Crd-kafkatopic.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/043-Crd-kafkatopic.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -128,217 +128,5 @@ spec: description: The session identifier for replicas change requests pertaining to this KafkaTopic resource. This is used by the Topic Operator to track the status of `ongoing` replicas change operations. description: Replication factor change status. description: The status of the topic. - - name: v1beta1 - served: true - storage: false - subresources: - status: {} - additionalPrinterColumns: - - name: Cluster - description: The name of the Kafka cluster this topic belongs to - jsonPath: .metadata.labels.strimzi\.io/cluster - type: string - - name: Partitions - description: The desired number of partitions in the topic - jsonPath: .spec.partitions - type: integer - - name: Replication factor - description: The desired number of replicas of each partition - jsonPath: .spec.replicas - type: integer - - name: Ready - description: The state of the custom resource - jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" - type: string - schema: - openAPIV3Schema: - type: object - properties: - apiVersion: - type: string - description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources" - kind: - type: string - description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds" - metadata: - type: object - spec: - type: object - properties: - topicName: - type: string - description: The name of the topic. When absent this will default to the metadata.name of the topic. It is recommended to not set this unless the topic name is not a valid Kubernetes resource name. - partitions: - type: integer - minimum: 1 - description: "The number of partitions the topic should have. This cannot be decreased after topic creation. It can be increased after topic creation, but it is important to understand the consequences that has, especially for topics with semantic partitioning. When absent this will default to the broker configuration for `num.partitions`." - replicas: - type: integer - minimum: 1 - maximum: 32767 - description: The number of replicas the topic should have. When absent this will default to the broker configuration for `default.replication.factor`. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: The topic configuration. - description: The specification of the topic. - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - type: - type: string - description: "The unique identifier of a condition, used to distinguish between other conditions in the resource." - status: - type: string - description: "The status of the condition, either True, False or Unknown." - lastTransitionTime: - type: string - description: "Last time the condition of a type changed from one status to another. The required format is 'yyyy-MM-ddTHH:mm:ssZ', in the UTC time zone." - reason: - type: string - description: The reason for the condition's last transition (a single word in CamelCase). - message: - type: string - description: Human-readable message indicating details about the condition's last transition. - description: List of status conditions. - observedGeneration: - type: integer - description: The generation of the CRD that was last reconciled by the operator. - topicName: - type: string - description: Topic name. - topicId: - type: string - description: "The topic's id. For a KafkaTopic with the ready condition, this will change only if the topic gets deleted and recreated with the same name." - replicasChange: - type: object - properties: - targetReplicas: - type: integer - description: The target replicas value requested by the user. This may be different from .spec.replicas when a change is ongoing. - state: - type: string - enum: - - pending - - ongoing - description: "Current state of the replicas change operation. This can be `pending`, when the change has been requested, or `ongoing`, when the change has been successfully submitted to Cruise Control." - message: - type: string - description: Message for the user related to the replicas change request. This may contain transient error messages that would disappear on periodic reconciliations. - sessionId: - type: string - description: The session identifier for replicas change requests pertaining to this KafkaTopic resource. This is used by the Topic Operator to track the status of `ongoing` replicas change operations. - description: Replication factor change status. - description: The status of the topic. - - name: v1alpha1 - served: true - storage: false - subresources: - status: {} - additionalPrinterColumns: - - name: Cluster - description: The name of the Kafka cluster this topic belongs to - jsonPath: .metadata.labels.strimzi\.io/cluster - type: string - - name: Partitions - description: The desired number of partitions in the topic - jsonPath: .spec.partitions - type: integer - - name: Replication factor - description: The desired number of replicas of each partition - jsonPath: .spec.replicas - type: integer - - name: Ready - description: The state of the custom resource - jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" - type: string - schema: - openAPIV3Schema: - type: object - properties: - apiVersion: - type: string - description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources" - kind: - type: string - description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds" - metadata: - type: object - spec: - type: object - properties: - topicName: - type: string - description: The name of the topic. When absent this will default to the metadata.name of the topic. It is recommended to not set this unless the topic name is not a valid Kubernetes resource name. - partitions: - type: integer - minimum: 1 - description: "The number of partitions the topic should have. This cannot be decreased after topic creation. It can be increased after topic creation, but it is important to understand the consequences that has, especially for topics with semantic partitioning. When absent this will default to the broker configuration for `num.partitions`." - replicas: - type: integer - minimum: 1 - maximum: 32767 - description: The number of replicas the topic should have. When absent this will default to the broker configuration for `default.replication.factor`. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: The topic configuration. - description: The specification of the topic. - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - type: - type: string - description: "The unique identifier of a condition, used to distinguish between other conditions in the resource." - status: - type: string - description: "The status of the condition, either True, False or Unknown." - lastTransitionTime: - type: string - description: "Last time the condition of a type changed from one status to another. The required format is 'yyyy-MM-ddTHH:mm:ssZ', in the UTC time zone." - reason: - type: string - description: The reason for the condition's last transition (a single word in CamelCase). - message: - type: string - description: Human-readable message indicating details about the condition's last transition. - description: List of status conditions. - observedGeneration: - type: integer - description: The generation of the CRD that was last reconciled by the operator. - topicName: - type: string - description: Topic name. - topicId: - type: string - description: "The topic's id. For a KafkaTopic with the ready condition, this will change only if the topic gets deleted and recreated with the same name." - replicasChange: - type: object - properties: - targetReplicas: - type: integer - description: The target replicas value requested by the user. This may be different from .spec.replicas when a change is ongoing. - state: - type: string - enum: - - pending - - ongoing - description: "Current state of the replicas change operation. This can be `pending`, when the change has been requested, or `ongoing`, when the change has been successfully submitted to Cruise Control." - message: - type: string - description: Message for the user related to the replicas change request. This may contain transient error messages that would disappear on periodic reconciliations. - sessionId: - type: string - description: The session identifier for replicas change requests pertaining to this KafkaTopic resource. This is used by the Topic Operator to track the status of `ongoing` replicas change operations. - description: Replication factor change status. - description: The status of the topic. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/044-Crd-kafkauser.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/044-Crd-kafkauser.yaml index 9f015abf2..a2b43c1cb 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/044-Crd-kafkauser.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/044-Crd-kafkauser.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -81,6 +81,10 @@ spec: required: - valueFrom description: "Specify the password for the user. If not set, a new password is generated by the User Operator." + renewalDays: + type: integer + minimum: 1 + description: "Number of days before certificate expiration when the user certificate is renewed. Both this property and `validityDays` must be configured together.The value must be greater than 0 and less than `validityDays`.If not configured, the Clients CA configuration is used." type: type: string enum: @@ -88,9 +92,20 @@ spec: - tls-external - scram-sha-512 description: Authentication type. + validityDays: + type: integer + minimum: 1 + description: "Number of days for which the user certificate is valid. Both this property and `renewalDays` must be configured together.The value must be greater than 0 and greater than `renewalDays`.If not configured, the Clients CA configuration is used." required: - type description: "Authentication mechanism enabled for this Kafka user. The supported authentication mechanisms are `scram-sha-512`, `tls`, and `tls-external`. \n\n* `scram-sha-512` generates a secret with SASL SCRAM-SHA-512 credentials.\n* `tls` generates a secret with user certificate for mutual TLS authentication.\n* `tls-external` does not generate a user certificate. But prepares the user for using mutual TLS authentication using a user certificate generated outside the User Operator.\n ACLs and quotas set for this user are configured in the `CN=` format.\n\nAuthentication is optional. If authentication is not configured, no credentials are generated. ACLs and quotas set for the user are configured in the `` format suitable for SASL authentication." + x-kubernetes-validations: + - rule: self.type == 'tls' || (!has(self.validityDays) && !has(self.renewalDays)) + message: "'validityDays' and 'renewalDays' can be configured only with 'type: tls'" + - rule: self.type != 'tls' || (has(self.renewalDays) == has(self.validityDays)) + message: "Both 'validityDays' and 'renewalDays' must be set together, or both must be unset." + - rule: self.type != 'tls' || !has(self.renewalDays) || !has(self.validityDays) || self.renewalDays < self.validityDays + message: '''renewalDays'' must be less than ''validityDays''.' authorization: type: object properties: @@ -104,7 +119,7 @@ spec: enum: - allow - deny - description: The type of the rule. Currently the only supported type is `allow`. ACL rules with type `allow` are used to allow user to execute the specified operations. Default value is `allow`. + description: The type of the rule. ACL rules with type `allow` are used to allow user to execute the specified operations. ACL rules with type `deny` are used to deny user to execute the specified operations. Default value is `allow`. resource: type: object properties: @@ -131,21 +146,6 @@ spec: host: type: string description: "The host from which the action described in the ACL rule is allowed or denied. If not set, it defaults to `*`, allowing or denying the action from any host." - operation: - type: string - enum: - - Read - - Write - - Create - - Delete - - Alter - - Describe - - ClusterAction - - AlterConfigs - - DescribeConfigs - - IdempotentWrite - - All - description: "Operation which will be allowed or denied. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All." operations: type: array items: @@ -165,6 +165,7 @@ spec: description: "List of operations to allow or deny. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All. Only certain operations work with the specified resource." required: - resource + - operations description: List of ACL rules which should be applied to this user. type: type: string @@ -194,469 +195,7 @@ spec: type: number minimum: 0 description: "A quota on the rate at which mutations are accepted for the create topics request, the create partitions request and the delete topics request. The rate is accumulated by the number of partitions created or deleted." - description: Quotas on requests to control the broker resources used by clients. Network bandwidth and request rate quotas can be enforced.Kafka documentation for Kafka User quotas can be found at http://kafka.apache.org/documentation/#design_quotas. - template: - type: object - properties: - secret: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for KafkaUser resources. The template allows users to specify how the `Secret` with password or TLS certificates is generated. - description: Template to specify how Kafka User `Secrets` are generated. - description: The specification of the user. - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - type: - type: string - description: "The unique identifier of a condition, used to distinguish between other conditions in the resource." - status: - type: string - description: "The status of the condition, either True, False or Unknown." - lastTransitionTime: - type: string - description: "Last time the condition of a type changed from one status to another. The required format is 'yyyy-MM-ddTHH:mm:ssZ', in the UTC time zone." - reason: - type: string - description: The reason for the condition's last transition (a single word in CamelCase). - message: - type: string - description: Human-readable message indicating details about the condition's last transition. - description: List of status conditions. - observedGeneration: - type: integer - description: The generation of the CRD that was last reconciled by the operator. - username: - type: string - description: Username. - secret: - type: string - description: The name of `Secret` where the credentials are stored. - description: The status of the Kafka User. - - name: v1beta1 - served: true - storage: false - subresources: - status: {} - additionalPrinterColumns: - - name: Cluster - description: The name of the Kafka cluster this user belongs to - jsonPath: .metadata.labels.strimzi\.io/cluster - type: string - - name: Authentication - description: How the user is authenticated - jsonPath: .spec.authentication.type - type: string - - name: Authorization - description: How the user is authorised - jsonPath: .spec.authorization.type - type: string - - name: Ready - description: The state of the custom resource - jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" - type: string - schema: - openAPIV3Schema: - type: object - properties: - apiVersion: - type: string - description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources" - kind: - type: string - description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds" - metadata: - type: object - spec: - type: object - properties: - authentication: - type: object - properties: - password: - type: object - properties: - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Selects a key of a Secret in the resource's namespace. - description: Secret from which the password should be read. - required: - - valueFrom - description: "Specify the password for the user. If not set, a new password is generated by the User Operator." - type: - type: string - enum: - - tls - - tls-external - - scram-sha-512 - description: Authentication type. - required: - - type - description: "Authentication mechanism enabled for this Kafka user. The supported authentication mechanisms are `scram-sha-512`, `tls`, and `tls-external`. \n\n* `scram-sha-512` generates a secret with SASL SCRAM-SHA-512 credentials.\n* `tls` generates a secret with user certificate for mutual TLS authentication.\n* `tls-external` does not generate a user certificate. But prepares the user for using mutual TLS authentication using a user certificate generated outside the User Operator.\n ACLs and quotas set for this user are configured in the `CN=` format.\n\nAuthentication is optional. If authentication is not configured, no credentials are generated. ACLs and quotas set for the user are configured in the `` format suitable for SASL authentication." - authorization: - type: object - properties: - acls: - type: array - items: - type: object - properties: - type: - type: string - enum: - - allow - - deny - description: The type of the rule. Currently the only supported type is `allow`. ACL rules with type `allow` are used to allow user to execute the specified operations. Default value is `allow`. - resource: - type: object - properties: - name: - type: string - description: Name of resource for which given ACL rule applies. Can be combined with `patternType` field to use prefix pattern. - patternType: - type: string - enum: - - literal - - prefix - description: "Describes the pattern used in the resource field. The supported types are `literal` and `prefix`. With `literal` pattern type, the resource field will be used as a definition of a full name. With `prefix` pattern type, the resource name will be used only as a prefix. Default value is `literal`." - type: - type: string - enum: - - topic - - group - - cluster - - transactionalId - description: "Resource type. The available resource types are `topic`, `group`, `cluster`, and `transactionalId`." - required: - - type - description: Indicates the resource for which given ACL rule applies. - host: - type: string - description: "The host from which the action described in the ACL rule is allowed or denied. If not set, it defaults to `*`, allowing or denying the action from any host." - operation: - type: string - enum: - - Read - - Write - - Create - - Delete - - Alter - - Describe - - ClusterAction - - AlterConfigs - - DescribeConfigs - - IdempotentWrite - - All - description: "Operation which will be allowed or denied. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All." - operations: - type: array - items: - type: string - enum: - - Read - - Write - - Create - - Delete - - Alter - - Describe - - ClusterAction - - AlterConfigs - - DescribeConfigs - - IdempotentWrite - - All - description: "List of operations to allow or deny. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All. Only certain operations work with the specified resource." - required: - - resource - description: List of ACL rules which should be applied to this user. - type: - type: string - enum: - - simple - description: Authorization type. Currently the only supported type is `simple`. `simple` authorization type uses the Kafka Admin API for managing the ACL rules. - required: - - acls - - type - description: Authorization rules for this Kafka user. - quotas: - type: object - properties: - producerByteRate: - type: integer - minimum: 0 - description: A quota on the maximum bytes per-second that each client group can publish to a broker before the clients in the group are throttled. Defined on a per-broker basis. - consumerByteRate: - type: integer - minimum: 0 - description: A quota on the maximum bytes per-second that each client group can fetch from a broker before the clients in the group are throttled. Defined on a per-broker basis. - requestPercentage: - type: integer - minimum: 0 - description: A quota on the maximum CPU utilization of each client group as a percentage of network and I/O threads. - controllerMutationRate: - type: number - minimum: 0 - description: "A quota on the rate at which mutations are accepted for the create topics request, the create partitions request and the delete topics request. The rate is accumulated by the number of partitions created or deleted." - description: Quotas on requests to control the broker resources used by clients. Network bandwidth and request rate quotas can be enforced.Kafka documentation for Kafka User quotas can be found at http://kafka.apache.org/documentation/#design_quotas. - template: - type: object - properties: - secret: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for KafkaUser resources. The template allows users to specify how the `Secret` with password or TLS certificates is generated. - description: Template to specify how Kafka User `Secrets` are generated. - description: The specification of the user. - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - type: - type: string - description: "The unique identifier of a condition, used to distinguish between other conditions in the resource." - status: - type: string - description: "The status of the condition, either True, False or Unknown." - lastTransitionTime: - type: string - description: "Last time the condition of a type changed from one status to another. The required format is 'yyyy-MM-ddTHH:mm:ssZ', in the UTC time zone." - reason: - type: string - description: The reason for the condition's last transition (a single word in CamelCase). - message: - type: string - description: Human-readable message indicating details about the condition's last transition. - description: List of status conditions. - observedGeneration: - type: integer - description: The generation of the CRD that was last reconciled by the operator. - username: - type: string - description: Username. - secret: - type: string - description: The name of `Secret` where the credentials are stored. - description: The status of the Kafka User. - - name: v1alpha1 - served: true - storage: false - subresources: - status: {} - additionalPrinterColumns: - - name: Cluster - description: The name of the Kafka cluster this user belongs to - jsonPath: .metadata.labels.strimzi\.io/cluster - type: string - - name: Authentication - description: How the user is authenticated - jsonPath: .spec.authentication.type - type: string - - name: Authorization - description: How the user is authorised - jsonPath: .spec.authorization.type - type: string - - name: Ready - description: The state of the custom resource - jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" - type: string - schema: - openAPIV3Schema: - type: object - properties: - apiVersion: - type: string - description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources" - kind: - type: string - description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds" - metadata: - type: object - spec: - type: object - properties: - authentication: - type: object - properties: - password: - type: object - properties: - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Selects a key of a Secret in the resource's namespace. - description: Secret from which the password should be read. - required: - - valueFrom - description: "Specify the password for the user. If not set, a new password is generated by the User Operator." - type: - type: string - enum: - - tls - - tls-external - - scram-sha-512 - description: Authentication type. - required: - - type - description: "Authentication mechanism enabled for this Kafka user. The supported authentication mechanisms are `scram-sha-512`, `tls`, and `tls-external`. \n\n* `scram-sha-512` generates a secret with SASL SCRAM-SHA-512 credentials.\n* `tls` generates a secret with user certificate for mutual TLS authentication.\n* `tls-external` does not generate a user certificate. But prepares the user for using mutual TLS authentication using a user certificate generated outside the User Operator.\n ACLs and quotas set for this user are configured in the `CN=` format.\n\nAuthentication is optional. If authentication is not configured, no credentials are generated. ACLs and quotas set for the user are configured in the `` format suitable for SASL authentication." - authorization: - type: object - properties: - acls: - type: array - items: - type: object - properties: - type: - type: string - enum: - - allow - - deny - description: The type of the rule. Currently the only supported type is `allow`. ACL rules with type `allow` are used to allow user to execute the specified operations. Default value is `allow`. - resource: - type: object - properties: - name: - type: string - description: Name of resource for which given ACL rule applies. Can be combined with `patternType` field to use prefix pattern. - patternType: - type: string - enum: - - literal - - prefix - description: "Describes the pattern used in the resource field. The supported types are `literal` and `prefix`. With `literal` pattern type, the resource field will be used as a definition of a full name. With `prefix` pattern type, the resource name will be used only as a prefix. Default value is `literal`." - type: - type: string - enum: - - topic - - group - - cluster - - transactionalId - description: "Resource type. The available resource types are `topic`, `group`, `cluster`, and `transactionalId`." - required: - - type - description: Indicates the resource for which given ACL rule applies. - host: - type: string - description: "The host from which the action described in the ACL rule is allowed or denied. If not set, it defaults to `*`, allowing or denying the action from any host." - operation: - type: string - enum: - - Read - - Write - - Create - - Delete - - Alter - - Describe - - ClusterAction - - AlterConfigs - - DescribeConfigs - - IdempotentWrite - - All - description: "Operation which will be allowed or denied. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All." - operations: - type: array - items: - type: string - enum: - - Read - - Write - - Create - - Delete - - Alter - - Describe - - ClusterAction - - AlterConfigs - - DescribeConfigs - - IdempotentWrite - - All - description: "List of operations to allow or deny. Supported operations are: Read, Write, Create, Delete, Alter, Describe, ClusterAction, AlterConfigs, DescribeConfigs, IdempotentWrite and All. Only certain operations work with the specified resource." - required: - - resource - description: List of ACL rules which should be applied to this user. - type: - type: string - enum: - - simple - description: Authorization type. Currently the only supported type is `simple`. `simple` authorization type uses the Kafka Admin API for managing the ACL rules. - required: - - acls - - type - description: Authorization rules for this Kafka user. - quotas: - type: object - properties: - producerByteRate: - type: integer - minimum: 0 - description: A quota on the maximum bytes per-second that each client group can publish to a broker before the clients in the group are throttled. Defined on a per-broker basis. - consumerByteRate: - type: integer - minimum: 0 - description: A quota on the maximum bytes per-second that each client group can fetch from a broker before the clients in the group are throttled. Defined on a per-broker basis. - requestPercentage: - type: integer - minimum: 0 - description: A quota on the maximum CPU utilization of each client group as a percentage of network and I/O threads. - controllerMutationRate: - type: number - minimum: 0 - description: "A quota on the rate at which mutations are accepted for the create topics request, the create partitions request and the delete topics request. The rate is accumulated by the number of partitions created or deleted." - description: Quotas on requests to control the broker resources used by clients. Network bandwidth and request rate quotas can be enforced.Kafka documentation for Kafka User quotas can be found at http://kafka.apache.org/documentation/#design_quotas. + description: "Quotas on requests to control the broker resources used by clients. Network bandwidth and request rate quotas can be enforced. For more information, see the Apache Kafka design documentation about quotas." template: type: object properties: @@ -714,3 +253,5 @@ spec: type: string description: The name of `Secret` where the credentials are stored. description: The status of the Kafka User. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkamirrormaker.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkamirrormaker.yaml deleted file mode 100644 index c8248d7cc..000000000 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkamirrormaker.yaml +++ /dev/null @@ -1,1584 +0,0 @@ -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - name: kafkamirrormakers.kafka.strimzi.io - labels: - app: strimzi - strimzi.io/crd-install: "true" - component: kafkamirrormakers.kafka.strimzi.io-crd -spec: - group: kafka.strimzi.io - names: - kind: KafkaMirrorMaker - listKind: KafkaMirrorMakerList - singular: kafkamirrormaker - plural: kafkamirrormakers - shortNames: - - kmm - categories: - - strimzi - scope: Namespaced - conversion: - strategy: None - versions: - - name: v1beta2 - served: true - storage: true - subresources: - status: {} - scale: - specReplicasPath: .spec.replicas - statusReplicasPath: .status.replicas - labelSelectorPath: .status.labelSelector - additionalPrinterColumns: - - name: Desired replicas - description: The desired number of Kafka MirrorMaker replicas - jsonPath: .spec.replicas - type: integer - - name: Consumer Bootstrap Servers - description: The boostrap servers for the consumer - jsonPath: .spec.consumer.bootstrapServers - type: string - priority: 1 - - name: Producer Bootstrap Servers - description: The boostrap servers for the producer - jsonPath: .spec.producer.bootstrapServers - type: string - priority: 1 - - name: Ready - description: The state of the custom resource - jsonPath: ".status.conditions[?(@.type==\"Ready\")].status" - type: string - schema: - openAPIV3Schema: - type: object - properties: - apiVersion: - type: string - description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources" - kind: - type: string - description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds" - metadata: - type: object - spec: - type: object - properties: - version: - type: string - description: The Kafka MirrorMaker version. Defaults to the latest version. Consult the documentation to understand the process required to upgrade or downgrade the version. - replicas: - type: integer - minimum: 0 - description: The number of pods in the `Deployment`. - image: - type: string - description: "The container image used for Kafka MirrorMaker pods. If no image name is explicitly specified, it is determined based on the `spec.version` configuration. The image names are specifically mapped to corresponding versions in the Cluster Operator configuration." - consumer: - type: object - properties: - numStreams: - type: integer - minimum: 1 - description: Specifies the number of consumer stream threads to create. - offsetCommitInterval: - type: integer - description: Specifies the offset auto-commit interval in ms. Default value is 60000. - bootstrapServers: - type: string - description: A list of host:port pairs for establishing the initial connection to the Kafka cluster. - groupId: - type: string - description: A unique string that identifies the consumer group this consumer belongs to. - authentication: - type: object - properties: - accessToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the access token which was obtained from the authorization server. - accessTokenIsJwt: - type: boolean - description: Configure whether access token should be treated as JWT. This should be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - accessTokenLocation: - type: string - description: Path to the token file containing an access token to be used for authentication. - audience: - type: string - description: "OAuth audience to use when authenticating against the authorization server. Some authorization servers require the audience to be explicitly set. The possible values depend on how the authorization server is configured. By default, `audience` is not specified when performing the token endpoint request." - certificateAndKey: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the Secret. - key: - type: string - description: The name of the private key in the Secret. - required: - - secretName - - certificate - - key - description: Reference to the `Secret` which holds the certificate and private key pair. - clientAssertion: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes secret containing the client assertion which was manually configured for the client. - clientAssertionLocation: - type: string - description: Path to the file containing the client assertion to be used for authentication. - clientAssertionType: - type: string - description: "The client assertion type. If not set, and either `clientAssertion` or `clientAssertionLocation` is configured, this value defaults to `urn:ietf:params:oauth:client-assertion-type:jwt-bearer`." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - maxTokenExpirySeconds: - type: integer - description: Set or limit time-to-live of the access tokens to the specified number of seconds. This should be set if the authorization server returns opaque tokens. - passwordSecret: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the password. - password: - type: string - description: The name of the key in the Secret under which the password is stored. - required: - - secretName - - password - description: Reference to the `Secret` which holds the password. - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." - refreshToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the refresh token which can be used to obtain access token from the authorization server. - saslExtensions: - additionalProperties: - type: string - type: object - description: SASL extensions parameters. - scope: - type: string - description: OAuth scope to use when authenticating against the authorization server. Some authorization servers require this to be set. The possible values depend on how authorization server is configured. By default `scope` is not specified when doing the token endpoint request. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. - type: - type: string - enum: - - tls - - scram-sha-256 - - scram-sha-512 - - plain - - oauth - description: "Authentication type. Currently the supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and 'oauth'. `scram-sha-256` and `scram-sha-512` types use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 Authentication, respectively. `plain` type uses SASL PLAIN Authentication. `oauth` type uses SASL OAUTHBEARER Authentication. The `tls` type uses TLS Client Authentication. The `tls` type is supported only over TLS connections." - username: - type: string - description: Username used for the authentication. - required: - - type - description: Authentication configuration for connecting to the cluster. - tls: - type: object - properties: - trustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection. - description: TLS configuration for connecting MirrorMaker to the cluster. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: "The MirrorMaker consumer config. Properties with the following prefixes cannot be set: ssl., bootstrap.servers, group.id, sasl., security., interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." - required: - - bootstrapServers - - groupId - description: Configuration of source cluster. - producer: - type: object - properties: - bootstrapServers: - type: string - description: A list of host:port pairs for establishing the initial connection to the Kafka cluster. - abortOnSendFailure: - type: boolean - description: Flag to set the MirrorMaker to exit on a failed send. Default value is `true`. - authentication: - type: object - properties: - accessToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the access token which was obtained from the authorization server. - accessTokenIsJwt: - type: boolean - description: Configure whether access token should be treated as JWT. This should be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - accessTokenLocation: - type: string - description: Path to the token file containing an access token to be used for authentication. - audience: - type: string - description: "OAuth audience to use when authenticating against the authorization server. Some authorization servers require the audience to be explicitly set. The possible values depend on how the authorization server is configured. By default, `audience` is not specified when performing the token endpoint request." - certificateAndKey: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the Secret. - key: - type: string - description: The name of the private key in the Secret. - required: - - secretName - - certificate - - key - description: Reference to the `Secret` which holds the certificate and private key pair. - clientAssertion: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes secret containing the client assertion which was manually configured for the client. - clientAssertionLocation: - type: string - description: Path to the file containing the client assertion to be used for authentication. - clientAssertionType: - type: string - description: "The client assertion type. If not set, and either `clientAssertion` or `clientAssertionLocation` is configured, this value defaults to `urn:ietf:params:oauth:client-assertion-type:jwt-bearer`." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - maxTokenExpirySeconds: - type: integer - description: Set or limit time-to-live of the access tokens to the specified number of seconds. This should be set if the authorization server returns opaque tokens. - passwordSecret: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the password. - password: - type: string - description: The name of the key in the Secret under which the password is stored. - required: - - secretName - - password - description: Reference to the `Secret` which holds the password. - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." - refreshToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the refresh token which can be used to obtain access token from the authorization server. - saslExtensions: - additionalProperties: - type: string - type: object - description: SASL extensions parameters. - scope: - type: string - description: OAuth scope to use when authenticating against the authorization server. Some authorization servers require this to be set. The possible values depend on how authorization server is configured. By default `scope` is not specified when doing the token endpoint request. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. - type: - type: string - enum: - - tls - - scram-sha-256 - - scram-sha-512 - - plain - - oauth - description: "Authentication type. Currently the supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and 'oauth'. `scram-sha-256` and `scram-sha-512` types use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 Authentication, respectively. `plain` type uses SASL PLAIN Authentication. `oauth` type uses SASL OAUTHBEARER Authentication. The `tls` type uses TLS Client Authentication. The `tls` type is supported only over TLS connections." - username: - type: string - description: Username used for the authentication. - required: - - type - description: Authentication configuration for connecting to the cluster. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: "The MirrorMaker producer config. Properties with the following prefixes cannot be set: ssl., bootstrap.servers, sasl., security., interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." - tls: - type: object - properties: - trustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection. - description: TLS configuration for connecting MirrorMaker to the cluster. - required: - - bootstrapServers - description: Configuration of target cluster. - resources: - type: object - properties: - claims: - type: array - items: - type: object - properties: - name: - type: string - request: - type: string - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: "^(\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\\+|-)?(([0-9]+(\\.[0-9]*)?)|(\\.[0-9]+))))?$" - x-kubernetes-int-or-string: true - type: object - description: CPU and memory resources to reserve. - whitelist: - type: string - description: "List of topics which are included for mirroring. This option allows any regular expression using Java-style regular expressions. Mirroring two topics named A and B is achieved by using the expression `A\\|B`. Or, as a special case, you can mirror all topics using the regular expression `*`. You can also specify multiple regular expressions separated by commas." - include: - type: string - description: "List of topics which are included for mirroring. This option allows any regular expression using Java-style regular expressions. Mirroring two topics named A and B is achieved by using the expression `A\\|B`. Or, as a special case, you can mirror all topics using the regular expression `*`. You can also specify multiple regular expressions separated by commas." - jvmOptions: - type: object - properties: - "-XX": - additionalProperties: - type: string - type: object - description: A map of -XX options to the JVM. - "-Xmx": - type: string - pattern: "^[0-9]+[mMgG]?$" - description: -Xmx option to to the JVM. - "-Xms": - type: string - pattern: "^[0-9]+[mMgG]?$" - description: -Xms option to to the JVM. - gcLoggingEnabled: - type: boolean - description: Specifies whether the Garbage Collection logging is enabled. The default is false. - javaSystemProperties: - type: array - items: - type: object - properties: - name: - type: string - description: The system property name. - value: - type: string - description: The system property value. - description: A map of additional system properties which will be passed using the `-D` option to the JVM. - description: JVM Options for pods. - logging: - type: object - properties: - loggers: - additionalProperties: - type: string - type: object - description: A Map from logger name to logger level. - type: - type: string - enum: - - inline - - external - description: "Logging type, must be either 'inline' or 'external'." - valueFrom: - type: object - properties: - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to the key in the ConfigMap containing the configuration. - description: '`ConfigMap` entry where the logging configuration is stored. ' - required: - - type - description: Logging configuration for MirrorMaker. - metricsConfig: - type: object - properties: - type: - type: string - enum: - - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. - valueFrom: - type: object - properties: - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' - required: - - type - - valueFrom - description: Metrics configuration. - tracing: - type: object - properties: - type: - type: string - enum: - - jaeger - - opentelemetry - description: "Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. As of Strimzi 0.37.0, `jaeger` type is not supported anymore and this option is ignored." - required: - - type - description: The configuration of tracing in Kafka MirrorMaker. - template: - type: object - properties: - deployment: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - deploymentStrategy: - type: string - enum: - - RollingUpdate - - Recreate - description: Pod replacement strategy for deployment configuration changes. Valid values are `RollingUpdate` and `Recreate`. Defaults to `RollingUpdate`. - description: Template for Kafka MirrorMaker `Deployment`. - pod: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - imagePullSecrets: - type: array - items: - type: object - properties: - name: - type: string - description: "List of references to secrets in the same namespace to use for pulling any of the images used by this Pod. When the `STRIMZI_IMAGE_PULL_SECRETS` environment variable in Cluster Operator and the `imagePullSecrets` option are specified, only the `imagePullSecrets` variable is used and the `STRIMZI_IMAGE_PULL_SECRETS` variable is ignored." - securityContext: - type: object - properties: - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - fsGroup: - type: integer - fsGroupChangePolicy: - type: string - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxChangePolicy: - type: string - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - supplementalGroups: - type: array - items: - type: integer - supplementalGroupsPolicy: - type: string - sysctls: - type: array - items: - type: object - properties: - name: - type: string - value: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Configures pod-level security attributes and common container settings. - terminationGracePeriodSeconds: - type: integer - minimum: 0 - description: "The grace period is the duration in seconds after the processes running in the pod are sent a termination signal, and the time when the processes are forcibly halted with a kill signal. Set this value to longer than the expected cleanup time for your process. Value must be a non-negative integer. A zero value indicates delete immediately. You might need to increase the grace period for very large Kafka clusters, so that the Kafka brokers have enough time to transfer their work to another broker before they are terminated. Defaults to 30 seconds." - affinity: - type: object - properties: - nodeAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - preference: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: object - properties: - nodeSelectorTerms: - type: array - items: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchFields: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - podAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - podAntiAffinity: - type: object - properties: - preferredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - podAffinityTerm: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - weight: - type: integer - requiredDuringSchedulingIgnoredDuringExecution: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - mismatchLabelKeys: - type: array - items: - type: string - namespaceSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - namespaces: - type: array - items: - type: string - topologyKey: - type: string - description: The pod's affinity rules. - tolerations: - type: array - items: - type: object - properties: - effect: - type: string - key: - type: string - operator: - type: string - tolerationSeconds: - type: integer - value: - type: string - description: The pod's tolerations. - topologySpreadConstraints: - type: array - items: - type: object - properties: - labelSelector: - type: object - properties: - matchExpressions: - type: array - items: - type: object - properties: - key: - type: string - operator: - type: string - values: - type: array - items: - type: string - matchLabels: - additionalProperties: - type: string - type: object - matchLabelKeys: - type: array - items: - type: string - maxSkew: - type: integer - minDomains: - type: integer - nodeAffinityPolicy: - type: string - nodeTaintsPolicy: - type: string - topologyKey: - type: string - whenUnsatisfiable: - type: string - description: The pod's topology spread constraints. - priorityClassName: - type: string - description: 'The name of the priority class used to assign priority to the pods. ' - schedulerName: - type: string - description: "The name of the scheduler used to dispatch this `Pod`. If not specified, the default scheduler will be used." - hostAliases: - type: array - items: - type: object - properties: - hostnames: - type: array - items: - type: string - ip: - type: string - description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. - enableServiceLinks: - type: boolean - description: Indicates whether information about services should be injected into Pod's environment variables. - tmpDirSizeLimit: - type: string - pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" - description: "Defines the total amount of pod memory allocated for the temporary `EmptyDir` volume `/tmp`. Specify the allocation in memory units, for example, `100Mi` for 100 mebibytes. Default value is `5Mi`. The `/tmp` volume is backed by pod memory, not disk storage, so avoid setting a high value as it consumes pod memory resources." - volumes: - type: array - items: - type: object - properties: - name: - type: string - description: Name to use for the volume. Required. - secret: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - optional: - type: boolean - secretName: - type: string - description: Secret to use populate the volume. - configMap: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - name: - type: string - optional: - type: boolean - description: ConfigMap to use to populate the volume. - emptyDir: - type: object - properties: - medium: - type: string - sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. - persistentVolumeClaim: - type: object - properties: - claimName: - type: string - readOnly: - type: boolean - description: PersistentVolumeClaim object to use to populate the volume. - csi: - type: object - properties: - driver: - type: string - fsType: - type: string - nodePublishSecretRef: - type: object - properties: - name: - type: string - readOnly: - type: boolean - volumeAttributes: - additionalProperties: - type: string - type: object - description: CSIVolumeSource object to use to populate the volume. - oneOf: - - properties: - secret: {} - configMap: {} - emptyDir: {} - persistentVolumeClaim: {} - csi: {} - description: Additional volumes that can be mounted to the pod. - description: Template for Kafka MirrorMaker `Pods`. - podDisruptionBudget: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata to apply to the `PodDisruptionBudgetTemplate` resource. - maxUnavailable: - type: integer - minimum: 0 - description: "Maximum number of unavailable pods to allow automatic Pod eviction. A Pod eviction is allowed when the `maxUnavailable` number of pods or fewer are unavailable after the eviction. Setting this value to 0 prevents all voluntary evictions, so the pods must be evicted manually. Defaults to 1." - description: Template for Kafka MirrorMaker `PodDisruptionBudget`. - mirrorMakerContainer: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: The environment variable key. - value: - type: string - description: The environment variable value. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a config map. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Reference to the secret or config map property to which the environment variable is set. - oneOf: - - properties: - value: {} - required: - - value - - properties: - valueFrom: {} - required: - - valueFrom - description: Environment variables which should be applied to the container. - securityContext: - type: object - properties: - allowPrivilegeEscalation: - type: boolean - appArmorProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - capabilities: - type: object - properties: - add: - type: array - items: - type: string - drop: - type: array - items: - type: string - privileged: - type: boolean - procMount: - type: string - readOnlyRootFilesystem: - type: boolean - runAsGroup: - type: integer - runAsNonRoot: - type: boolean - runAsUser: - type: integer - seLinuxOptions: - type: object - properties: - level: - type: string - role: - type: string - type: - type: string - user: - type: string - seccompProfile: - type: object - properties: - localhostProfile: - type: string - type: - type: string - windowsOptions: - type: object - properties: - gmsaCredentialSpec: - type: string - gmsaCredentialSpecName: - type: string - hostProcess: - type: boolean - runAsUserName: - type: string - description: Security context for the container. - volumeMounts: - type: array - items: - type: object - properties: - mountPath: - type: string - mountPropagation: - type: string - name: - type: string - readOnly: - type: boolean - recursiveReadOnly: - type: string - subPath: - type: string - subPathExpr: - type: string - description: Additional volume mounts which should be applied to the container. - description: Template for Kafka MirrorMaker container. - serviceAccount: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - description: Template for the Kafka MirrorMaker service account. - description: "Template to specify how Kafka MirrorMaker resources, `Deployments` and `Pods`, are generated." - livenessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod liveness checking. - readinessProbe: - type: object - properties: - initialDelaySeconds: - type: integer - minimum: 0 - description: The initial delay before first the health is first checked. Default to 15 seconds. Minimum value is 0. - timeoutSeconds: - type: integer - minimum: 1 - description: The timeout for each attempted health check. Default to 5 seconds. Minimum value is 1. - periodSeconds: - type: integer - minimum: 1 - description: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1. - successThreshold: - type: integer - minimum: 1 - description: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1. - failureThreshold: - type: integer - minimum: 1 - description: Minimum consecutive failures for the probe to be considered failed after having succeeded. Defaults to 3. Minimum value is 1. - description: Pod readiness checking. - oneOf: - - properties: - include: {} - required: - - include - - properties: - whitelist: {} - required: - - whitelist - required: - - replicas - - consumer - - producer - description: The specification of Kafka MirrorMaker. - status: - type: object - properties: - conditions: - type: array - items: - type: object - properties: - type: - type: string - description: "The unique identifier of a condition, used to distinguish between other conditions in the resource." - status: - type: string - description: "The status of the condition, either True, False or Unknown." - lastTransitionTime: - type: string - description: "Last time the condition of a type changed from one status to another. The required format is 'yyyy-MM-ddTHH:mm:ssZ', in the UTC time zone." - reason: - type: string - description: The reason for the condition's last transition (a single word in CamelCase). - message: - type: string - description: Human-readable message indicating details about the condition's last transition. - description: List of status conditions. - observedGeneration: - type: integer - description: The generation of the CRD that was last reconciled by the operator. - labelSelector: - type: string - description: Label selector for pods providing this resource. - replicas: - type: integer - description: The current number of pods being used to provide this resource. - description: The status of Kafka MirrorMaker. diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/04A-Crd-kafkanodepool.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkanodepool.yaml similarity index 88% rename from packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/04A-Crd-kafkanodepool.yaml rename to packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkanodepool.yaml index 445411c27..25e729af2 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/04A-Crd-kafkanodepool.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/045-Crd-kafkanodepool.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -70,7 +70,7 @@ spec: description: The storage class to use for dynamic volume allocation. deleteClaim: type: boolean - description: Specifies if the persistent volume claim has to be deleted when the cluster is un-deployed. + description: Specifies whether the persistent volume claim is deleted when a Kafka node is deleted. Optional. Defaults to `false`. id: type: integer minimum: 0 @@ -80,18 +80,6 @@ spec: enum: - shared description: "Specifies whether this volume should be used for storing KRaft metadata. This property is optional. When set, the only currently supported value is `shared`. At most one volume can have this property set." - overrides: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation for this broker. - broker: - type: integer - description: Id of the kafka broker (broker identifier). - description: Overrides for individual brokers. The `overrides` field allows you to specify a different configuration for different brokers. selector: additionalProperties: type: string @@ -111,6 +99,9 @@ spec: - persistent-claim - jbod description: "Storage type, must be either 'ephemeral', 'persistent-claim', or 'jbod'." + volumeAttributesClass: + type: string + description: Specifies `VolumeAttributeClass` name for dynamically configuring storage attributes. volumes: type: array items: @@ -121,7 +112,7 @@ spec: description: The storage class to use for dynamic volume allocation. deleteClaim: type: boolean - description: Specifies if the persistent volume claim has to be deleted when the cluster is un-deployed. + description: Specifies whether the persistent volume claim is deleted when a Kafka node is deleted. Optional. Defaults to `false`. id: type: integer minimum: 0 @@ -131,18 +122,6 @@ spec: enum: - shared description: "Specifies whether this volume should be used for storing KRaft metadata. This property is optional. When set, the only currently supported value is `shared`. At most one volume can have this property set." - overrides: - type: array - items: - type: object - properties: - class: - type: string - description: The storage class to use for dynamic volume allocation for this broker. - broker: - type: integer - description: Id of the kafka broker (broker identifier). - description: Overrides for individual brokers. The `overrides` field allows you to specify a different configuration for different brokers. selector: additionalProperties: type: string @@ -161,6 +140,9 @@ spec: - ephemeral - persistent-claim description: "Storage type, must be either 'ephemeral' or 'persistent-claim'." + volumeAttributesClass: + type: string + description: Specifies `VolumeAttributeClass` name for dynamically configuring storage attributes. required: - type description: List of volumes as Storage objects representing the JBOD disks array. @@ -174,7 +156,7 @@ spec: enum: - controller - broker - description: "The roles that the nodes in this pool will have when KRaft mode is enabled. Supported values are 'broker' and 'controller'. This field is required. When KRaft mode is disabled, the only allowed value if `broker`." + description: The roles assigned to the node pool. Supported values are `broker` and `controller`. This property is required. resources: type: object properties: @@ -758,6 +740,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -793,7 +804,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -814,20 +825,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -835,7 +846,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -854,7 +865,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -862,7 +881,75 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + templatedVolumes: + type: array + items: + type: object + properties: + name: + type: string + description: Name to use for the volume. Required. + secret: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + optional: + type: boolean + secretName: + type: string + description: "`Secret` to use to populate the volume. The name of the Secret and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + configMap: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + name: + type: string + optional: + type: boolean + description: "`ConfigMap` to use to populate the volume. The name of the ConfigMap and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + persistentVolumeClaim: + type: object + properties: + claimName: + type: string + readOnly: + type: boolean + description: "`PersistentVolumeClaim` object to use to populate the volume. The name of the Persistent Volume Claim can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + oneOf: + - properties: + secret: {} + configMap: {} + persistentVolumeClaim: {} + description: Additional volumes that can be mounted to the pod. These volumes can use templates to mount different volumes into individual Pods. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka `Pods`. perPodService: type: object @@ -1278,3 +1365,5 @@ spec: type: string description: Label selector for pods providing this resource. description: The status of the KafkaNodePool. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/046-Crd-kafkabridge.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/046-Crd-kafkabridge.yaml index b4c0dfcca..f54a5a0ff 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/046-Crd-kafkabridge.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/046-Crd-kafkabridge.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -62,10 +62,10 @@ spec: replicas: type: integer minimum: 0 - description: The number of pods in the `Deployment`. Defaults to `1`. + description: The number of pods in the `Deployment`. Required in the `v1` version of the Strimzi API. image: type: string - description: "The container image used for Kafka Bridge pods. If no image name is explicitly specified, the image name corresponds to the image specified in the Cluster Operator configuration. If an image name is not defined in the Cluster Operator configuration, a default value is used." + description: "The container image used for HTTP Bridge pods. If no image name is explicitly specified, the image name corresponds to the image specified in the Cluster Operator configuration. If an image name is not defined in the Cluster Operator configuration, a default value is used." bootstrapServers: type: string description: A list of host:port pairs for establishing the initial connection to the Kafka cluster. @@ -98,32 +98,10 @@ spec: required: - secretName description: Trusted certificates for TLS connection. - description: TLS configuration for connecting Kafka Bridge to the cluster. + description: TLS configuration for connecting HTTP Bridge to the cluster. authentication: type: object properties: - accessToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the access token which was obtained from the authorization server. - accessTokenIsJwt: - type: boolean - description: Configure whether access token should be treated as JWT. This should be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - accessTokenLocation: - type: string - description: Path to the token file containing an access token to be used for authentication. - audience: - type: string - description: "OAuth audience to use when authenticating against the authorization server. Some authorization servers require the audience to be explicitly set. The possible values depend on how the authorization server is configured. By default, `audience` is not specified when performing the token endpoint request." certificateAndKey: type: object properties: @@ -135,68 +113,16 @@ spec: description: The name of the file certificate in the Secret. key: type: string - description: The name of the private key in the Secret. + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." required: - secretName - certificate - key description: Reference to the `Secret` which holds the certificate and private key pair. - clientAssertion: + config: + x-kubernetes-preserve-unknown-fields: true type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes secret containing the client assertion which was manually configured for the client. - clientAssertionLocation: - type: string - description: Path to the file containing the client assertion to be used for authentication. - clientAssertionType: - type: string - description: "The client assertion type. If not set, and either `clientAssertion` or `clientAssertionLocation` is configured, this value defaults to `urn:ietf:params:oauth:client-assertion-type:jwt-bearer`." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - maxTokenExpirySeconds: - type: integer - description: Set or limit time-to-live of the access tokens to the specified number of seconds. This should be set if the authorization server returns opaque tokens. + description: Configuration for the custom authentication mechanism. Only properties with the `sasl.` and `ssl.keystore.` prefixes are allowed. Specify other options in the regular configuration section of the custom resource. passwordSecret: type: object properties: @@ -210,59 +136,9 @@ spec: - secretName - password description: Reference to the `Secret` which holds the password. - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." - refreshToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the refresh token which can be used to obtain access token from the authorization server. - saslExtensions: - additionalProperties: - type: string - type: object - description: SASL extensions parameters. - scope: - type: string - description: OAuth scope to use when authenticating against the authorization server. Some authorization servers require this to be set. The possible values depend on how authorization server is configured. By default `scope` is not specified when doing the token endpoint request. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. + sasl: + type: boolean + description: Enable or disable SASL on this authentication mechanism. type: type: string enum: @@ -270,8 +146,8 @@ spec: - scram-sha-256 - scram-sha-512 - plain - - oauth - description: "Authentication type. Currently the supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and 'oauth'. `scram-sha-256` and `scram-sha-512` types use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 Authentication, respectively. `plain` type uses SASL PLAIN Authentication. `oauth` type uses SASL OAUTHBEARER Authentication. The `tls` type uses TLS Client Authentication. The `tls` type is supported only over TLS connections." + - custom + description: "Specifies the authentication type. Supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and `custom`. `tls` uses TLS client authentication and is supported only over TLS connections. `scram-sha-256` and `scram-sha-512` use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 authentication, respectively. `plain` uses SASL PLAIN authentication. `custom` allows you to configure a custom authentication mechanism." username: type: string description: Username used for the authentication. @@ -284,7 +160,34 @@ spec: port: type: integer minimum: 1023 - description: The port which is the server listening on. + description: Port the server listens on. + tls: + type: object + properties: + certificateAndKey: + type: object + properties: + secretName: + type: string + description: The name of the Secret containing the certificate. + certificate: + type: string + description: The name of the file certificate in the Secret. + key: + type: string + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." + required: + - secretName + - certificate + - key + description: Reference to the `Secret` which holds the certificate and private key pair. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: "Additional configuration for the HTTP server TLS. Properties with the following prefixes cannot be set: ssl. (with the exception of: ssl.enabled.cipher.suites, ssl.enabled.protocols)." + required: + - certificateAndKey + description: TLS configuration for clients connections to the HTTP Bridge. cors: type: object properties: @@ -396,7 +299,7 @@ spec: type: string description: The system property value. description: A map of additional system properties which will be passed using the `-D` option to the JVM. - description: '**Currently not supported** JVM Options for pods.' + description: JVM Options for pods. logging: type: object properties: @@ -427,23 +330,70 @@ spec: description: '`ConfigMap` entry where the logging configuration is stored. ' required: - type - description: Logging configuration for Kafka Bridge. + description: Logging configuration for HTTP Bridge. clientRackInitImage: type: string description: The image of the init container used for initializing the `client.rack`. rack: type: object properties: + envVarName: + type: string + description: The name of the environment variable that defines the rack ID. Its value sets the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect or MirrorMaker 2. topologyKey: type: string example: topology.kubernetes.io/zone description: "A key that matches labels assigned to the Kubernetes cluster nodes. The value of the label is used to set a broker's `broker.rack` config, and the `client.rack` config for Kafka Connect or MirrorMaker 2." - required: - - topologyKey + type: + type: string + enum: + - topology-label + - environment-variable + description: "Specifies the rack awareness type. Supported types are `topology-label` and `environment-variable`. `topology-label` uses a Kubernetes worker node label to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. `environment-variable` uses an environment variable to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. When not specified, `topology-label` type is used by default." description: Configuration of the node label which will be used as the client.rack consumer configuration. - enableMetrics: - type: boolean - description: Enable the metrics for the Kafka Bridge. Default is false. + x-kubernetes-validations: + - rule: (has(self.type) && self.type != "topology-label") || self.topologyKey != "" + message: topologyKey property is required + - rule: has(self.type) == false || self.type != "environment-variable" || self.envVarName != "" + message: envVarName property is required + metricsConfig: + type: object + properties: + type: + type: string + enum: + - jmxPrometheusExporter + - strimziMetricsReporter + description: Metrics type. The supported types are `jmxPrometheusExporter` and `strimziMetricsReporter`. Type `jmxPrometheusExporter` uses the Prometheus JMX Exporter to expose Kafka JMX metrics in Prometheus format through an HTTP endpoint. Type `strimziMetricsReporter` uses the Strimzi Metrics Reporter to directly expose Kafka metrics in Prometheus format through an HTTP endpoint. + valueFrom: + type: object + properties: + configMapKeyRef: + type: object + properties: + key: + type: string + name: + type: string + optional: + type: boolean + description: Reference to the key in the ConfigMap containing the configuration. + description: ConfigMap entry where the Prometheus JMX Exporter configuration is stored. + values: + type: object + properties: + allowList: + type: array + items: + type: string + description: A list of regex patterns to filter the metrics to collect. Should contain at least one element. + description: Configuration values for the Strimzi Metrics Reporter. + required: + - type + description: Metrics configuration. + x-kubernetes-validations: + - rule: self.type != 'jmxPrometheusExporter' || has(self.valueFrom) + message: valueFrom property is required livenessProbe: type: object properties: @@ -518,7 +468,7 @@ spec: - RollingUpdate - Recreate description: Pod replacement strategy for deployment configuration changes. Valid values are `RollingUpdate` and `Recreate`. Defaults to `RollingUpdate`. - description: Template for Kafka Bridge `Deployment`. + description: Template for HTTP Bridge `Deployment`. pod: type: object properties: @@ -1020,6 +970,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -1055,7 +1034,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -1076,20 +1055,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -1097,7 +1076,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -1116,7 +1095,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -1124,8 +1111,12 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. - description: Template for Kafka Bridge `Pods`. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." + description: Template for HTTP Bridge `Pods`. apiService: type: object properties: @@ -1158,7 +1149,7 @@ spec: - IPv4 - IPv6 description: "Specifies the IP Families used by the service. Available options are `IPv4` and `IPv6`. If unspecified, Kubernetes will choose the default value based on the `ipFamilyPolicy` setting." - description: Template for Kafka Bridge API `Service`. + description: Template for HTTP Bridge API `Service`. podDisruptionBudget: type: object properties: @@ -1180,7 +1171,7 @@ spec: type: integer minimum: 0 description: "Maximum number of unavailable pods to allow automatic Pod eviction. A Pod eviction is allowed when the `maxUnavailable` number of pods or fewer are unavailable after the eviction. Setting this value to 0 prevents all voluntary evictions, so the pods must be evicted manually. Defaults to 1." - description: Template for Kafka Bridge `PodDisruptionBudget`. + description: Template for HTTP Bridge `PodDisruptionBudget`. bridgeContainer: type: object properties: @@ -1323,7 +1314,7 @@ spec: subPathExpr: type: string description: Additional volume mounts which should be applied to the container. - description: Template for the Kafka Bridge container. + description: Template for the HTTP Bridge container. clusterRoleBinding: type: object properties: @@ -1341,7 +1332,7 @@ spec: type: object description: Annotations added to the Kubernetes resource. description: Metadata applied to the resource. - description: Template for the Kafka Bridge ClusterRoleBinding. + description: Template for the HTTP Bridge ClusterRoleBinding. serviceAccount: type: object properties: @@ -1359,7 +1350,7 @@ spec: type: object description: Annotations added to the Kubernetes resource. description: Metadata applied to the resource. - description: Template for the Kafka Bridge service account. + description: Template for the HTTP Bridge service account. initContainer: type: object properties: @@ -1502,23 +1493,27 @@ spec: subPathExpr: type: string description: Additional volume mounts which should be applied to the container. - description: Template for the Kafka Bridge init container. - description: Template for Kafka Bridge resources. The template allows users to specify how a `Deployment` and `Pod` is generated. + description: Template for the HTTP Bridge init container. + description: Template for HTTP Bridge resources. The template allows users to specify how a `Deployment` and `Pod` is generated. tracing: type: object properties: type: type: string enum: - - jaeger - opentelemetry - description: "Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. As of Strimzi 0.37.0, `jaeger` type is not supported anymore and this option is ignored." + description: Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. required: - type - description: The configuration of tracing in Kafka Bridge. + description: The configuration of tracing in HTTP Bridge. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: "Additional configuration for the HTTP bridge. The following prefixes cannot be set: kafka., http., bridge.metrics. The following options cannot be set: bridge.id, bridge.tracing, bridge.metrics." required: + - replicas - bootstrapServers - description: The specification of the Kafka Bridge. + description: The specification of the HTTP Bridge. status: type: object properties: @@ -1548,11 +1543,13 @@ spec: description: The generation of the CRD that was last reconciled by the operator. url: type: string - description: The URL at which external client applications can access the Kafka Bridge. + description: The URL at which external client applications can access the HTTP Bridge. replicas: type: integer description: The current number of pods being used to provide this resource. labelSelector: type: string description: Label selector for pods providing this resource. - description: The status of the Kafka Bridge. + description: The status of the HTTP Bridge. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/047-Crd-kafkaconnector.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/047-Crd-kafkaconnector.yaml index 25a8b74a9..bd50cf772 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/047-Crd-kafkaconnector.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/047-Crd-kafkaconnector.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -78,13 +78,13 @@ spec: type: integer description: "The maximum number of connector restarts that the operator will try. If the connector remains in a failed state after reaching this limit, it must be restarted manually by the user. Defaults to an unlimited number of restarts." description: Automatic restart of connector and tasks configuration. + version: + type: string + description: Desired version or version range to respect when starting the Kafka Connector. This is only supported when using Kafka Connect version 4.1.0 and higher. config: x-kubernetes-preserve-unknown-fields: true type: object - description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max." - pause: - type: boolean - description: Whether the connector should be paused. Defaults to false. + description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max, connector.plugin.version." state: type: string enum: @@ -170,3 +170,5 @@ spec: type: string description: The list of topics used by the Kafka Connector. description: The status of the Kafka Connector. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/048-Crd-kafkamirrormaker2.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/048-Crd-kafkamirrormaker2.yaml index 0ffe03902..b3c0c57f6 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/048-Crd-kafkamirrormaker2.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/048-Crd-kafkamirrormaker2.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -59,81 +59,38 @@ spec: description: The Kafka Connect version. Defaults to the latest version. Consult the user documentation to understand the process required to upgrade or downgrade the version. replicas: type: integer - description: The number of pods in the Kafka Connect group. Defaults to `3`. + description: The number of pods in the Kafka Connect group. Required in the `v1` version of the Strimzi API. image: type: string description: "The container image used for Kafka Connect pods. If no image name is explicitly specified, it is determined based on the `spec.version` configuration. The image names are specifically mapped to corresponding versions in the Cluster Operator configuration." - connectCluster: - type: string - description: The cluster alias used for Kafka Connect. The value must match the alias of the *target* Kafka cluster as specified in the `spec.clusters` configuration. The target Kafka cluster is used by the underlying Kafka Connect framework for its internal topics. - clusters: - type: array - items: - type: object - properties: - alias: - type: string - pattern: "^[a-zA-Z0-9\\._\\-]{1,100}$" - description: Alias used to reference the Kafka cluster. - bootstrapServers: - type: string - description: A comma-separated list of `host:port` pairs for establishing the connection to the Kafka cluster. - tls: - type: object - properties: - trustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection. - description: TLS configuration for connecting MirrorMaker 2 connectors to a cluster. - authentication: - type: object - properties: - accessToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the access token which was obtained from the authorization server. - accessTokenIsJwt: - type: boolean - description: Configure whether access token should be treated as JWT. This should be set to `false` if the authorization server returns opaque tokens. Defaults to `true`. - accessTokenLocation: - type: string - description: Path to the token file containing an access token to be used for authentication. - audience: - type: string - description: "OAuth audience to use when authenticating against the authorization server. Some authorization servers require the audience to be explicitly set. The possible values depend on how the authorization server is configured. By default, `audience` is not specified when performing the token endpoint request." - certificateAndKey: + target: + type: object + properties: + alias: + type: string + pattern: "^[a-zA-Z0-9\\._\\-]{1,100}$" + description: Alias used to reference the Kafka cluster. + bootstrapServers: + type: string + description: A comma-separated list of `host:port` pairs for establishing the connection to the Kafka cluster. + groupId: + type: string + description: A unique ID that identifies the Connect cluster group. Required. + configStorageTopic: + type: string + description: The name of the Kafka topic where connector configurations are stored. Required. + statusStorageTopic: + type: string + description: The name of the Kafka topic where connector and task statuses are stored. Required. + offsetStorageTopic: + type: string + description: The name of the Kafka topic where source connector offsets are stored. Required. + tls: + type: object + properties: + trustedCertificates: + type: array + items: type: object properties: secretName: @@ -141,171 +98,198 @@ spec: description: The name of the Secret containing the certificate. certificate: type: string - description: The name of the file certificate in the Secret. - key: + description: The name of the file certificate in the secret. + pattern: type: string - description: The name of the private key in the Secret. + description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." + oneOf: + - properties: + certificate: {} + required: + - certificate + - properties: + pattern: {} + required: + - pattern required: - secretName - - certificate - - key - description: Reference to the `Secret` which holds the certificate and private key pair. - clientAssertion: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes secret containing the client assertion which was manually configured for the client. - clientAssertionLocation: - type: string - description: Path to the file containing the client assertion to be used for authentication. - clientAssertionType: - type: string - description: "The client assertion type. If not set, and either `clientAssertion` or `clientAssertionLocation` is configured, this value defaults to `urn:ietf:params:oauth:client-assertion-type:jwt-bearer`." - clientId: - type: string - description: OAuth Client ID which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - clientSecret: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the OAuth client secret which the Kafka client can use to authenticate against the OAuth server and use the token endpoint URI. - connectTimeoutSeconds: - type: integer - description: "The connect timeout in seconds when connecting to authorization server. If not set, the effective connect timeout is 60 seconds." - disableTlsHostnameVerification: - type: boolean - description: Enable or disable TLS hostname verification. Default value is `false`. - enableMetrics: - type: boolean - description: Enable or disable OAuth metrics. Default value is `false`. - httpRetries: - type: integer - description: "The maximum number of retries to attempt if an initial HTTP request fails. If not set, the default is to not attempt any retries." - httpRetryPauseMs: - type: integer - description: "The pause to take before retrying a failed HTTP request. If not set, the default is to not pause at all but to immediately repeat a request." - includeAcceptHeader: - type: boolean - description: Whether the Accept header should be set in requests to the authorization servers. The default value is `true`. - maxTokenExpirySeconds: - type: integer - description: Set or limit time-to-live of the access tokens to the specified number of seconds. This should be set if the authorization server returns opaque tokens. - passwordSecret: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the password. - password: - type: string - description: The name of the key in the Secret under which the password is stored. - required: - - secretName - - password - description: Reference to the `Secret` which holds the password. - readTimeoutSeconds: - type: integer - description: "The read timeout in seconds when connecting to authorization server. If not set, the effective read timeout is 60 seconds." - refreshToken: - type: object - properties: - key: - type: string - description: The key under which the secret value is stored in the Kubernetes Secret. - secretName: - type: string - description: The name of the Kubernetes Secret containing the secret value. - required: - - key - - secretName - description: Link to Kubernetes Secret containing the refresh token which can be used to obtain access token from the authorization server. - saslExtensions: - additionalProperties: + description: Trusted certificates for TLS connection. + description: TLS configuration for connecting MirrorMaker 2 connectors to a cluster. + authentication: + type: object + properties: + certificateAndKey: + type: object + properties: + secretName: type: string - type: object - description: SASL extensions parameters. - scope: - type: string - description: OAuth scope to use when authenticating against the authorization server. Some authorization servers require this to be set. The possible values depend on how authorization server is configured. By default `scope` is not specified when doing the token endpoint request. - tlsTrustedCertificates: - type: array - items: - type: object - properties: - secretName: - type: string - description: The name of the Secret containing the certificate. - certificate: - type: string - description: The name of the file certificate in the secret. - pattern: - type: string - description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." - oneOf: - - properties: - certificate: {} - required: - - certificate - - properties: - pattern: {} - required: - - pattern - required: - - secretName - description: Trusted certificates for TLS connection to the OAuth server. - tokenEndpointUri: - type: string - description: Authorization server token endpoint URI. - type: - type: string - enum: - - tls - - scram-sha-256 - - scram-sha-512 - - plain - - oauth - description: "Authentication type. Currently the supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and 'oauth'. `scram-sha-256` and `scram-sha-512` types use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 Authentication, respectively. `plain` type uses SASL PLAIN Authentication. `oauth` type uses SASL OAUTHBEARER Authentication. The `tls` type uses TLS Client Authentication. The `tls` type is supported only over TLS connections." - username: - type: string - description: Username used for the authentication. - required: - - type - description: Authentication configuration for connecting to the cluster. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: "The MirrorMaker 2 cluster config. Properties with the following prefixes cannot be set: ssl., sasl., security., listeners, plugin.path, rest., bootstrap.servers, consumer.interceptor.classes, producer.interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." - required: - - alias - - bootstrapServers - description: Kafka clusters for mirroring. + description: The name of the Secret containing the certificate. + certificate: + type: string + description: The name of the file certificate in the Secret. + key: + type: string + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." + required: + - secretName + - certificate + - key + description: Reference to the `Secret` which holds the certificate and private key pair. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: Configuration for the custom authentication mechanism. Only properties with the `sasl.` and `ssl.keystore.` prefixes are allowed. Specify other options in the regular configuration section of the custom resource. + passwordSecret: + type: object + properties: + secretName: + type: string + description: The name of the Secret containing the password. + password: + type: string + description: The name of the key in the Secret under which the password is stored. + required: + - secretName + - password + description: Reference to the `Secret` which holds the password. + sasl: + type: boolean + description: Enable or disable SASL on this authentication mechanism. + type: + type: string + enum: + - tls + - scram-sha-256 + - scram-sha-512 + - plain + - custom + description: "Specifies the authentication type. Supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and `custom`. `tls` uses TLS client authentication and is supported only over TLS connections. `scram-sha-256` and `scram-sha-512` use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 authentication, respectively. `plain` uses SASL PLAIN authentication. `custom` allows you to configure a custom authentication mechanism." + username: + type: string + description: Username used for the authentication. + required: + - type + description: Authentication configuration for connecting to the cluster. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: "The MirrorMaker 2 cluster config. Properties with the following prefixes cannot be set: group.id, config.storage.topic, offset.storage.topic, status.storage.topic, ssl., sasl., security., listeners, plugin.path, rest., bootstrap.servers, consumer.interceptor.classes, producer.interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." + required: + - alias + - bootstrapServers + - groupId + - configStorageTopic + - statusStorageTopic + - offsetStorageTopic + description: The target Apache Kafka cluster. The target Kafka cluster is used by the underlying Kafka Connect framework for its internal topics. mirrors: type: array items: type: object properties: - sourceCluster: - type: string - description: The alias of the source cluster used by the Kafka MirrorMaker 2 connectors. The alias must match a cluster in the list at `spec.clusters`. - targetCluster: - type: string - description: The alias of the target cluster used by the Kafka MirrorMaker 2 connectors. The alias must match a cluster in the list at `spec.clusters`. + source: + type: object + properties: + alias: + type: string + pattern: "^[a-zA-Z0-9\\._\\-]{1,100}$" + description: Alias used to reference the Kafka cluster. + bootstrapServers: + type: string + description: A comma-separated list of `host:port` pairs for establishing the connection to the Kafka cluster. + tls: + type: object + properties: + trustedCertificates: + type: array + items: + type: object + properties: + secretName: + type: string + description: The name of the Secret containing the certificate. + certificate: + type: string + description: The name of the file certificate in the secret. + pattern: + type: string + description: "Pattern for the certificate files in the secret. Use the link:https://en.wikipedia.org/wiki/Glob_(programming)[_glob syntax_] for the pattern. All files in the secret that match the pattern are used." + oneOf: + - properties: + certificate: {} + required: + - certificate + - properties: + pattern: {} + required: + - pattern + required: + - secretName + description: Trusted certificates for TLS connection. + description: TLS configuration for connecting MirrorMaker 2 connectors to a cluster. + authentication: + type: object + properties: + certificateAndKey: + type: object + properties: + secretName: + type: string + description: The name of the Secret containing the certificate. + certificate: + type: string + description: The name of the file certificate in the Secret. + key: + type: string + description: "The name of the private key in the secret. The private key must be in unencrypted PKCS #8 format. For more information, see RFC 5208: https://datatracker.ietf.org/doc/html/rfc5208." + required: + - secretName + - certificate + - key + description: Reference to the `Secret` which holds the certificate and private key pair. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: Configuration for the custom authentication mechanism. Only properties with the `sasl.` and `ssl.keystore.` prefixes are allowed. Specify other options in the regular configuration section of the custom resource. + passwordSecret: + type: object + properties: + secretName: + type: string + description: The name of the Secret containing the password. + password: + type: string + description: The name of the key in the Secret under which the password is stored. + required: + - secretName + - password + description: Reference to the `Secret` which holds the password. + sasl: + type: boolean + description: Enable or disable SASL on this authentication mechanism. + type: + type: string + enum: + - tls + - scram-sha-256 + - scram-sha-512 + - plain + - custom + description: "Specifies the authentication type. Supported types are `tls`, `scram-sha-256`, `scram-sha-512`, `plain`, and `custom`. `tls` uses TLS client authentication and is supported only over TLS connections. `scram-sha-256` and `scram-sha-512` use SASL SCRAM-SHA-256 and SASL SCRAM-SHA-512 authentication, respectively. `plain` uses SASL PLAIN authentication. `custom` allows you to configure a custom authentication mechanism." + username: + type: string + description: Username used for the authentication. + required: + - type + description: Authentication configuration for connecting to the cluster. + config: + x-kubernetes-preserve-unknown-fields: true + type: object + description: "The MirrorMaker 2 cluster config. Properties with the following prefixes cannot be set: group.id, config.storage.topic, offset.storage.topic, status.storage.topic, ssl., sasl., security., listeners, plugin.path, rest., bootstrap.servers, consumer.interceptor.classes, producer.interceptor.classes (with the exception of: ssl.endpoint.identification.algorithm, ssl.cipher.suites, ssl.protocol, ssl.enabled.protocols)." + required: + - alias + - bootstrapServers + description: The source Apache Kafka cluster. The source Kafka cluster is used by the Kafka MirrorMaker 2 connectors. sourceConnector: type: object properties: @@ -313,13 +297,13 @@ spec: type: integer minimum: 1 description: The maximum number of tasks for the Kafka Connector. - pause: - type: boolean - description: Whether the connector should be paused. Defaults to false. + version: + type: string + description: Desired version or version range to respect when starting the Kafka Connector. This is only supported when using Kafka Connect version 4.1.0 and higher. config: x-kubernetes-preserve-unknown-fields: true type: object - description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max." + description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max, connector.plugin.version." state: type: string enum: @@ -362,62 +346,6 @@ spec: - fromConfigMap description: Configuration for altering offsets. description: The specification of the Kafka MirrorMaker 2 source connector. - heartbeatConnector: - type: object - properties: - tasksMax: - type: integer - minimum: 1 - description: The maximum number of tasks for the Kafka Connector. - pause: - type: boolean - description: Whether the connector should be paused. Defaults to false. - config: - x-kubernetes-preserve-unknown-fields: true - type: object - description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max." - state: - type: string - enum: - - paused - - stopped - - running - description: The state the connector should be in. Defaults to running. - autoRestart: - type: object - properties: - enabled: - type: boolean - description: Whether automatic restart for failed connectors and tasks should be enabled or disabled. - maxRestarts: - type: integer - description: "The maximum number of connector restarts that the operator will try. If the connector remains in a failed state after reaching this limit, it must be restarted manually by the user. Defaults to an unlimited number of restarts." - description: Automatic restart of connector and tasks configuration. - listOffsets: - type: object - properties: - toConfigMap: - type: object - properties: - name: - type: string - description: Reference to the ConfigMap where the list of offsets will be written to. - required: - - toConfigMap - description: Configuration for listing offsets. - alterOffsets: - type: object - properties: - fromConfigMap: - type: object - properties: - name: - type: string - description: Reference to the ConfigMap where the new offsets are stored. - required: - - fromConfigMap - description: Configuration for altering offsets. - description: The specification of the Kafka MirrorMaker 2 heartbeat connector. checkpointConnector: type: object properties: @@ -425,13 +353,13 @@ spec: type: integer minimum: 1 description: The maximum number of tasks for the Kafka Connector. - pause: - type: boolean - description: Whether the connector should be paused. Defaults to false. + version: + type: string + description: Desired version or version range to respect when starting the Kafka Connector. This is only supported when using Kafka Connect version 4.1.0 and higher. config: x-kubernetes-preserve-unknown-fields: true type: object - description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max." + description: "The Kafka Connector configuration. The following properties cannot be set: name, connector.class, tasks.max, connector.plugin.version." state: type: string enum: @@ -477,24 +405,17 @@ spec: topicsPattern: type: string description: "A regular expression matching the topics to be mirrored, for example, \"topic1\\|topic2\\|topic3\". Comma-separated lists are also supported." - topicsBlacklistPattern: - type: string - description: A regular expression matching the topics to exclude from mirroring. Comma-separated lists are also supported. topicsExcludePattern: type: string description: A regular expression matching the topics to exclude from mirroring. Comma-separated lists are also supported. groupsPattern: type: string description: A regular expression matching the consumer groups to be mirrored. Comma-separated lists are also supported. - groupsBlacklistPattern: - type: string - description: A regular expression matching the consumer groups to exclude from mirroring. Comma-separated lists are also supported. groupsExcludePattern: type: string description: A regular expression matching the consumer groups to exclude from mirroring. Comma-separated lists are also supported. required: - - sourceCluster - - targetCluster + - source description: Configuration of the MirrorMaker 2 connectors. resources: type: object @@ -657,13 +578,25 @@ spec: rack: type: object properties: + envVarName: + type: string + description: The name of the environment variable that defines the rack ID. Its value sets the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect or MirrorMaker 2. topologyKey: type: string example: topology.kubernetes.io/zone description: "A key that matches labels assigned to the Kubernetes cluster nodes. The value of the label is used to set a broker's `broker.rack` config, and the `client.rack` config for Kafka Connect or MirrorMaker 2." - required: - - topologyKey + type: + type: string + enum: + - topology-label + - environment-variable + description: "Specifies the rack awareness type. Supported types are `topology-label` and `environment-variable`. `topology-label` uses a Kubernetes worker node label to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. `environment-variable` uses an environment variable to set the `broker.rack` configuration for Kafka brokers and the `client.rack` configuration for Kafka Connect and MirrorMaker 2. When not specified, `topology-label` type is used by default." description: Configuration of the node label which will be used as the `client.rack` consumer configuration. + x-kubernetes-validations: + - rule: (has(self.type) && self.type != "topology-label") || self.topologyKey != "" + message: topologyKey property is required + - rule: has(self.type) == false || self.type != "environment-variable" || self.envVarName != "" + message: envVarName property is required metricsConfig: type: object properties: @@ -671,7 +604,8 @@ spec: type: string enum: - jmxPrometheusExporter - description: Metrics type. Only 'jmxPrometheusExporter' supported currently. + - strimziMetricsReporter + description: Metrics type. The supported types are `jmxPrometheusExporter` and `strimziMetricsReporter`. Type `jmxPrometheusExporter` uses the Prometheus JMX Exporter to expose Kafka JMX metrics in Prometheus format through an HTTP endpoint. Type `strimziMetricsReporter` uses the Strimzi Metrics Reporter to directly expose Kafka metrics in Prometheus format through an HTTP endpoint. valueFrom: type: object properties: @@ -685,50 +619,36 @@ spec: optional: type: boolean description: Reference to the key in the ConfigMap containing the configuration. - description: 'ConfigMap entry where the Prometheus JMX Exporter configuration is stored. ' + description: ConfigMap entry where the Prometheus JMX Exporter configuration is stored. + values: + type: object + properties: + allowList: + type: array + items: + type: string + description: A list of regex patterns to filter the metrics to collect. Should contain at least one element. + description: Configuration values for the Strimzi Metrics Reporter. required: - type - - valueFrom description: Metrics configuration. + x-kubernetes-validations: + - rule: self.type != 'jmxPrometheusExporter' || has(self.valueFrom) + message: valueFrom property is required tracing: type: object properties: type: type: string enum: - - jaeger - opentelemetry - description: "Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. As of Strimzi 0.37.0, `jaeger` type is not supported anymore and this option is ignored." + description: Type of the tracing used. Currently the only supported type is `opentelemetry` for OpenTelemetry tracing. required: - type description: The configuration of tracing in Kafka Connect. template: type: object properties: - deployment: - type: object - properties: - metadata: - type: object - properties: - labels: - additionalProperties: - type: string - type: object - description: Labels added to the Kubernetes resource. - annotations: - additionalProperties: - type: string - type: object - description: Annotations added to the Kubernetes resource. - description: Metadata applied to the resource. - deploymentStrategy: - type: string - enum: - - RollingUpdate - - Recreate - description: Pod replacement strategy for deployment configuration changes. Valid values are `RollingUpdate` and `Recreate`. Defaults to `RollingUpdate`. - description: Template for Kafka Connect `Deployment`. podSet: type: object properties: @@ -1248,6 +1168,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -1283,7 +1232,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -1304,20 +1253,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -1325,7 +1274,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -1344,7 +1293,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -1352,7 +1309,75 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + templatedVolumes: + type: array + items: + type: object + properties: + name: + type: string + description: Name to use for the volume. Required. + secret: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + optional: + type: boolean + secretName: + type: string + description: "`Secret` to use to populate the volume. The name of the Secret and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + configMap: + type: object + properties: + defaultMode: + type: integer + items: + type: array + items: + type: object + properties: + key: + type: string + mode: + type: integer + path: + type: string + name: + type: string + optional: + type: boolean + description: "`ConfigMap` to use to populate the volume. The name of the ConfigMap and the items key and path fields can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + persistentVolumeClaim: + type: object + properties: + claimName: + type: string + readOnly: + type: boolean + description: "`PersistentVolumeClaim` object to use to populate the volume. The name of the Persistent Volume Claim can use placeholders that would be replaced for every individual node. Valid placeholders that you can use in the template are `{nodeId}` and `{nodePodName}`." + oneOf: + - properties: + secret: {} + configMap: {} + persistentVolumeClaim: {} + description: Additional volumes that can be mounted to the pod. These volumes can use templates to mount different volumes into individual Pods. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka Connect `Pods`. apiService: type: object @@ -2265,6 +2290,35 @@ spec: ip: type: string description: The pod's HostAliases. HostAliases is an optional list of hosts and IPs that will be injected into the Pod's hosts file if specified. + dnsPolicy: + type: string + enum: + - ClusterFirst + - ClusterFirstWithHostNet + - Default + - None + description: "The pod's DNSPolicy. Defaults to `ClusterFirst`. Valid values are `ClusterFirstWithHostNet`, `ClusterFirst`, `Default` or `None`." + dnsConfig: + type: object + properties: + nameservers: + type: array + items: + type: string + options: + type: array + items: + type: object + properties: + name: + type: string + value: + type: string + searches: + type: array + items: + type: string + description: "The pod's DNSConfig. If specified, it will be merged to the generated DNS configuration based on the DNSPolicy." enableServiceLinks: type: boolean description: Indicates whether information about services should be injected into Pod's environment variables. @@ -2300,7 +2354,7 @@ spec: type: boolean secretName: type: string - description: Secret to use populate the volume. + description: '`Secret` to use to populate the volume.' configMap: type: object properties: @@ -2321,20 +2375,20 @@ spec: type: string optional: type: boolean - description: ConfigMap to use to populate the volume. + description: '`ConfigMap` to use to populate the volume.' emptyDir: type: object properties: medium: type: string + enum: + - Memory + description: "Medium represents the type of storage medium should back this volume. Valid values are unset or `Memory`. When not set, it will use the node's default medium." sizeLimit: - type: object - properties: - amount: - type: string - format: - type: string - description: EmptyDir to use to populate the volume. + type: string + pattern: "^([0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$" + description: The total amount of local storage required for this EmptyDir volume (for example 1Gi). + description: '`EmptyDir` to use to populate the volume.' persistentVolumeClaim: type: object properties: @@ -2342,7 +2396,7 @@ spec: type: string readOnly: type: boolean - description: PersistentVolumeClaim object to use to populate the volume. + description: '`PersistentVolumeClaim` object to use to populate the volume.' csi: type: object properties: @@ -2361,7 +2415,15 @@ spec: additionalProperties: type: string type: object - description: CSIVolumeSource object to use to populate the volume. + description: '`CSIVolumeSource` object to use to populate the volume.' + image: + type: object + properties: + pullPolicy: + type: string + reference: + type: string + description: '`ImageVolumeSource` object to use to populate the volume.' oneOf: - properties: secret: {} @@ -2369,7 +2431,11 @@ spec: emptyDir: {} persistentVolumeClaim: {} csi: {} + image: {} description: Additional volumes that can be mounted to the pod. + hostUsers: + type: boolean + description: "Use the host user namespace. Optional. Defaults to `true`. When `true` or not set, the pod runs in the host user namespace. This is required when the pod needs features available only in the host namespace, such as loading kernel modules with `CAP_SYS_MODULE`.When set to `false`, the pod runs in a new user namespace. Setting `false` helps mitigate container breakout vulnerabilities and allows containers to run as `root` without granting `root` privileges on the host. This property is alpha-level in Kubernetes and is supported only by Kubernetes clusters that enable the `UserNamespacesSupport` feature." description: Template for Kafka Connect Build `Pods`. The build pod is used only on Kubernetes. buildContainer: type: object @@ -2572,119 +2638,10 @@ spec: description: Metadata applied to the resource. description: Template for Secret of the Kafka Connect Cluster JMX authentication. description: "Template for Kafka Connect and Kafka MirrorMaker 2 resources. The template allows users to specify how the `Pods`, `Service`, and other services are generated." - externalConfiguration: - type: object - properties: - env: - type: array - items: - type: object - properties: - name: - type: string - description: Name of the environment variable which will be passed to the Kafka Connect pods. The name of the environment variable cannot start with `KAFKA_` or `STRIMZI_`. - valueFrom: - type: object - properties: - secretKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a Secret. - configMapKeyRef: - type: object - properties: - key: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a ConfigMap. - oneOf: - - properties: - secretKeyRef: {} - required: - - secretKeyRef - - properties: - configMapKeyRef: {} - required: - - configMapKeyRef - description: Value of the environment variable which will be passed to the Kafka Connect pods. It can be passed either as a reference to Secret or ConfigMap field. The field has to specify exactly one Secret or ConfigMap. - required: - - name - - valueFrom - description: Makes data from a Secret or ConfigMap available in the Kafka Connect pods as environment variables. - volumes: - type: array - items: - type: object - properties: - name: - type: string - description: Name of the volume which will be added to the Kafka Connect pods. - secret: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - optional: - type: boolean - secretName: - type: string - description: Reference to a key in a Secret. Exactly one Secret or ConfigMap has to be specified. - configMap: - type: object - properties: - defaultMode: - type: integer - items: - type: array - items: - type: object - properties: - key: - type: string - mode: - type: integer - path: - type: string - name: - type: string - optional: - type: boolean - description: Reference to a key in a ConfigMap. Exactly one Secret or ConfigMap has to be specified. - oneOf: - - properties: - secret: {} - required: - - secret - - properties: - configMap: {} - required: - - configMap - required: - - name - description: Makes data from a Secret or ConfigMap available in the Kafka Connect pods as volumes. - description: Pass data from Secrets or ConfigMaps to the Kafka Connect pods and use them to configure connectors. required: - - connectCluster + - replicas + - target + - mirrors description: The specification of the Kafka MirrorMaker 2 cluster. status: type: object @@ -2759,3 +2716,5 @@ spec: type: integer description: The current number of pods being used to provide this resource. description: The status of the Kafka MirrorMaker 2 cluster. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/049-Crd-kafkarebalance.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/049-Crd-kafkarebalance.yaml index 024b15248..ae9ff2edd 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/049-Crd-kafkarebalance.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/crds/049-Crd-kafkarebalance.yaml @@ -21,7 +21,7 @@ spec: conversion: strategy: None versions: - - name: v1beta2 + - name: v1 served: true storage: true subresources: @@ -149,8 +149,17 @@ spec: sessionId: type: string description: The session identifier for requests to Cruise Control pertaining to this KafkaRebalance resource. This is used by the Kafka Rebalance operator to track the status of ongoing rebalancing operations. + progress: + type: object + properties: + rebalanceProgressConfigMap: + type: string + description: The name of the `ConfigMap` containing information related to the progress of a partition rebalance. + description: A reference to Config Map with the progress information. optimizationResult: x-kubernetes-preserve-unknown-fields: true type: object description: A JSON object describing the optimization result. description: The status of the Kafka rebalance. + required: + - spec diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-cruise-control.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-cruise-control.json index ef2ff14ed..0b32e3ef9 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-cruise-control.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-cruise-control.json @@ -464,7 +464,7 @@ }, { "datasource": "${DS_PROMETHEUS}", - "description": "Goal Violation Rate", + "description": "Goal violations per anomaly detector execution. Because Cruise Control emits cumulative counts, the Prometheus query window must be set to the anomaly detector run interval to preserve discrete per run counts (5 minutes by default)", "fieldConfig": { "defaults": { "color": { @@ -474,7 +474,7 @@ "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", - "axisLabel": "violations/sec", + "axisLabel": "violations", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", @@ -544,17 +544,16 @@ "targets": [ { "datasource": "${DS_PROMETHEUS}", - "expr": "sum(irate(kafka_cruisecontrol_anomalydetector_goal_violation_rate_count{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",kubernetes_pod_name=~\"$strimzi_cluster_name-cruise-control-.*\"}[5m]))", - "format": "time_series", + "expr": "round(max_over_time(increase(kafka_cruisecontrol_anomalydetector_goal_violation_rate_count{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",kubernetes_pod_name=~\"$strimzi_cluster_name-cruise-control-.*\"}[10m])[10m:]))", "hide": false, "instant": false, "interval": "", "intervalFactor": 1, - "legendFormat": "Goal Violation Rate", + "legendFormat": "Goal Violations", "refId": "A" } ], - "title": "Goal Violation Rate", + "title": "Goal Violations", "type": "timeseries" }, { @@ -2070,6 +2069,7 @@ }, "timezone": "", "title": "Strimzi Cruise Control", + "uid": "fb7da220847d2e2c", "version": 4, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-bridge.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-bridge.json index c9c51ec82..213352d0f 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-bridge.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-bridge.json @@ -231,7 +231,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -307,7 +307,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -383,7 +383,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -459,7 +459,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -3020,6 +3020,7 @@ }, "timezone": "", "title": "Strimzi Kafka Bridge", + "uid": "069669ed52041ef4", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-connect.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-connect.json index 4d278036f..d8e9d67c4 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-connect.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-connect.json @@ -771,7 +771,7 @@ "inspect": false }, "mappings": [], - "noValue": "0", + "noValue": "N/A", "thresholds": { "mode": "absolute", "steps": [ @@ -1803,6 +1803,7 @@ }, "timezone": "", "title": "Strimzi Kafka Connect", + "uid": "646c07d84adaf50d", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-exporter.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-exporter.json index 20a78fd36..6acac8ca8 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-exporter.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-exporter.json @@ -1627,6 +1627,7 @@ }, "timezone": "browser", "title": "Strimzi Kafka Exporter", + "uid": "52a9e2042fa225a2", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-mirror-maker-2.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-mirror-maker-2.json index 280912661..888f5753f 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-mirror-maker-2.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-mirror-maker-2.json @@ -133,7 +133,7 @@ "step": 40 } ], - "title": "Number of Connectors", + "title": "Total Number of Running Connectors", "type": "stat" }, { @@ -211,45 +211,18 @@ }, { "datasource": "${DS_PROMETHEUS}", + "description": "Time duration between record timestamp in the source topic and the time when the source connector handles the record (aka. MM2 consumer lag time).", "fieldConfig": { "defaults": { "color": { - "mode": "palette-classic" + "mode": "thresholds" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "Bytes/sec", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "align": "auto", + "cellOptions": {}, + "filterable": false }, "mappings": [], - "min": 0, "thresholds": { "mode": "absolute", "steps": [ @@ -263,145 +236,233 @@ } ] }, - "unit": "bytes" + "unit": "none" }, - "overrides": [] + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "partition" + }, + "properties": [ + { + "id": "custom.width", + "value": 76 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Partition" + }, + "properties": [ + { + "id": "custom.width", + "value": 140 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Current" + }, + "properties": [ + { + "id": "custom.width", + "value": 126 + }, + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Average" + }, + "properties": [ + { + "id": "custom.width", + "value": 118 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Min" + }, + "properties": [ + { + "id": "custom.width", + "value": 109 + }, + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Max" + }, + "properties": [ + { + "id": "custom.width", + "value": 107 + }, + { + "id": "unit", + "value": "ms" + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Time" + }, + "properties": [ + { + "id": "custom.width", + "value": 188 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "Topic" + }, + "properties": [ + { + "id": "custom.width", + "value": 579 + } + ] + } + ] }, "gridPos": { "h": 8, - "w": 8, + "w": 16, "x": 8, "y": 1 }, - "id": 28, + "id": 39, "options": { - "legend": { - "calcs": [ - "mean", - "lastNotNull" + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true + "show": false }, - "tooltip": { - "mode": "multi", - "sort": "none" - } + "frameIndex": 0, + "showHeader": true, + "sortBy": [ + { + "desc": false, + "displayName": "Time" + } + ] }, "pluginVersion": "7.4.5", "targets": [ { "datasource": "${DS_PROMETHEUS}", - "expr": "sum(rate(kafka_consumer_incoming_byte_total{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}[5m]))", - "format": "time_series", + "exemplar": false, + "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", + "format": "table", + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "A" + }, + { + "datasource": "${DS_PROMETHEUS}", + "exemplar": false, + "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_avg{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", + "format": "table", "hide": false, - "intervalFactor": 2, - "legendFormat": "{{kubernetes_pod_name}}", - "metric": "", - "refId": "A", - "step": 4 + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "B" + }, + { + "datasource": "${DS_PROMETHEUS}", + "exemplar": false, + "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_min{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", + "format": "table", + "hide": false, + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "C" + }, + { + "datasource": "${DS_PROMETHEUS}", + "exemplar": false, + "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_max{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", + "format": "table", + "hide": false, + "instant": true, + "interval": "", + "legendFormat": "", + "refId": "D" } ], - "title": "Incoming Bytes per Second", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "Bytes/sec", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false + "title": "Record Age", + "transformations": [ + { + "id": "organize", + "options": { + "excludeByName": {}, + "indexByName": { + "Time": 0, + "Value": 3, + "partition": 2, + "topic": 1 }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" + "renameByName": { + "Value": "Lag", + "partition": "Partition", + "topic": "Topic" } - }, - "mappings": [], - "min": 0, - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" + } }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 8, - "x": 16, - "y": 1 - }, - "id": 29, - "options": { - "legend": { - "calcs": [ - "mean", - "lastNotNull" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(rate(kafka_producer_outgoing_byte_total{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}[5m]))", - "format": "time_series", - "hide": false, - "intervalFactor": 2, - "legendFormat": "{{kubernetes_pod_name}}", - "metric": "", - "refId": "A", - "step": 4 + "id": "merge", + "options": {} + }, + { + "id": "organize", + "options": { + "excludeByName": {}, + "indexByName": {}, + "renameByName": { + "Value #A": "Current", + "Value #Avg": "Average", + "Value #B": "Average", + "Value #C": "Min", + "Value #D": "Max", + "partition": "Partition", + "topic": "Topic", + "{partition=\"2\", topic=\"my-cluster-source.kafka-test-apps\"}": "Min" + } + } } ], - "title": "Outgoing Bytes per Second", - "type": "timeseries" + "type": "table" }, { "datasource": "${DS_PROMETHEUS}", @@ -476,7 +537,7 @@ "step": 40 } ], - "title": "Number of Tasks", + "title": "Total Number of Running Tasks", "type": "stat" }, { @@ -542,7 +603,7 @@ "targets": [ { "datasource": "${DS_PROMETHEUS}", - "exemplar": true, + "exemplar": false, "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_byte_rate{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"})", "format": "time_series", "interval": "", @@ -557,27 +618,25 @@ }, { "datasource": "${DS_PROMETHEUS}", + "description": "Consumer lag which in the context of the replication indicates whether the consumer in the source connector can keep up with the rate records are produced in the source.", "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "msgs", + "axisLabel": "offset", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -586,15 +645,9 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, - "decimals": 0, "mappings": [], "thresholds": { "mode": "absolute", @@ -615,222 +668,43 @@ }, "gridPos": { "h": 8, - "w": 8, + "w": 12, "x": 0, "y": 9 }, - "id": 43, + "id": 34, "options": { "legend": { - "calcs": [ - "lastNotNull" - ], - "displayMode": "table", + "calcs": [], + "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", "targets": [ { "datasource": "${DS_PROMETHEUS}", - "expr": "sum(kafka_connect_source_task_source_record_active_count{connector=~\"\\\".*SourceConnector\\\"\"})", + "exemplar": false, + "expr": "sum(kafka_consumer_fetch_manager_records_lag{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\",clientid=~\".*replication-consumer\\\"\"}) by (topic, partition)", "interval": "", - "legendFormat": "", + "legendFormat": "{{topic}} (partition: {{partition}})", "refId": "A" } ], - "title": "Source Connector - Connect Outstanding Messages Queue", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "bytes", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 8, - "x": 8, - "y": 9 - }, - "id": 44, - "options": { - "legend": { - "calcs": [ - "lastNotNull" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(kafka_producer_buffer_available_bytes{clientid=~\"\\\".*SourceConnector-[0-9]+\\\"\"})", - "interval": "", - "legendFormat": "{{clientid}}", - "refId": "A" - } - ], - "title": "Source Connector - Producer Available Buffer", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "ms", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 8, - "x": 16, - "y": 9 - }, - "id": 45, - "options": { - "legend": { - "calcs": [ - "lastNotNull" - ], - "displayMode": "table", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(kafka_connect_connector_task_offset_commit_avg_time_ms{connector=~\"\\\".*SourceConnector\\\"\", task=~\"[0-9]+\"})/count(kafka_connect_connector_task_offset_commit_avg_time_ms{connector=~\"\\\".*SourceConnector\\\"\", task=~\"[0-9]+\"})", - "interval": "", - "legendFormat": "", - "refId": "A" - } - ], - "title": "Source Connector - Average Offset Commit Time", + "title": "Consumer Lag", "type": "timeseries" }, { "datasource": "${DS_PROMETHEUS}", + "description": "Consumer lag which in the context of the replication indicates whether the consumer in the source connector can keep up with the rate records are produced in the source.", "fieldConfig": { "defaults": { "color": { @@ -838,11 +712,8 @@ }, "custom": { "align": "auto", - "cellOptions": { - "type": "auto" - }, - "filterable": false, - "inspect": false + "cellOptions": {}, + "filterable": false }, "mappings": [], "thresholds": { @@ -857,103 +728,17 @@ "value": 80 } ] - }, - "unit": "none" - }, - "overrides": [ - { - "matcher": { - "id": "byName", - "options": "partition" - }, - "properties": [ - { - "id": "custom.width", - "value": 76 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Partition" - }, - "properties": [ - { - "id": "custom.width", - "value": 76 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Current" - }, - "properties": [ - { - "id": "custom.width", - "value": 65 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Average" - }, - "properties": [ - { - "id": "custom.width", - "value": 70 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Min" - }, - "properties": [ - { - "id": "custom.width", - "value": 45 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Max" - }, - "properties": [ - { - "id": "custom.width", - "value": 41 - } - ] - }, - { - "matcher": { - "id": "byName", - "options": "Time" - }, - "properties": [ - { - "id": "custom.width", - "value": 146 - } - ] } - ] + }, + "overrides": [] }, "gridPos": { - "h": 10, + "h": 8, "w": 12, - "x": 0, - "y": 17 + "x": 12, + "y": 9 }, - "id": 39, + "id": 36, "options": { "cellHeight": "sm", "footer": { @@ -964,62 +749,23 @@ ], "show": false }, - "frameIndex": 0, - "showHeader": true, - "sortBy": [ - { - "desc": false, - "displayName": "Time" - } - ] + "frameIndex": 1, + "showHeader": true }, "pluginVersion": "7.4.5", "targets": [ { "datasource": "${DS_PROMETHEUS}", "exemplar": false, - "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", + "expr": "topk(20, sum(kafka_consumer_fetch_manager_records_lag{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\",clientid=~\".*replication-consumer\\\"\"}) by (partition, topic))", "format": "table", "instant": true, "interval": "", - "legendFormat": "", + "legendFormat": "{{topic}} (partition: {{partition}})", "refId": "A" - }, - { - "datasource": "${DS_PROMETHEUS}", - "exemplar": true, - "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_avg{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", - "format": "table", - "hide": false, - "instant": true, - "interval": "", - "legendFormat": "", - "refId": "B" - }, - { - "datasource": "${DS_PROMETHEUS}", - "exemplar": true, - "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_min{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", - "format": "table", - "hide": false, - "instant": true, - "interval": "", - "legendFormat": "", - "refId": "C" - }, - { - "datasource": "${DS_PROMETHEUS}", - "exemplar": true, - "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_record_age_ms_max{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", - "format": "table", - "hide": false, - "instant": true, - "interval": "", - "legendFormat": "", - "refId": "D" } ], - "title": "Record Age", + "title": "Consumer Lag", "transformations": [ { "id": "organize", @@ -1037,33 +783,198 @@ "topic": "Topic" } } - }, - { - "id": "merge", - "options": {} - }, - { - "id": "organize", - "options": { - "excludeByName": {}, - "indexByName": {}, - "renameByName": { - "Value #A": "Current", - "Value #Avg": "Average", - "Value #B": "Average", - "Value #C": "Min", - "Value #D": "Max", - "partition": "Partition", - "topic": "Topic", - "{partition=\"2\", topic=\"my-cluster-source.kafka-test-apps\"}": "Min" - } - } } ], "type": "table" }, { "datasource": "${DS_PROMETHEUS}", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "graph": false, + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": {}, + "thresholdsStyle": {} + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "binBps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 17 + }, + "id": 28, + "options": { + "legend": { + "calcs": [ + "mean", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + }, + "tooltipOptions": { + "mode": "single" + } + }, + "pluginVersion": "7.4.5", + "targets": [ + { + "datasource": "${DS_PROMETHEUS}", + "expr": "sum(rate(kafka_consumer_incoming_byte_total{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}[5m])) by (kubernetes_pod_name)", + "format": "time_series", + "hide": false, + "interval": "", + "intervalFactor": 2, + "legendFormat": "{{kubernetes_pod_name}}", + "metric": "", + "refId": "A", + "step": 4 + } + ], + "title": "Incoming Bytes per Second", + "type": "timeseries" + }, + { + "datasource": "${DS_PROMETHEUS}", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "graph": false, + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": {}, + "thresholdsStyle": {} + }, + "mappings": [], + "min": 0, + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "binBps" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 17 + }, + "id": 29, + "options": { + "legend": { + "calcs": [ + "mean", + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + }, + "tooltipOptions": { + "mode": "single" + } + }, + "pluginVersion": "7.4.5", + "targets": [ + { + "datasource": "${DS_PROMETHEUS}", + "expr": "sum(rate(kafka_producer_outgoing_byte_total{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}[5m])) by (kubernetes_pod_name)", + "format": "time_series", + "hide": false, + "intervalFactor": 2, + "legendFormat": "{{kubernetes_pod_name}}", + "metric": "", + "refId": "A", + "step": 4 + } + ], + "title": "Outgoing Bytes per Second", + "type": "timeseries" + }, + { + "datasource": "${DS_PROMETHEUS}", + "description": "The end to end latency from the record timestamp in the source topic to the time when the record is written successfully to the target cluster.", "fieldConfig": { "defaults": { "color": { @@ -1071,11 +982,8 @@ }, "custom": { "align": "auto", - "cellOptions": { - "type": "auto" - }, - "filterable": false, - "inspect": false + "cellOptions": {}, + "filterable": false }, "mappings": [], "thresholds": { @@ -1126,6 +1034,10 @@ { "id": "custom.width", "value": 65 + }, + { + "id": "unit", + "value": "ms" } ] }, @@ -1138,6 +1050,10 @@ { "id": "custom.width", "value": 70 + }, + { + "id": "unit", + "value": "ms" } ] }, @@ -1150,6 +1066,10 @@ { "id": "custom.width", "value": 45 + }, + { + "id": "unit", + "value": "ms" } ] }, @@ -1162,6 +1082,10 @@ { "id": "custom.width", "value": 41 + }, + { + "id": "unit", + "value": "ms" } ] }, @@ -1176,14 +1100,26 @@ "value": 155 } ] + }, + { + "matcher": { + "id": "byName", + "options": "Topic" + }, + "properties": [ + { + "id": "custom.width", + "value": 552 + } + ] } ] }, "gridPos": { - "h": 5, + "h": 8, "w": 12, - "x": 12, - "y": 17 + "x": 0, + "y": 25 }, "id": 40, "options": { @@ -1219,7 +1155,7 @@ }, { "datasource": "${DS_PROMETHEUS}", - "exemplar": true, + "exemplar": false, "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_replication_latency_ms_avg{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", "format": "table", "hide": false, @@ -1230,7 +1166,7 @@ }, { "datasource": "${DS_PROMETHEUS}", - "exemplar": true, + "exemplar": false, "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_replication_latency_ms_min{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", "format": "table", "hide": false, @@ -1241,7 +1177,7 @@ }, { "datasource": "${DS_PROMETHEUS}", - "exemplar": true, + "exemplar": false, "expr": "sum(kafka_connect_mirror_mirrorsourceconnector_replication_latency_ms_max{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\"}) by (partition, topic)", "format": "table", "hide": false, @@ -1296,6 +1232,7 @@ }, { "datasource": "${DS_PROMETHEUS}", + "description": "The latency for synchronizing the consumer groups of the source cluster to the target cluster.", "fieldConfig": { "defaults": { "color": { @@ -1303,11 +1240,8 @@ }, "custom": { "align": "auto", - "cellOptions": { - "type": "auto" - }, - "filterable": true, - "inspect": false + "cellOptions": {}, + "filterable": true }, "mappings": [], "thresholds": { @@ -1333,7 +1267,7 @@ "properties": [ { "id": "unit", - "value": "dtdurationms" + "value": "ms" }, { "id": "custom.width", @@ -1356,10 +1290,10 @@ ] }, "gridPos": { - "h": 5, + "h": 8, "w": 12, "x": 12, - "y": 22 + "y": 25 }, "id": 41, "options": { @@ -1379,7 +1313,7 @@ "refId": "A" } ], - "title": "Offset Synchronization Latency", + "title": "Consumer Group Offset Sync Latency", "transformations": [ { "id": "organize", @@ -1401,13 +1335,13 @@ "topic": 1 }, "renameByName": { + "Value": "Latency", "group": "Group", "instance": "", "partition": "Partition", "source": "Source", "target": "Target", - "topic": "Topic", - "Value": "Latency" + "topic": "Topic" } } } @@ -1422,9 +1356,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, @@ -1432,11 +1363,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -1445,13 +1376,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "mappings": [], "thresholds": { @@ -1467,129 +1393,247 @@ } ] }, + "unit": "bytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 0, + "y": 33 + }, + "id": 44, + "options": { + "legend": { + "calcs": [ + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + }, + "tooltipOptions": { + "mode": "single" + } + }, + "pluginVersion": "7.4.5", + "targets": [ + { + "datasource": "${DS_PROMETHEUS}", + "expr": "sum(kafka_producer_buffer_available_bytes{clientid=~\"\\\".*SourceConnector-[0-9]+\\\"\"})", + "interval": "", + "legendFormat": "{{clientid}}", + "refId": "A" + } + ], + "title": "Source Connector - Producer Available Buffer", + "type": "timeseries" + }, + { + "datasource": "${DS_PROMETHEUS}", + "description": "Time duration that this task takes to commit its offsets to the target.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "graph": false, + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": {}, + "thresholdsStyle": {} + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "ms" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 8, + "x": 8, + "y": 33 + }, + "id": 45, + "options": { + "legend": { + "calcs": [ + "lastNotNull" + ], + "displayMode": "table", + "placement": "bottom", + "showLegend": true + }, + "tooltip": { + "mode": "multi", + "sort": "none" + }, + "tooltipOptions": { + "mode": "single" + } + }, + "pluginVersion": "7.4.5", + "targets": [ + { + "datasource": "${DS_PROMETHEUS}", + "expr": "sum(kafka_connect_connector_task_offset_commit_avg_time_ms{connector=~\"\\\".*SourceConnector\\\"\", task=~\"[0-9]+\"})/count(kafka_connect_connector_task_offset_commit_avg_time_ms{connector=~\"\\\".*SourceConnector\\\"\", task=~\"[0-9]+\"})", + "interval": "", + "legendFormat": "", + "refId": "A" + } + ], + "title": "Source Connector - Average Offset Commit Time", + "type": "timeseries" + }, + { + "datasource": "${DS_PROMETHEUS}", + "description": "The number of records that this task has consumed from the source but not yet produced to the target.", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisLabel": "msgs", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 10, + "gradientMode": "none", + "hideFrom": { + "graph": false, + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "never", + "spanNulls": false, + "stacking": {}, + "thresholdsStyle": {} + }, + "decimals": 0, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, "unit": "short" }, - "overrides": [] + "overrides": [ + { + "__systemRef": "hideSeriesFrom", + "matcher": { + "id": "byNames", + "options": { + "mode": "exclude", + "names": [ + "sum(kafka_connect_source_task_source_record_active_count{connector=~\"\\\".*SourceConnector\\\"\"})" + ], + "prefix": "All except:", + "readOnly": true + } + }, + "properties": [ + { + "id": "custom.hideFrom", + "value": { + "graph": true, + "legend": false, + "tooltip": false + } + } + ] + } + ] }, "gridPos": { - "h": 9, - "w": 12, - "x": 0, - "y": 27 + "h": 8, + "w": 8, + "x": 16, + "y": 33 }, - "id": 34, + "id": 43, "options": { "legend": { - "calcs": [], - "displayMode": "list", + "calcs": [ + "lastNotNull" + ], + "displayMode": "table", "placement": "bottom", "showLegend": true }, "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", "targets": [ { "datasource": "${DS_PROMETHEUS}", - "exemplar": true, - "expr": "sum(kafka_consumer_fetch_manager_records_lag{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\",clientid!~\"consumer-mirrormaker2-.*\"}) by (topic, partition)", + "expr": "sum(kafka_connect_source_task_source_record_active_count{connector=~\"\\\".*SourceConnector\\\"\"})", "interval": "", - "legendFormat": "{{topic}} (partition: {{partition}})", + "legendFormat": "", "refId": "A" } ], - "title": "Consumer Lag", + "title": "Source Connector - Connect Outstanding Messages Queue", "type": "timeseries" }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "custom": { - "align": "auto", - "cellOptions": { - "type": "auto" - }, - "filterable": false, - "inspect": false - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - } - }, - "overrides": [] - }, - "gridPos": { - "h": 9, - "w": 12, - "x": 12, - "y": 27 - }, - "id": 36, - "options": { - "cellHeight": "sm", - "footer": { - "countRows": false, - "fields": "", - "reducer": [ - "sum" - ], - "show": false - }, - "showHeader": true - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "exemplar": true, - "expr": "sum(kafka_consumer_fetch_manager_records_lag{strimzi_io_kind=~\"KafkaMirrorMaker2\",strimzi_io_cluster=\"$strimzi_mirror_maker_cluster_name\",clientid!~\"consumer-mirrormaker2-.*\"}) by (partition, topic)", - "format": "table", - "instant": true, - "interval": "", - "legendFormat": "{{topic}} (partition: {{partition}})", - "refId": "A" - } - ], - "title": "Consumer Lag", - "transformations": [ - { - "id": "organize", - "options": { - "excludeByName": {}, - "indexByName": { - "Time": 0, - "Value": 3, - "partition": 2, - "topic": 1 - }, - "renameByName": { - "Value": "Lag", - "partition": "Partition", - "topic": "Topic" - } - } - } - ], - "type": "table" - }, { "collapsed": false, "datasource": "${DS_PROMETHEUS}", @@ -1597,7 +1641,7 @@ "h": 1, "w": 24, "x": 0, - "y": 36 + "y": 41 }, "id": 53, "panels": [], @@ -1614,6 +1658,7 @@ "datasource": "${DS_PROMETHEUS}", "fieldConfig": { "defaults": { + "custom": {}, "mappings": [], "thresholds": { "mode": "absolute", @@ -1635,7 +1680,7 @@ "h": 3, "w": 12, "x": 0, - "y": 37 + "y": 42 }, "id": 55, "options": { @@ -1654,6 +1699,7 @@ }, "showUnfilled": true, "sizing": "auto", + "text": {}, "valueMode": "color" }, "pluginVersion": "7.4.5", @@ -1679,9 +1725,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, @@ -1689,11 +1732,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -1702,13 +1745,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "decimals": 0, "mappings": [], @@ -1735,7 +1773,7 @@ "h": 6, "w": 12, "x": 12, - "y": 37 + "y": 42 }, "id": 59, "options": { @@ -1748,6 +1786,9 @@ "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", @@ -1767,6 +1808,7 @@ "datasource": "${DS_PROMETHEUS}", "fieldConfig": { "defaults": { + "custom": {}, "mappings": [], "thresholds": { "mode": "absolute", @@ -1788,7 +1830,7 @@ "h": 3, "w": 12, "x": 0, - "y": 40 + "y": 45 }, "id": 57, "options": { @@ -1807,6 +1849,7 @@ }, "showUnfilled": true, "sizing": "auto", + "text": {}, "valueMode": "color" }, "pluginVersion": "7.4.5", @@ -1829,7 +1872,7 @@ "h": 1, "w": 24, "x": 0, - "y": 43 + "y": 48 }, "id": 51, "panels": [], @@ -1850,9 +1893,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "Cores", "axisPlacement": "auto", "barAlignment": 0, @@ -1860,11 +1900,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -1873,13 +1913,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "mappings": [], "thresholds": { @@ -1903,7 +1938,7 @@ "h": 7, "w": 6, "x": 0, - "y": 44 + "y": 49 }, "id": 30, "options": { @@ -1918,12 +1953,16 @@ "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", "targets": [ { "datasource": "${DS_PROMETHEUS}", + "exemplar": false, "expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_mirror_maker_cluster_name-mirrormaker2-.+\",container=\"$strimzi_mirror_maker_cluster_name-mirrormaker2\"}[5m])) by (pod)", "format": "time_series", "hide": false, @@ -1945,9 +1984,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "Memory", "axisPlacement": "auto", "barAlignment": 0, @@ -1955,11 +1991,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -1968,13 +2004,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "mappings": [], "thresholds": { @@ -1998,7 +2029,7 @@ "h": 7, "w": 6, "x": 6, - "y": 44 + "y": 49 }, "id": 31, "options": { @@ -2013,6 +2044,9 @@ "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", @@ -2040,9 +2074,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "% time in GC", "axisPlacement": "auto", "barAlignment": 0, @@ -2050,11 +2081,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -2063,13 +2094,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "mappings": [], "max": 100, @@ -2094,7 +2120,7 @@ "h": 7, "w": 6, "x": 12, - "y": 44 + "y": 49 }, "id": 32, "options": { @@ -2110,6 +2136,9 @@ "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", @@ -2138,9 +2167,6 @@ "mode": "palette-classic" }, "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, @@ -2148,11 +2174,11 @@ "fillOpacity": 10, "gradientMode": "none", "hideFrom": { + "graph": false, "legend": false, "tooltip": false, "viz": false }, - "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, @@ -2161,13 +2187,8 @@ }, "showPoints": "never", "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } + "stacking": {}, + "thresholdsStyle": {} }, "decimals": 0, "mappings": [], @@ -2192,7 +2213,7 @@ "h": 7, "w": 6, "x": 18, - "y": 44 + "y": 49 }, "id": 49, "options": { @@ -2207,6 +2228,9 @@ "tooltip": { "mode": "multi", "sort": "none" + }, + "tooltipOptions": { + "mode": "single" } }, "pluginVersion": "7.4.5", @@ -2224,8 +2248,9 @@ "type": "timeseries" } ], - "refresh": "5s", - "schemaVersion": 39, + "refresh": false, + "schemaVersion": 27, + "style": "dark", "tags": [ "Strimzi", "Kafka", @@ -2234,7 +2259,12 @@ "templating": { "list": [ { - "current": {}, + "current": { + "selected": false, + "text": "Prometheus", + "value": "Prometheus" + }, + "description": null, "error": null, "hide": 0, "includeAll": false, @@ -2249,36 +2279,57 @@ "type": "datasource" }, { - "current": {}, + "allValue": null, + "current": { + "selected": false, + "text": "default", + "value": "default" + }, "datasource": "${DS_PROMETHEUS}", "definition": "", + "description": null, + "error": null, "hide": 0, "includeAll": false, "label": "Namespace", "multi": false, "name": "kubernetes_namespace", "options": [], - "query": "query_result(kafka_connect_worker_connector_count{strimzi_io_kind=\"KafkaMirrorMaker2\"})", + "query": { + "query": "query_result(kafka_connect_worker_connector_count{strimzi_io_kind=\"KafkaMirrorMaker2\"})", + "refId": "Prometheus-kubernetes_namespace-Variable-Query" + }, "refresh": 1, "regex": "/.*namespace=\"([^\"]*).*/", "skipUrlSync": false, "sort": 0, "tagValuesQuery": "", + "tags": [], "tagsQuery": "", "type": "query", "useTags": false }, { - "current": {}, + "allValue": null, + "current": { + "selected": false, + "text": "my-mirror-maker-2", + "value": "my-mirror-maker-2" + }, "datasource": "${DS_PROMETHEUS}", "definition": "", + "description": null, + "error": null, "hide": 0, "includeAll": false, "label": "Cluster Name", "multi": false, "name": "strimzi_mirror_maker_cluster_name", "options": [], - "query": "query_result(kafka_connect_worker_connector_count{namespace=\"$kubernetes_namespace\", strimzi_io_kind=\"KafkaMirrorMaker2\"})", + "query": { + "query": "query_result(kafka_connect_worker_connector_count{namespace=\"$kubernetes_namespace\", strimzi_io_kind=\"KafkaMirrorMaker2\"})", + "refId": "Prometheus-strimzi_mirror_maker_cluster_name-Variable-Query" + }, "refresh": 1, "regex": "/.*strimzi_io_cluster=\"([^\"]*).*/", "skipUrlSync": false, @@ -2321,6 +2372,7 @@ }, "timezone": "", "title": "Strimzi Kafka Mirror Maker 2", - "version": 9, + "uid": "6e9806b052af8d25", + "version": 11, "weekStart": "" -} +} \ No newline at end of file diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-oauth.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-oauth.json index be572c9f1..732283910 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-oauth.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka-oauth.json @@ -153,7 +153,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "irate(strimzi_oauth_http_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) * 30", + "expr": "irate(strimzi_oauth_http_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) * 30", "format": "time_series", "instant": false, "interval": "60s", @@ -249,7 +249,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "irate(strimzi_oauth_validation_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) * 30", + "expr": "irate(strimzi_oauth_validation_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) * 30", "format": "time_series", "instant": false, "interval": "60s", @@ -344,7 +344,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "strimzi_oauth_http_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}", + "expr": "strimzi_oauth_http_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}", "format": "time_series", "instant": false, "interval": "60s", @@ -439,7 +439,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "strimzi_oauth_validation_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}", + "expr": "strimzi_oauth_validation_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}", "format": "time_series", "instant": false, "interval": "60s", @@ -534,7 +534,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "irate(strimzi_oauth_http_requests_totaltimems{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) / irate(strimzi_oauth_http_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval])", + "expr": "irate(strimzi_oauth_http_requests_totaltimems_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) / irate(strimzi_oauth_http_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval])", "format": "time_series", "instant": false, "interval": "60s", @@ -630,7 +630,7 @@ { "datasource": "${DS_PROMETHEUS}", "exemplar": true, - "expr": "irate(strimzi_oauth_validation_requests_totaltimems{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) / irate(strimzi_oauth_validation_requests_count{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval])\n", + "expr": "irate(strimzi_oauth_validation_requests_totaltimems_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval]) / irate(strimzi_oauth_validation_requests_count_total{namespace=\"$kubernetes_namespace\",pod=~\"$strimzi_cluster_name-$kafka_broker\"}[$__rate_interval])\n", "format": "time_series", "instant": false, "interval": "60s", @@ -764,6 +764,7 @@ }, "timezone": "", "title": "Strimzi Kafka OAuth", + "uid": "3797efba280f9ed6", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka.json index 4a4176b70..62fbf4946 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kafka.json @@ -447,7 +447,7 @@ "match": "null", "result": { "color": "#508642", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -526,7 +526,7 @@ "match": "null", "result": { "color": "#508642", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -1438,7 +1438,7 @@ "match": "null", "result": { "color": "rgba(237, 129, 40, 0.89)", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -1519,7 +1519,7 @@ "match": "null", "result": { "color": "rgba(237, 129, 40, 0.89)", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -1600,7 +1600,7 @@ "match": "null", "result": { "color": "rgba(237, 129, 40, 0.89)", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -1681,7 +1681,7 @@ "match": "null", "result": { "color": "rgba(237, 129, 40, 0.89)", - "text": "0" + "text": "N/A" } }, "type": "special" @@ -2383,7 +2383,7 @@ }, "gridPos": { "h": 8, - "w": 8, + "w": 12, "x": 0, "y": 47 }, @@ -2475,8 +2475,8 @@ }, "gridPos": { "h": 8, - "w": 8, - "x": 8, + "w": 12, + "x": 12, "y": 47 }, "id": 105, @@ -2567,9 +2567,9 @@ }, "gridPos": { "h": 8, - "w": 8, - "x": 16, - "y": 47 + "w": 12, + "x": 0, + "y": 55 }, "id": 106, "options": { @@ -2636,7 +2636,7 @@ "gridPos": { "h": 8, "w": 12, - "x": 0, + "x": 12, "y": 55 }, "id": 91, @@ -2722,97 +2722,6 @@ } ], "type": "table" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 12, - "x": 12, - "y": 55 - }, - "id": 110, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(kafka_server_zookeeperclientmetrics_zookeeperrequestlatencyms{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$kafka_broker\",strimzi_io_name=\"$strimzi_cluster_name-kafka\"}) by (kubernetes_pod_name)", - "format": "time_series", - "instant": false, - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "ZK request latecy", - "type": "timeseries" } ], "refresh": "5s", @@ -2974,6 +2883,7 @@ }, "timezone": "", "title": "Strimzi Kafka", + "uid": "7ddce8bf9ea7ddb5", "version": 4, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kraft.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kraft.json index 96055f96e..d46f434ec 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kraft.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-kraft.json @@ -2115,6 +2115,7 @@ }, "timezone": "", "title": "Strimzi KRaft", + "uid": "d1b98e4d984de29d", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-operators.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-operators.json index 8b3a05af0..5c074e8fd 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-operators.json +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-operators.json @@ -154,7 +154,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -226,7 +226,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -298,7 +298,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -370,7 +370,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -442,7 +442,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -493,13 +493,13 @@ "targets": [ { "datasource": "${DS_PROMETHEUS}", - "expr": "sum(strimzi_resources{kind=\"KafkaMirrorMaker\"})", + "expr": "sum(strimzi_resources{kind=\"KafkaNodePool\"})", "format": "time_series", "intervalFactor": 1, "refId": "A" } ], - "title": "Mirror Maker CRs", + "title": "Node Pool CRs", "type": "stat" }, { @@ -514,7 +514,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -586,7 +586,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -658,7 +658,7 @@ "options": { "match": "null", "result": { - "text": "0" + "text": "N/A" } }, "type": "special" @@ -1771,6 +1771,7 @@ }, "timezone": "", "title": "Strimzi Operators", + "uid": "31545fecf446bf88", "version": 6, "weekStart": "" } diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-zookeeper.json b/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-zookeeper.json deleted file mode 100644 index 4a8aa7524..000000000 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/files/grafana-dashboards/strimzi-zookeeper.json +++ /dev/null @@ -1,1412 +0,0 @@ -{ - "__requires": [ - { - "type": "grafana", - "id": "grafana", - "name": "Grafana", - "version": "7.4.5" - }, - { - "type": "panel", - "id": "stat", - "name": "Stat" - }, - { - "type": "datasource", - "id": "prometheus", - "name": "Prometheus" - }, - { - "type": "panel", - "id": "timeseries", - "name": "Timeseries" - } - ], - "annotations": { - "list": [ - { - "builtIn": 1, - "datasource": { - "type": "datasource", - "uid": "grafana" - }, - "enable": true, - "hide": true, - "iconColor": "rgba(0, 211, 255, 1)", - "limit": 100, - "name": "Annotations & Alerts", - "showIn": 0, - "type": "dashboard" - } - ] - }, - "editable": true, - "fiscalYearStartMonth": 0, - "graphTooltip": 0, - "id": 11, - "links": [], - "panels": [ - { - "collapsed": false, - "datasource": { - "type": "datasource", - "uid": "grafana" - }, - "gridPos": { - "h": 1, - "w": 24, - "x": 0, - "y": 0 - }, - "id": 26, - "panels": [], - "targets": [ - { - "datasource": { - "type": "datasource", - "uid": "grafana" - }, - "refId": "A" - } - ], - "title": "ZooKeeper", - "type": "row" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Quorum size of ZooKeeper ensemble", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "#d44a3a", - "value": null - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 2 - }, - { - "color": "#299c46", - "value": 3 - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 0, - "y": 1 - }, - "id": 52, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "max(zookeeper_quorumsize{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",strimzi_io_kind=\"Kafka\"})", - "format": "time_series", - "intervalFactor": 1, - "refId": "A" - } - ], - "title": "Quorum Size", - "type": "stat" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Number of active connections", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "#299c46", - "value": null - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 60 - }, - { - "color": "#d44a3a", - "value": 120 - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 4, - "y": 1 - }, - "id": 54, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(zookeeper_numaliveconnections{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",strimzi_io_kind=\"Kafka\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\"})", - "format": "time_series", - "intervalFactor": 1, - "refId": "A" - } - ], - "title": "Active Connections", - "type": "stat" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Number of queued requests in the server. This goes up when the server receives more requests than it can process", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 8, - "x": 8, - "y": 1 - }, - "id": 12, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(zookeeper_outstandingrequests{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",strimzi_io_kind=\"Kafka\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\"}) by (kubernetes_pod_name)", - "format": "time_series", - "interval": "", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "Outstanding Requests", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Amount of time (in ms) it takes for the server to respond to a client request", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "Request Latency (ms)", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "ms" - }, - "overrides": [] - }, - "gridPos": { - "h": 8, - "w": 8, - "x": 16, - "y": 1 - }, - "id": 6, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(zookeeper_avgrequestlatency{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\"}) by (kubernetes_pod_name)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "Request Latency - Average", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "#299c46", - "value": null - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 500 - }, - { - "color": "#d44a3a", - "value": 800 - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 0, - "y": 5 - }, - "id": 64, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "avg(zookeeper_inmemorydatatree_nodecount{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",strimzi_io_kind=\"Kafka\"})", - "format": "time_series", - "intervalFactor": 1, - "refId": "A" - } - ], - "title": "Number of ZNodes", - "type": "stat" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Number of watchers", - "fieldConfig": { - "defaults": { - "color": { - "mode": "thresholds" - }, - "mappings": [ - { - "options": { - "match": "null", - "result": { - "text": "N/A" - } - }, - "type": "special" - } - ], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "#299c46", - "value": null - }, - { - "color": "rgba(237, 129, 40, 0.89)", - "value": 100 - }, - { - "color": "#d44a3a", - "value": 200 - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 4, - "w": 4, - "x": 4, - "y": 5 - }, - "id": 66, - "maxDataPoints": 100, - "options": { - "colorMode": "value", - "graphMode": "none", - "justifyMode": "auto", - "orientation": "horizontal", - "reduceOptions": { - "calcs": [ - "lastNotNull" - ], - "fields": "", - "values": false - }, - "showPercentChange": false, - "textMode": "auto", - "wideLayout": true - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(zookeeper_inmemorydatatree_watchcount{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\",strimzi_io_kind=\"Kafka\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\"})", - "format": "time_series", - "intervalFactor": 1, - "refId": "A" - } - ], - "title": "Number of watchers", - "type": "stat" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "ZooKeeper pods memory usage", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 0, - "y": 9 - }, - "id": 87, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(container_memory_usage_bytes{namespace=\"$kubernetes_namespace\",container=\"zookeeper\",pod=~\"$strimzi_cluster_name-$zk_node\"}) by (pod)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{pod}}", - "refId": "A" - } - ], - "title": "Memory Usage", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Aggregated ZooKeeper pods CPU usage", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 6, - "y": 9 - }, - "id": 85, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(rate(container_cpu_usage_seconds_total{namespace=\"$kubernetes_namespace\",container=\"zookeeper\",pod=~\"$strimzi_cluster_name-$zk_node\"}[5m])) by (pod)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{pod}}", - "refId": "A" - } - ], - "title": "CPU Usage", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Kafka broker pods disk usage", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "bytes" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 12, - "y": 9 - }, - "id": 89, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(kubelet_volume_stats_available_bytes{namespace=\"$kubernetes_namespace\",persistentvolumeclaim=~\"data(-[0-9]+)?-$strimzi_cluster_name-$zk_node\", persistentvolumeclaim=~\".*zookeeper(-[0-9]+)?$\"}) by (persistentvolumeclaim)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{persistentvolumeclaim}}", - "refId": "A" - } - ], - "title": "Available Disk Space", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "Open File Descriptors", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "none" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 18, - "y": 9 - }, - "id": 96, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(process_open_fds{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\",container=\"zookeeper\"}) by (kubernetes_pod_name)", - "format": "time_series", - "hide": false, - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "Open File Descriptors", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "decbytes" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 0, - "y": 16 - }, - "id": 91, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(jvm_memory_used_bytes{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\",strimzi_io_name=\"$strimzi_cluster_name-zookeeper\"}) by (kubernetes_pod_name)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "JVM Memory Used", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "ms" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 6, - "y": 16 - }, - "id": 93, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(rate(jvm_gc_collection_seconds_sum{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\",strimzi_io_name=\"$strimzi_cluster_name-zookeeper\"}[5m])) by (kubernetes_pod_name)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "JVM GC Time", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 12, - "y": 16 - }, - "id": 95, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(rate(jvm_gc_collection_seconds_count{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\",strimzi_io_name=\"$strimzi_cluster_name-zookeeper\"}[5m])) by (kubernetes_pod_name)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "JVM GC Count", - "type": "timeseries" - }, - { - "datasource": "${DS_PROMETHEUS}", - "description": "JVM thread count", - "fieldConfig": { - "defaults": { - "color": { - "mode": "palette-classic" - }, - "custom": { - "axisBorderShow": false, - "axisCenteredZero": false, - "axisColorMode": "text", - "axisLabel": "", - "axisPlacement": "auto", - "barAlignment": 0, - "drawStyle": "line", - "fillOpacity": 10, - "gradientMode": "none", - "hideFrom": { - "legend": false, - "tooltip": false, - "viz": false - }, - "insertNulls": false, - "lineInterpolation": "linear", - "lineWidth": 1, - "pointSize": 5, - "scaleDistribution": { - "type": "linear" - }, - "showPoints": "never", - "spanNulls": false, - "stacking": { - "group": "A", - "mode": "none" - }, - "thresholdsStyle": { - "mode": "off" - } - }, - "mappings": [], - "thresholds": { - "mode": "absolute", - "steps": [ - { - "color": "green", - "value": null - }, - { - "color": "red", - "value": 80 - } - ] - }, - "unit": "short" - }, - "overrides": [] - }, - "gridPos": { - "h": 7, - "w": 6, - "x": 18, - "y": 16 - }, - "id": 97, - "options": { - "legend": { - "calcs": [], - "displayMode": "list", - "placement": "bottom", - "showLegend": true - }, - "tooltip": { - "mode": "multi", - "sort": "none" - } - }, - "pluginVersion": "7.4.5", - "targets": [ - { - "datasource": "${DS_PROMETHEUS}", - "expr": "sum(jvm_threads_current{namespace=\"$kubernetes_namespace\",kubernetes_pod_name=~\"$strimzi_cluster_name-$zk_node\",strimzi_io_name=\"$strimzi_cluster_name-zookeeper\"}) by (kubernetes_pod_name)", - "format": "time_series", - "intervalFactor": 1, - "legendFormat": "{{kubernetes_pod_name}}", - "refId": "A" - } - ], - "title": "JVM thread count", - "type": "timeseries" - } - ], - "refresh": "5s", - "schemaVersion": 39, - "tags": [ - "Strimzi", - "Kafka", - "Zookeeper" - ], - "templating": { - "list": [ - { - "current": {}, - "error": null, - "hide": 0, - "includeAll": false, - "label": "datasource", - "multi": false, - "name": "DS_PROMETHEUS", - "options": [], - "query": "prometheus", - "refresh": 1, - "regex": "", - "skipUrlSync": false, - "type": "datasource" - }, - { - "current": {}, - "datasource": "${DS_PROMETHEUS}", - "definition": "", - "hide": 0, - "includeAll": false, - "label": "Namespace", - "multi": false, - "name": "kubernetes_namespace", - "options": [], - "query": "query_result(zookeeper_inmemorydatatree_nodecount)", - "refresh": 1, - "regex": "/.*namespace=\"([^\"]*).*/", - "skipUrlSync": false, - "sort": 0, - "tagValuesQuery": "", - "tagsQuery": "", - "type": "query", - "useTags": false - }, - { - "current": {}, - "datasource": "${DS_PROMETHEUS}", - "definition": "", - "hide": 0, - "includeAll": false, - "label": "Cluster Name", - "multi": false, - "name": "strimzi_cluster_name", - "options": [], - "query": "query_result(zookeeper_inmemorydatatree_nodecount{namespace=\"$kubernetes_namespace\"})", - "refresh": 1, - "regex": "/.*strimzi_io_cluster=\"([^\"]*).*/", - "skipUrlSync": false, - "sort": 0, - "tagValuesQuery": "", - "tagsQuery": "", - "type": "query", - "useTags": false - }, - { - "allValue": ".*", - "current": {}, - "datasource": "${DS_PROMETHEUS}", - "definition": "", - "hide": 0, - "includeAll": true, - "label": "Node", - "multi": false, - "name": "zk_node", - "options": [], - "query": "query_result(zookeeper_inmemorydatatree_nodecount{namespace=\"$kubernetes_namespace\",strimzi_io_cluster=\"$strimzi_cluster_name\"})", - "refresh": 1, - "regex": "/.*pod_name=\"$strimzi_cluster_name-([^\"]*).*/", - "skipUrlSync": false, - "sort": 0, - "tagValuesQuery": "", - "tagsQuery": "", - "type": "query", - "useTags": false - } - ] - }, - "time": { - "from": "now-1h", - "to": "now" - }, - "timepicker": { - "refresh_intervals": [ - "5s", - "10s", - "30s", - "1m", - "5m", - "15m", - "30m", - "1h", - "2h", - "1d" - ], - "time_options": [ - "5m", - "15m", - "1h", - "6h", - "12h", - "24h", - "2d", - "7d", - "30d" - ] - }, - "timezone": "", - "title": "Strimzi ZooKeeper", - "version": 6, - "weekStart": "" -} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-ClusterRole-strimzi-cluster-operator-role.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-ClusterRole-strimzi-cluster-operator-role.yaml index b29eaa921..8ccd010ce 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-ClusterRole-strimzi-cluster-operator-role.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-ClusterRole-strimzi-cluster-operator-role.yaml @@ -64,13 +64,19 @@ rules: - delete - patch - update +- apiGroups: + - "" + resources: + # For inplace resizing, we need access to pods/resize subresource + - pods/resize + verbs: + - patch + - update - apiGroups: - "apps" resources: # The cluster operator needs to access and manage deployments to run deployment based Strimzi components - deployments - # The cluster operator needs to access and manage stateful sets to run stateful sets based Strimzi components - - statefulsets # The cluster operator needs to access replica-sets to manage Strimzi components and to determine error states - replicasets verbs: @@ -162,4 +168,17 @@ rules: - delete - patch - update +- apiGroups: + - gateway.networking.k8s.io + resources: + # The cluster operator needs to access and manage TLSRoutes to expose Strimzi components for external access + - tlsroutes + verbs: + - get + - list + - watch + - create + - delete + - patch + - update {{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-RoleBinding-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-RoleBinding-strimzi-cluster-operator.yaml index dae5166d6..d8b9f4fac 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-RoleBinding-strimzi-cluster-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/020-RoleBinding-strimzi-cluster-operator.yaml @@ -1,39 +1,35 @@ {{- if .Values.rbac.create -}} -{{- $root := . -}} -{{- $watchNamespaces := .Values.watchNamespaces -}} -{{- if $root.Values.watchAnyNamespace }} - {{- $watchNamespaces = list -}} -{{- end }} -{{- range append $watchNamespaces .Release.Namespace }} +{{- $namespaces := include "strimzi.watchNamespacesList" . | fromYamlArray }} +{{- range $i, $ns := $namespaces }} --- apiVersion: rbac.authorization.k8s.io/v1 -{{- if $root.Values.watchAnyNamespace }} +{{- if $.Values.watchAnyNamespace }} kind: ClusterRoleBinding {{- else }} kind: RoleBinding {{- end }} metadata: - {{- if $root.Values.watchAnyNamespace }} + {{- if $.Values.watchAnyNamespace }} name: strimzi-cluster-operator-namespaced {{- else }} name: strimzi-cluster-operator - {{- end }} - {{- if not $root.Values.watchAnyNamespace }} - namespace: {{ . }} + {{- end }} + {{- if not $.Values.watchAnyNamespace }} + namespace: {{ $ns }} {{- end }} labels: - app: {{ template "strimzi.name" $root }} - chart: {{ template "strimzi.chart" $root }} + app: {{ template "strimzi.name" $ }} + chart: {{ template "strimzi.chart" $ }} component: role-binding - release: {{ $root.Release.Name }} - heritage: {{ $root.Release.Service }} + release: {{ $.Release.Name }} + heritage: {{ $.Release.Service }} subjects: - kind: ServiceAccount - name: {{ $root.Values.serviceAccount }} - namespace: {{ $root.Release.Namespace }} + name: {{ $.Values.serviceAccount }} + namespace: {{ $.Release.Namespace }} roleRef: kind: ClusterRole name: strimzi-cluster-operator-namespaced apiGroup: rbac.authorization.k8s.io {{- end }} -{{- end }} +{{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-ClusterRole-strimzi-cluster-operator-role.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-ClusterRole-strimzi-cluster-operator-role.yaml index 0f70ff131..30b753bd4 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-ClusterRole-strimzi-cluster-operator-role.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-ClusterRole-strimzi-cluster-operator-role.yaml @@ -30,7 +30,6 @@ rules: - kafkanodepools - kafkaconnects - kafkaconnectors - - kafkamirrormakers - kafkabridges - kafkamirrormaker2s - kafkarebalances @@ -49,7 +48,6 @@ rules: - kafkanodepools/status - kafkaconnects/status - kafkaconnectors/status - - kafkamirrormakers/status - kafkabridges/status - kafkamirrormaker2s/status - kafkarebalances/status @@ -57,10 +55,23 @@ rules: - get - patch - update +- apiGroups: + - "kafka.strimzi.io" + resources: + # Needed for environments with enabled OwnerReferencesPermissionEnforcement admission controller (e.g. OpenShift) + - kafkas/finalizers + - kafkanodepools/finalizers + - kafkaconnects/finalizers + - kafkaconnectors/finalizers + - kafkabridges/finalizers + - kafkamirrormaker2s/finalizers + - kafkarebalances/finalizers + verbs: + - update - apiGroups: - "core.strimzi.io" resources: - # The cluster operator uses StrimziPodSets to manage the Kafka and ZooKeeper pods + # The cluster operator uses StrimziPodSets to manage the Kafka, Kafka Connect, and Kafka MirrorMaker 2 pods - strimzipodsets verbs: - get @@ -79,6 +90,13 @@ rules: - get - patch - update +- apiGroups: + - "core.strimzi.io" + resources: + # Needed for environments with enabled OwnerReferencesPermissionEnforcement admission controller (e.g. OpenShift) + - strimzipodsets/finalizers + verbs: + - update - apiGroups: - "kafka.strimzi.io" resources: diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-RoleBinding-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-RoleBinding-strimzi-cluster-operator.yaml index 57cac997f..ed68d1aaa 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-RoleBinding-strimzi-cluster-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/023-RoleBinding-strimzi-cluster-operator.yaml @@ -1,39 +1,35 @@ {{- if .Values.rbac.create -}} -{{- $root := . -}} -{{- $watchNamespaces := .Values.watchNamespaces -}} -{{- if $root.Values.watchAnyNamespace }} - {{- $watchNamespaces = list -}} -{{- end }} -{{- range append $watchNamespaces .Release.Namespace }} +{{- $namespaces := include "strimzi.watchNamespacesList" . | fromYamlArray }} +{{- range $i, $ns := $namespaces }} --- apiVersion: rbac.authorization.k8s.io/v1 -{{- if $root.Values.watchAnyNamespace }} +{{- if $.Values.watchAnyNamespace }} kind: ClusterRoleBinding {{- else }} kind: RoleBinding {{- end }} metadata: - {{- if $root.Values.watchAnyNamespace }} + {{- if $.Values.watchAnyNamespace }} name: strimzi-cluster-operator-watched {{- else }} name: strimzi-cluster-operator-watched {{- end }} - {{- if not $root.Values.watchAnyNamespace }} - namespace: {{ . }} + {{- if not $.Values.watchAnyNamespace }} + namespace: {{ $ns }} {{- end }} labels: - app: {{ template "strimzi.name" $root }} - chart: {{ template "strimzi.chart" $root }} + app: {{ template "strimzi.name" $ }} + chart: {{ template "strimzi.chart" $ }} component: role-binding - release: {{ $root.Release.Name }} - heritage: {{ $root.Release.Service }} + release: {{ $.Release.Name }} + heritage: {{ $.Release.Service }} subjects: - kind: ServiceAccount - name: {{ $root.Values.serviceAccount }} - namespace: {{ $root.Release.Namespace }} + name: {{ $.Values.serviceAccount }} + namespace: {{ $.Release.Namespace }} roleRef: kind: ClusterRole name: strimzi-cluster-operator-watched apiGroup: rbac.authorization.k8s.io {{- end }} -{{- end }} +{{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-ClusterRole-strimzi-entity-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-ClusterRole-strimzi-entity-operator.yaml index f47e0023a..6f370424f 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-ClusterRole-strimzi-entity-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-ClusterRole-strimzi-entity-operator.yaml @@ -46,6 +46,14 @@ rules: - get - patch - update +- apiGroups: + - "kafka.strimzi.io" + resources: + # Needed for environments with enabled OwnerReferencesPermissionEnforcement admission controller (e.g. OpenShift) + - kafkatopics/finalizers + - kafkausers/finalizers + verbs: + - update - apiGroups: - "" resources: diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml index f222a4b0a..93bd641d9 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/031-RoleBinding-strimzi-cluster-operator-entity-operator-delegation.yaml @@ -1,37 +1,33 @@ {{- if .Values.rbac.create -}} -{{- $root := . -}} -{{- $watchNamespaces := .Values.watchNamespaces -}} -{{- if $root.Values.watchAnyNamespace }} - {{- $watchNamespaces = list -}} -{{- end }} -{{- range append $watchNamespaces .Release.Namespace }} +{{- $namespaces := include "strimzi.watchNamespacesList" . | fromYamlArray }} +{{- range $i, $ns := $namespaces }} --- apiVersion: rbac.authorization.k8s.io/v1 -{{- if $root.Values.watchAnyNamespace }} +{{- if $.Values.watchAnyNamespace }} kind: ClusterRoleBinding {{- else }} kind: RoleBinding {{- end }} metadata: name: strimzi-cluster-operator-entity-operator-delegation - {{- if not $root.Values.watchAnyNamespace }} - namespace: {{ . }} + {{- if not $.Values.watchAnyNamespace }} + namespace: {{ $ns }} {{- end }} labels: - app: {{ template "strimzi.name" $root }} - chart: {{ template "strimzi.chart" $root }} + app: {{ template "strimzi.name" $ }} + chart: {{ template "strimzi.chart" $ }} component: entity-operator-role-binding - release: {{ $root.Release.Name }} - heritage: {{ $root.Release.Service }} + release: {{ $.Release.Name }} + heritage: {{ $.Release.Service }} # The Entity Operator cluster role must be bound to the cluster operator service account so that it can delegate the cluster role to the Entity Operator. # This must be done to avoid escalating privileges which would be blocked by Kubernetes. subjects: - kind: ServiceAccount - name: {{ $root.Values.serviceAccount }} - namespace: {{ $root.Release.Namespace }} + name: {{ $.Values.serviceAccount }} + namespace: {{ $.Release.Namespace }} roleRef: kind: ClusterRole name: strimzi-entity-operator apiGroup: rbac.authorization.k8s.io {{- end }} -{{- end }} +{{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/050-ConfigMap-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/050-ConfigMap-strimzi-cluster-operator.yaml index 35233a1dc..fa451e149 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/050-ConfigMap-strimzi-cluster-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/050-ConfigMap-strimzi-cluster-operator.yaml @@ -30,10 +30,6 @@ data: logger.kafka.name = org.apache.kafka logger.kafka.level = WARN - # Zookeeper is very verbose even on INFO level -> We set it to WARN by default - logger.zookeepertrustmanager.name = org.apache.zookeeper - logger.zookeepertrustmanager.level = WARN - # Keeps separate level for Netty logging -> to not be changed by the root logger logger.netty.name = io.netty logger.netty.level = INFO diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/051-PodDisruptionBudget-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/051-PodDisruptionBudget-strimzi-cluster-operator.yaml index eb461f333..f5e308d59 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/051-PodDisruptionBudget-strimzi-cluster-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/051-PodDisruptionBudget-strimzi-cluster-operator.yaml @@ -13,4 +13,7 @@ spec: {{- if .Values.podDisruptionBudget.maxUnavailable }} maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable }} {{- end }} -{{- end }} \ No newline at end of file + {{- if (semverCompare ">= 1.30-0" .Capabilities.KubeVersion.Version) }} + unhealthyPodEvictionPolicy: {{ .Values.podDisruptionBudget.unhealthyPodEvictionPolicy }} + {{- end }} +{{- end }} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/052-NetworkPolicy-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/052-NetworkPolicy-strimzi-cluster-operator.yaml new file mode 100644 index 000000000..cee59b0d4 --- /dev/null +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/052-NetworkPolicy-strimzi-cluster-operator.yaml @@ -0,0 +1,21 @@ +{{- if .Values.operatorNetworkPolicy.enabled -}} +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: strimzi-cluster-operator-network-policy +spec: + egress: + {{- with .Values.operatorNetworkPolicy.egress }} + {{- toYaml . | nindent 4 }} + {{- end }} + ingress: + {{- with .Values.operatorNetworkPolicy.ingress }} + {{- toYaml . | nindent 4 }} + {{- end }} + podSelector: + matchLabels: + name: strimzi-cluster-operator + policyTypes: + - Egress + - Ingress +{{- end }} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml index 16e4f0f22..8e5a1676d 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml @@ -1,3 +1,4 @@ +--- apiVersion: apps/v1 kind: Deployment metadata: @@ -9,8 +10,19 @@ metadata: component: deployment release: {{ .Release.Name }} heritage: {{ .Release.Service }} + {{- with .Values.deploymentLabels }} + {{- toYaml . | nindent 4 }} + {{- end }} + {{- with .Values.deploymentAnnotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} spec: replicas: {{ .Values.replicas }} + {{- with .Values.deploymentStrategy }} + strategy: + {{- toYaml . | trim | nindent 4 }} + {{- end }} {{- if .Values.revisionHistoryLimit }} revisionHistoryLimit: {{ .Values.revisionHistoryLimit }} {{- end }} @@ -32,6 +44,7 @@ spec: {{- end }} spec: serviceAccountName: {{ .Values.serviceAccount }} + automountServiceAccountToken: false {{- /* imagePullSecrets is allowed to be for backwards compatibility */ -}} {{- if kindIs "string" .Values.image.imagePullSecrets }} imagePullSecrets: @@ -42,6 +55,12 @@ spec: {{- toYaml . | nindent 8 }} {{- end }} {{- end }} + {{- if kindIs "bool" .Values.hostUsers }} + hostUsers: {{ .Values.hostUsers }} + {{- end }} + {{- if kindIs "bool" .Values.enableServiceLinks }} + enableServiceLinks: {{ .Values.enableServiceLinks }} + {{- end }} {{- with .Values.podSecurityContext }} securityContext: {{ toYaml . | nindent 8 }} {{- end }} @@ -49,6 +68,23 @@ spec: priorityClassName: {{ .Values.priorityClassName }} {{- end }} volumes: + - name: kube-api-access + projected: + defaultMode: 420 + sources: + - serviceAccountToken: + path: token + - configMap: + name: kube-root-ca.crt + items: + - key: ca.crt + path: ca.crt + - downwardAPI: + items: + - path: namespace + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace - name: strimzi-tmp emptyDir: medium: Memory @@ -68,6 +104,9 @@ spec: args: - /opt/strimzi/bin/cluster_operator_run.sh volumeMounts: + - mountPath: /var/run/secrets/kubernetes.io/serviceaccount + name: kube-api-access + readOnly: true - name: strimzi-tmp mountPath: /tmp - name: {{ .Values.logVolume }} @@ -77,10 +116,8 @@ spec: {{- if .Values.watchAnyNamespace }} value: "*" {{- else }} - {{- if .Values.watchNamespaces -}} - {{- $ns := .Values.watchNamespaces -}} - {{- $ns := append $ns .Release.Namespace }} - value: "{{ join "," $ns }}" + {{- if .Values.watchNamespaces }} + value: "{{ include "strimzi.watchNamespacesList" . | fromYamlArray | join "," }}" {{- else }} valueFrom: fieldRef: @@ -88,9 +125,9 @@ spec: {{- end }} {{- end }} - name: STRIMZI_FULL_RECONCILIATION_INTERVAL_MS - value: {{ .Values.fullReconciliationIntervalMs | quote }} + value: {{ .Values.fullReconciliationIntervalMs | int64 | quote }} - name: STRIMZI_OPERATION_TIMEOUT_MS - value: {{ .Values.operationTimeoutMs | quote }} + value: {{ .Values.operationTimeoutMs | int64 | quote }} {{- template "strimzi.kafka.image.map" . }} - name: STRIMZI_DEFAULT_TOPIC_OPERATOR_IMAGE value: {{ template "strimzi.image" (set . "key" "topicOperator") }} @@ -102,6 +139,8 @@ spec: value: {{ template "strimzi.image" (set . "key" "kafkaBridge") }} - name: STRIMZI_DEFAULT_KANIKO_EXECUTOR_IMAGE value: {{ template "strimzi.image" (set . "key" "kanikoExecutor") }} + - name: STRIMZI_DEFAULT_BUILDAH_IMAGE + value: {{ template "strimzi.image" (set . "key" "buildah") }} - name: STRIMZI_DEFAULT_MAVEN_BUILDER value: {{ template "strimzi.image" (set . "key" "mavenBuilder") }} - name: STRIMZI_OPERATOR_NAMESPACE @@ -124,7 +163,8 @@ spec: - name: STRIMZI_IMAGE_PULL_POLICY value: {{ .Values.image.imagePullPolicy }} {{- end }} - {{ if ne .Values.kubernetesServiceDnsDomain "cluster.local" }}- name: KUBERNETES_SERVICE_DNS_DOMAIN + {{ if ne .Values.kubernetesServiceDnsDomain "cluster.local" }} + - name: KUBERNETES_SERVICE_DNS_DOMAIN value: {{ .Values.kubernetesServiceDnsDomain | quote }}{{ end }} - name: STRIMZI_FEATURE_GATES value: {{ .Values.featureGates | quote }} @@ -138,7 +178,7 @@ spec: {{- end }} {{- if ne (int .Values.connectBuildTimeoutMs) 300000 }} - name: STRIMZI_CONNECT_BUILD_TIMEOUT_MS - value: {{ .Values.connectBuildTimeoutMs | quote }} + value: {{ .Values.connectBuildTimeoutMs | int64 | quote }} {{- end }} {{- if ne .Values.generatePodDisruptionBudget true}} - name: STRIMZI_POD_DISRUPTION_BUDGET_GENERATION @@ -192,3 +232,7 @@ spec: tolerations: {{- toYaml . | nindent 8 }} {{- end }} + {{- with .Values.topologySpreadConstraints }} + topologySpreadConstraints: + {{- toYaml . | nindent 8 }} + {{- end }} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/070-ClusterRole-strimzi-admin.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/070-ClusterRole-strimzi-admin.yaml index 32e48d8c6..c86547197 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/070-ClusterRole-strimzi-admin.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/070-ClusterRole-strimzi-admin.yaml @@ -21,8 +21,6 @@ rules: - kafkanodepools/scale - kafkaconnects - kafkaconnects/scale - - kafkamirrormakers - - kafkamirrormakers/scale - kafkausers - kafkatopics - kafkabridges diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/080-ClusterRole-strimzi-view.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/080-ClusterRole-strimzi-view.yaml index b29a20ee6..7f850f807 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/080-ClusterRole-strimzi-view.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/080-ClusterRole-strimzi-view.yaml @@ -18,7 +18,6 @@ rules: - kafkas - kafkanodepools - kafkaconnects - - kafkamirrormakers - kafkausers - kafkatopics - kafkabridges diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_helpers.tpl b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_helpers.tpl index c23d603fb..e67e89c0c 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_helpers.tpl +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_helpers.tpl @@ -61,3 +61,12 @@ To use, add the following key/value pairs to the scope: {{- end -}} {{- join "," $pluckedList -}} {{- end -}} + +{{/* + Create a list of comma-separated namespaces the operators should watch. +*/}} +{{- define "strimzi.watchNamespacesList" -}} +{{- $namespacesList := .Values.watchNamespaces | default (list) -}} +{{- $returnList := append $namespacesList .Release.Namespace | sortAlpha | uniq -}} +{{- toYaml $returnList -}} +{{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_kafka_image_map.tpl b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_kafka_image_map.tpl index 9bd1e9d54..599dab86a 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_kafka_image_map.tpl +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/templates/_kafka_image_map.tpl @@ -6,31 +6,25 @@ {{/* Generate the kafka image map */}} {{- define "strimzi.kafka.image.map" }} - name: STRIMZI_DEFAULT_KAFKA_EXPORTER_IMAGE - value: {{ template "strimzi.image" (merge . (dict "key" "kafkaExporter" "tagSuffix" "-kafka-3.9.1")) }} + value: {{ template "strimzi.image" (merge . (dict "key" "kafkaExporter" "tagSuffix" "-kafka-4.3.1")) }} - name: STRIMZI_DEFAULT_CRUISE_CONTROL_IMAGE - value: {{ template "strimzi.image" (merge . (dict "key" "cruiseControl" "tagSuffix" "-kafka-3.9.1")) }} + value: {{ template "strimzi.image" (merge . (dict "key" "cruiseControl" "tagSuffix" "-kafka-4.3.1")) }} - name: STRIMZI_KAFKA_IMAGES value: | - 3.8.0={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-3.8.0")) }} - 3.8.1={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-3.8.1")) }} - 3.9.0={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-3.9.0")) }} - 3.9.1={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-3.9.1")) }} + 4.2.0={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-4.2.0")) }} + 4.2.1={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-4.2.1")) }} + 4.3.0={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-4.3.0")) }} + 4.3.1={{ template "strimzi.image" (merge . (dict "key" "kafka" "tagSuffix" "-kafka-4.3.1")) }} - name: STRIMZI_KAFKA_CONNECT_IMAGES value: | - 3.8.0={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-3.8.0")) }} - 3.8.1={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-3.8.1")) }} - 3.9.0={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-3.9.0")) }} - 3.9.1={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-3.9.1")) }} - - name: STRIMZI_KAFKA_MIRROR_MAKER_IMAGES - value: | - 3.8.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker" "tagSuffix" "-kafka-3.8.0")) }} - 3.8.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker" "tagSuffix" "-kafka-3.8.1")) }} - 3.9.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker" "tagSuffix" "-kafka-3.9.0")) }} - 3.9.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker" "tagSuffix" "-kafka-3.9.1")) }} + 4.2.0={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-4.2.0")) }} + 4.2.1={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-4.2.1")) }} + 4.3.0={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-4.3.0")) }} + 4.3.1={{ template "strimzi.image" (merge . (dict "key" "kafkaConnect" "tagSuffix" "-kafka-4.3.1")) }} - name: STRIMZI_KAFKA_MIRROR_MAKER_2_IMAGES value: | - 3.8.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-3.8.0")) }} - 3.8.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-3.8.1")) }} - 3.9.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-3.9.0")) }} - 3.9.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-3.9.1")) }} + 4.2.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-4.2.0")) }} + 4.2.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-4.2.1")) }} + 4.3.0={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-4.3.0")) }} + 4.3.1={{ template "strimzi.image" (merge . (dict "key" "kafkaMirrorMaker2" "tagSuffix" "-kafka-4.3.1")) }} {{- end -}} diff --git a/packages/system/kafka-operator/charts/strimzi-kafka-operator/values.yaml b/packages/system/kafka-operator/charts/strimzi-kafka-operator/values.yaml index 0c3fef280..c6009aaa3 100644 --- a/packages/system/kafka-operator/charts/strimzi-kafka-operator/values.yaml +++ b/packages/system/kafka-operator/charts/strimzi-kafka-operator/values.yaml @@ -3,14 +3,13 @@ # Default replicas for the cluster operator replicas: 1 -# If you set `watchNamespaces` to the same value as ``.Release.Namespace` (e.g. `helm ... --namespace $NAMESPACE`), -# the chart will fail because duplicate RoleBindings will be attempted to be created in the same namespace +# Contains `.Release.Namespace` by default watchNamespaces: [] watchAnyNamespace: false defaultImageRegistry: quay.io defaultImageRepository: strimzi -defaultImageTag: 0.45.1-rc1 +defaultImageTag: 1.2.0 image: registry: "" @@ -37,14 +36,30 @@ tmpDirSizeLimit: 1Mi extraEnvs: [] tolerations: [] +topologySpreadConstraints: [] affinity: {} annotations: {} labels: {} nodeSelector: {} +deploymentAnnotations: {} +deploymentLabels: {} +deploymentStrategy: {} priorityClassName: "" +hostUsers: NULL +enableServiceLinks: NULL podSecurityContext: {} -securityContext: {} +securityContext: + # Specifies container security context. Defaults to a security context that matches the Restricted Kubernetes + # Pod Security Standard (https://kubernetes.io/docs/concepts/security/pod-security-standards/). + allowPrivilegeEscalation: false + runAsNonRoot: true + readOnlyRootFilesystem: true + capabilities: + drop: + - ALL + seccompProfile: + type: RuntimeDefault rbac: create: yes serviceAccountCreate: yes @@ -56,7 +71,10 @@ leaderElection: # https://kubernetes.io/docs/tasks/run-application/configure-pdb/ podDisruptionBudget: enabled: false - # The PDB definition only has two attributes to control the availability requirements: minAvailable or maxUnavailable (mutually exclusive). + # The PDB definition three attributes to control the availability requirements: + # minAvailable or maxUnavailable (mutually exclusive). + # unhealthyPodEvictionPolicy + # # Field maxUnavailable tells how many pods can be down and minAvailable tells how many pods must be running in a cluster. # The pdb template will check values according to below order @@ -71,6 +89,29 @@ podDisruptionBudget: # If both values are set, the template will use the first one and ignore the second one. currently by default minAvailable is set to 1 minAvailable: 1 maxUnavailable: + unhealthyPodEvictionPolicy: IfHealthyBudget + +operatorNetworkPolicy: + # If enabled, this will generate a networkPolicy for the strimzi-operator. + # This flag DOES NOT control operator generated networkPolicies! + # This flag ONLY controls access for the strimzi-operator deployment. + enabled: false + # For ingress we permit access to the metrics endpoint + ingress: + - ports: + - protocol: TCP + port: http + # The operator must communicate with DNS, the Kubernetes API server, + # and all managed Kafka components. + # + # The operands use numerous ports that differ per component and may exist in + # "arbitrary" namespaces. This makes a generic egress NetworkPolicy + # impractical to define. Thus we set `{}` an unrestrictred value. + # + # Any meaningful network restrictions must be defined per cluster, + # with a full knowledge of the Kafka topology and port configuration. + egress: + - {} # If you are using the grafana dashboard sidecar, # you can import some default dashboards here @@ -82,9 +123,8 @@ dashboards: annotations: {} extraLabels: {} -# Docker images that operator uses to provision various components of Strimzi. To use your own registry prefix the -# repository name with your registry URL. -# Ex) repository: registry.xyzcorp.com/strimzi/kafka +# Docker images that operator uses to provision various components of Strimzi. +# To use your own registry or repository, define them for each component below or generally via defaultImageRegistry and defaultImageRepository above kafka: image: registry: "" @@ -115,18 +155,12 @@ kafkaInit: repository: "" name: operator tag: "" -kafkaMirrorMaker: - image: - registry: "" - repository: "" - name: kafka - tagPrefix: "" kafkaBridge: image: registry: "" repository: name: kafka-bridge - tag: 0.31.2 + tag: 1.1.0 kafkaExporter: image: registry: "" @@ -151,6 +185,12 @@ kanikoExecutor: repository: "" name: kaniko-executor tag: "" +buildah: + image: + registry: "" + repository: "" + name: buildah + tag: "" mavenBuilder: image: registry: "" @@ -182,4 +222,4 @@ generateNetworkPolicy: true # Override the value for Connect build timeout connectBuildTimeoutMs: 300000 # Controls whether Strimzi generates pod disruption budget resources (By default true) -generatePodDisruptionBudget: true \ No newline at end of file +generatePodDisruptionBudget: true From a909b733ead2fc646b5816ea91aa468978da3174 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 13 Oct 2026 07:28:46 +0300 Subject: [PATCH 5/9] chore(kafka, kafka-operator): drop the JVM sizing workaround The workaround passed MaxRAM and ActiveProcessorCount to every Strimzi JVM because the JDK 17 in the 0.45 images could not see the container limits on kernels without CONFIG_MEMCG_V1 (JDK-8347811). The Strimzi 1.2.0 images ship JDK 21.0.12, which carries the fix. On such a kernel, with a 1Gi limit, Strimzi's heap probe in the Kafka image now estimates a 495M maximum heap, where the 0.45.1-rc1 image estimated 13.9G. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- packages/apps/kafka/templates/_jvm.tpl | 20 ----- packages/apps/kafka/templates/kafka.yaml | 11 +-- .../apps/kafka/templates/migration-hook.yaml | 11 --- .../apps/kafka/tests/jvm_sizing_test.yaml | 83 ------------------- packages/system/kafka-operator/Makefile | 3 - .../kafka-operator/tests/jvm_sizing_test.yaml | 31 ------- packages/system/kafka-operator/values.yaml | 16 ---- 7 files changed, 1 insertion(+), 174 deletions(-) delete mode 100644 packages/apps/kafka/templates/_jvm.tpl delete mode 100644 packages/apps/kafka/tests/jvm_sizing_test.yaml delete mode 100644 packages/system/kafka-operator/tests/jvm_sizing_test.yaml diff --git a/packages/apps/kafka/templates/_jvm.tpl b/packages/apps/kafka/templates/_jvm.tpl deleted file mode 100644 index 6b541ef63..000000000 --- a/packages/apps/kafka/templates/_jvm.tpl +++ /dev/null @@ -1,20 +0,0 @@ -{{- /* - JAVA_TOOL_OPTIONS for a Strimzi-managed container, from its resource limits. - The JDK 17 in the Strimzi images cannot see the container limits on kernels - built without CONFIG_MEMCG_V1 (JDK-8347811) and sizes itself from the host: - Strimzi's heap probe then hands every broker up to a 5Gi heap. Every java in - the container reads this variable, the probe included, so Strimzi's own - heap rules work again. jvmOptions -XX would not do: Strimzi moves those into - KAFKA_JVM_PERFORMANCE_OPTS, and kafka-run-class.sh drops its G1 tuning when - that variable is set. ActiveProcessorCount only when the chart sets a CPU - limit: a default a LimitRange adds at admission is not visible here, and - Strimzi template env takes no resourceFieldRef, so such a JVM still counts - the host CPUs. -*/ -}} -{{- define "kafka.javaToolOptions" -}} -{{- $opts := list (printf "-XX:MaxRAM=%d" (include "cozy-lib.resources.toFloat" .memory | float64 | int64)) -}} -{{- with .cpu -}} -{{- $opts = append $opts (printf "-XX:ActiveProcessorCount=%d" (include "cozy-lib.resources.toFloat" . | float64 | ceil | int64)) -}} -{{- end -}} -{{- join " " $opts -}} -{{- end -}} diff --git a/packages/apps/kafka/templates/kafka.yaml b/packages/apps/kafka/templates/kafka.yaml index e1b5dc0ed..b7039ace1 100644 --- a/packages/apps/kafka/templates/kafka.yaml +++ b/packages/apps/kafka/templates/kafka.yaml @@ -80,8 +80,7 @@ spec: # The topic-operator and user-operator are JVM processes and OOMKill at # that default, leaving the entity-operator in CrashLoopBackOff so # KafkaTopics/KafkaUsers never reconcile. Set explicit resources so they - # don't inherit the namespace default. JAVA_TOOL_OPTIONS: see - # kafka.javaToolOptions. + # don't inherit the namespace default. topicOperator: resources: requests: @@ -107,11 +106,3 @@ spec: labels: policy.cozystack.io/allow-to-apiserver: "true" enableServiceLinks: false - topicOperatorContainer: - env: - - name: JAVA_TOOL_OPTIONS - value: {{ include "kafka.javaToolOptions" (dict "memory" $entityOperatorMemory) | quote }} - userOperatorContainer: - env: - - name: JAVA_TOOL_OPTIONS - value: {{ include "kafka.javaToolOptions" (dict "memory" $entityOperatorMemory) | quote }} diff --git a/packages/apps/kafka/templates/migration-hook.yaml b/packages/apps/kafka/templates/migration-hook.yaml index 646a6ae1d..5c9e95032 100644 --- a/packages/apps/kafka/templates/migration-hook.yaml +++ b/packages/apps/kafka/templates/migration-hook.yaml @@ -107,12 +107,6 @@ spec: roles: - broker resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | nindent 18 }} - # JAVA_TOOL_OPTIONS: see kafka.javaToolOptions. - template: - kafkaContainer: - env: - - name: JAVA_TOOL_OPTIONS - value: {{ include "kafka.javaToolOptions" (include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | fromYaml).limits | quote }} storage: type: jbod volumes: @@ -152,11 +146,6 @@ spec: roles: - controller resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | nindent 18 }} - template: - kafkaContainer: - env: - - name: JAVA_TOOL_OPTIONS - value: {{ include "kafka.javaToolOptions" (include "cozy-lib.resources.defaultingSanitize" (list .Values.kafka.resourcesPreset .Values.kafka.resources $) | fromYaml).limits | quote }} storage: type: jbod volumes: diff --git a/packages/apps/kafka/tests/jvm_sizing_test.yaml b/packages/apps/kafka/tests/jvm_sizing_test.yaml deleted file mode 100644 index 223131814..000000000 --- a/packages/apps/kafka/tests/jvm_sizing_test.yaml +++ /dev/null @@ -1,83 +0,0 @@ -suite: Kafka JVMs are sized from their container limits - -templates: - - templates/migration-hook.yaml - - templates/kafka.yaml - -release: - name: test-kafka - namespace: tenant-test - -tests: - - it: passes the default preset limits to every node pool JVM - template: templates/migration-hook.yaml - documentIndex: 3 - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "release-namespace: \\$\\{ns\\}\\s+spec:\\s+replicas: \\d+\\s+roles:\\s+- broker\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=1073741824 -XX:ActiveProcessorCount=1\"\\s+storage:" - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "roles:\\s+- controller\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=1073741824 -XX:ActiveProcessorCount=1\"\\s+storage:" - - # Strimzi moves jvmOptions -XX into KAFKA_JVM_PERFORMANCE_OPTS, and - # kafka-run-class.sh then drops its G1 settings. - - it: leaves the node pool JVM options to Strimzi and Kafka - template: templates/migration-hook.yaml - documentSelector: - path: kind - value: Job - asserts: - - notMatchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "jvmOptions" - - - it: rounds a fractional CPU limit up - template: templates/migration-hook.yaml - documentIndex: 3 - set: - kafka.resources: - cpu: 2500m - memory: 3Gi - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "roles:\\s+- controller\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=3221225472 -XX:ActiveProcessorCount=3\"\\s+storage:" - - - it: takes the limits, not the requests, when the allocation ratios shrink the requests - template: templates/migration-hook.yaml - documentIndex: 3 - set: - _cluster: - cpu-allocation-ratio: "4" - memory-allocation-ratio: "2" - kafka.resources: - cpu: "2" - memory: 2Gi - asserts: - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "release-namespace: \\$\\{ns\\}\\s+spec:\\s+replicas: \\d+\\s+roles:\\s+- broker\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=2147483648 -XX:ActiveProcessorCount=2\"\\s+storage:" - - matchRegex: - path: spec.template.spec.containers[0].args[0] - pattern: "roles:\\s+- controller\\s+resources:\\s+limits:(\\s+\\S+: \\S+)+\\s+requests:(\\s+\\S+: \\S+)+(\\s+#[^\\n]*)*\\s+template:\\s+kafkaContainer:\\s+env:\\s+- name: JAVA_TOOL_OPTIONS\\s+value: \"-XX:MaxRAM=2147483648 -XX:ActiveProcessorCount=2\"\\s+storage:" - - - it: passes the entity operator memory limit to its JVMs - template: templates/kafka.yaml - asserts: - - equal: - path: spec.entityOperator.topicOperator.resources.limits.memory - value: 512Mi - - equal: - path: spec.entityOperator.template.topicOperatorContainer.env - value: - - name: JAVA_TOOL_OPTIONS - value: -XX:MaxRAM=536870912 - - equal: - path: spec.entityOperator.userOperator.resources.limits.memory - value: 512Mi - - equal: - path: spec.entityOperator.template.userOperatorContainer.env - value: - - name: JAVA_TOOL_OPTIONS - value: -XX:MaxRAM=536870912 diff --git a/packages/system/kafka-operator/Makefile b/packages/system/kafka-operator/Makefile index fb8122538..592b653a3 100644 --- a/packages/system/kafka-operator/Makefile +++ b/packages/system/kafka-operator/Makefile @@ -3,9 +3,6 @@ export NAMESPACE=cozy-$(NAME) include ../../../hack/package.mk -test: - helm unittest . - update: rm -rf charts helm pull oci://quay.io/strimzi-helm/strimzi-kafka-operator --untar --untardir charts --version 1.2.0 diff --git a/packages/system/kafka-operator/tests/jvm_sizing_test.yaml b/packages/system/kafka-operator/tests/jvm_sizing_test.yaml deleted file mode 100644 index e72a9a657..000000000 --- a/packages/system/kafka-operator/tests/jvm_sizing_test.yaml +++ /dev/null @@ -1,31 +0,0 @@ -suite: cluster operator JVM is sized from the container limits -release: - name: kafka-operator - namespace: cozy-kafka-operator -templates: - - charts/strimzi-kafka-operator/templates/060-Deployment-strimzi-cluster-operator.yaml -tests: - - it: passes the container limits to the JVM, which cannot read them on kernels without cgroup v1 controllers - asserts: - # $(VAR) expands only from entries earlier in the list. - - equal: - path: spec.template.spec.containers[0].env[19] - value: - name: CONTAINER_MEMORY_LIMIT - valueFrom: - resourceFieldRef: - resource: limits.memory - - equal: - path: spec.template.spec.containers[0].env[20] - value: - name: CONTAINER_CPU_LIMIT - valueFrom: - resourceFieldRef: - resource: limits.cpu - - equal: - path: spec.template.spec.containers[0].env[21] - value: - name: JAVA_OPTS - value: -XX:MaxRAM=$(CONTAINER_MEMORY_LIMIT) -XX:ActiveProcessorCount=$(CONTAINER_CPU_LIMIT) - - exists: - path: spec.template.spec.containers[0].resources.limits.cpu diff --git a/packages/system/kafka-operator/values.yaml b/packages/system/kafka-operator/values.yaml index 8fb5509ad..ddfa2c9b8 100644 --- a/packages/system/kafka-operator/values.yaml +++ b/packages/system/kafka-operator/values.yaml @@ -5,19 +5,3 @@ strimzi-kafka-operator: resources: limits: memory: 512Mi - # The JDK 17 in the operator image decides whether to read the container - # limits by looking at /proc/cgroups, which loses its memory line on kernels - # built without CONFIG_MEMCG_V1 (6.12+). The JVM then sizes its heap from - # host RAM and the container is OOMKilled. Drop this once the image ships a - # JDK with JDK-8347811 (17.0.20, 21.0.10). - extraEnvs: - - name: CONTAINER_MEMORY_LIMIT - valueFrom: - resourceFieldRef: - resource: limits.memory - - name: CONTAINER_CPU_LIMIT - valueFrom: - resourceFieldRef: - resource: limits.cpu - - name: JAVA_OPTS - value: -XX:MaxRAM=$(CONTAINER_MEMORY_LIMIT) -XX:ActiveProcessorCount=$(CONTAINER_CPU_LIMIT) From f293d64e847e87f71be0f20ca56d22648c3c6231 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Wed, 14 Oct 2026 00:57:53 +0300 Subject: [PATCH 6/9] fix(kafka): delete the release's StrimziPodSets on uninstall On a cluster upgraded from Strimzi 0.x, deleting a Kafka application leaves its brokers running. kube-controller-manager's garbage collector first saw the StrimziPodSets under v1beta2 and never updates that record, so once v1beta2 is no longer served it retries them forever instead of deleting them, until kube-controller-manager restarts (kubernetes/kubernetes#102641). The pre-delete hook now deletes the release's StrimziPodSets after its node pools, so uninstall no longer depends on garbage collection for the brokers. An absent CRD is not a failure; any other error fails the hook like the pool delete does, because Helm never runs it for the release again. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- hack/kafka-pre-delete-hook.bats | 110 +++++++++++++++++- .../apps/kafka/templates/hooks/delete.yaml | 26 +++++ packages/apps/kafka/tests/delete_test.yaml | 30 ++++- 3 files changed, 162 insertions(+), 4 deletions(-) diff --git a/hack/kafka-pre-delete-hook.bats b/hack/kafka-pre-delete-hook.bats index e8af58aec..90e852670 100644 --- a/hack/kafka-pre-delete-hook.bats +++ b/hack/kafka-pre-delete-hook.bats @@ -32,7 +32,11 @@ # the hook skips keeps its brokers running after the release is gone; # * a KafkaNodePool CRD that is absent exits 0, and any other failure of # that delete exits non-zero, so the uninstall is retried rather than -# finishing with the pools still in place. +# finishing with the pools still in place; +# * the release's StrimziPodSets are deleted after the pools, with the same +# exit codes. Garbage collection does not take them on a cluster upgraded +# from Strimzi 0.x until kube-controller-manager restarts +# (kubernetes/kubernetes#102641), so the hook is what stops the brokers. # # Run from the repo root with make bats-unit-tests. Direct status checks # retain compatibility with cozytest, which has no Bats run/setup helpers. @@ -65,7 +69,8 @@ render_hook() { # The KafkaNodePool delete answers from POOL_OUT / POOL_RC instead, so a test # can fail one call without failing the other. POOL_RC defaults to a clean # delete, because the topic-path tests above are about the topic call; every -# pool-path test sets it explicitly. +# pool-path test sets it explicitly. The StrimziPodSet delete answers from +# SPS_OUT / SPS_RC the same way. # No line below is a bare column-0 `}`, so cozytest.sh's parser leaves it alone. write_fake_kubectl() { cat > "$1/kubectl" <<'KEOF' @@ -75,6 +80,9 @@ case "$*" in *kafkanodepools*) printf '%s\n' "${POOL_OUT:-kafkanodepool.kafka.strimzi.io \"b-0\" deleted}" exit "${POOL_RC:-0}" ;; + *strimzipodsets*) + printf '%s\n' "${SPS_OUT:-strimzipodset.core.strimzi.io \"kafka-test-b-0\" deleted}" + exit "${SPS_RC:-0}" ;; esac if [ -z "${DELETE_RC:-}" ]; then echo "fake kubectl: DELETE_RC unset; the test would assert nothing" >&2 @@ -284,6 +292,11 @@ KEOF cat "$tmp/calls" false fi + if ! grep -q -- "delete strimzipodsets.core.strimzi.io -l strimzi.io/cluster=kafka-test " "$tmp/calls"; then + echo "FAIL: the StrimziPodSets were skipped after the topic wait timed out" + cat "$tmp/calls" + false + fi rm -rf "$tmp" } @@ -342,3 +355,96 @@ KEOF fi rm -rf "$tmp" } + +@test "kafka pre-delete hook deletes this release's StrimziPodSets after its node pools" { + tmp=$(mktemp -d) + write_fake_kubectl "$tmp" + render_hook "$KAFKA_CHART" kafka-test tenant-test kafka-test-pre-delete "$tmp/hook.sh" + + export KLOG="$tmp/calls" + export DELETE_RC=0 + export POOL_RC=0 + export SPS_RC=0 + rc=0 + PATH="$tmp:$PATH" sh "$tmp/hook.sh" > "$tmp/out" 2> "$tmp/err" || rc=$? + if [ "$rc" -ne 0 ]; then + echo "FAIL: hook exited $rc on a clean delete" + cat "$tmp/out" "$tmp/err" + false + fi + # Anchored like the selectors above. + sps_line=$(grep -n -- "delete strimzipodsets.core.strimzi.io -l strimzi.io/cluster=kafka-test " "$tmp/calls" | cut -d: -f1) + pool_line=$(grep -n -- "delete kafkanodepools.kafka.strimzi.io -l strimzi.io/cluster=kafka-test " "$tmp/calls" | cut -d: -f1) + if [ -z "$sps_line" ] || [ -z "$pool_line" ]; then + echo "FAIL: expected one release-scoped delete of each kind" + cat "$tmp/calls" + false + fi + # Deleting the sets while their pools exist lets the operator recreate them. + if [ "$sps_line" -le "$pool_line" ]; then + echo "FAIL: the StrimziPodSets were deleted before the node pools" + cat "$tmp/calls" + false + fi + if ! grep -q -- "strimzipodsets.* --wait=false" "$tmp/calls"; then + echo "FAIL: the StrimziPodSet delete waits, which spends the Job's deadline" + cat "$tmp/calls" + false + fi + rm -rf "$tmp" +} + +@test "kafka pre-delete hook exits 0 when the StrimziPodSet CRD is absent" { + for msg in 'error: the server doesn'"'"'t have a resource type "strimzipodsets"' \ + 'error: no matches for kind "StrimziPodSet" in version "core.strimzi.io/v1"'; do + tmp=$(mktemp -d) + write_fake_kubectl "$tmp" + render_hook "$KAFKA_CHART" kafka-test tenant-test kafka-test-pre-delete "$tmp/hook.sh" + + export KLOG="$tmp/calls" + export DELETE_RC=0 + export POOL_RC=0 + export SPS_RC=1 + export SPS_OUT="$msg" + rc=0 + PATH="$tmp:$PATH" sh "$tmp/hook.sh" > "$tmp/out" 2> "$tmp/err" || rc=$? + if [ "$rc" -ne 0 ]; then + echo "FAIL: hook exited $rc on an absent StrimziPodSet CRD, wedging the uninstall" + cat "$tmp/out" "$tmp/err" + false + fi + if ! grep -q 'StrimziPodSet CRD absent' "$tmp/out"; then + echo "FAIL: the absent CRD was not reported as such" + cat "$tmp/out" + false + fi + rm -rf "$tmp" + done +} + +@test "kafka pre-delete hook exits non-zero when the StrimziPodSet delete fails" { + # A set left here keeps its brokers running after the release is gone, and + # Helm never runs this hook for the release again. + tmp=$(mktemp -d) + write_fake_kubectl "$tmp" + render_hook "$KAFKA_CHART" kafka-test tenant-test kafka-test-pre-delete "$tmp/hook.sh" + + export KLOG="$tmp/calls" + export DELETE_RC=0 + export POOL_RC=0 + export SPS_RC=1 + export SPS_OUT='Error from server (Forbidden): strimzipodsets.core.strimzi.io is forbidden' + rc=0 + PATH="$tmp:$PATH" sh "$tmp/hook.sh" > "$tmp/out" 2> "$tmp/err" || rc=$? + if [ "$rc" -eq 0 ]; then + echo "FAIL: hook exited 0 on a Forbidden, so the uninstall would finish and leave the brokers" + cat "$tmp/out" "$tmp/err" + false + fi + if ! grep -q '^ERROR: deleting StrimziPodSets failed' "$tmp/err"; then + echo "FAIL: the failure was not reported as an error" + cat "$tmp/err" + false + fi + rm -rf "$tmp" +} diff --git a/packages/apps/kafka/templates/hooks/delete.yaml b/packages/apps/kafka/templates/hooks/delete.yaml index e54c5b716..e37778919 100644 --- a/packages/apps/kafka/templates/hooks/delete.yaml +++ b/packages/apps/kafka/templates/hooks/delete.yaml @@ -126,6 +126,29 @@ spec: else printf '%s\n' "${out}" fi + + # Garbage collection should take the StrimziPodSets with their + # pools, but on a cluster upgraded from Strimzi 0.x it does not + # until kube-controller-manager restarts: its collector still + # tracks them under v1beta2, which is no longer served, and + # retries them forever (kubernetes/kubernetes#102641). Deleting + # them here does not depend on that. Fails like the pool call, + # for the same reason. + echo "Deleting StrimziPodSets for {{ .Release.Name }}" + if ! out=$(kubectl -n {{ .Release.Namespace }} delete strimzipodsets.core.strimzi.io \ + -l 'strimzi.io/cluster={{ .Release.Name }}' \ + --ignore-not-found=true \ + --wait=false 2>&1); then + printf '%s\n' "${out}" + case "${out}" in + *"the server doesn't have a resource type"*|*"no matches for kind"*) + echo "StrimziPodSet CRD absent; nothing to delete" ;; + *) + echo "ERROR: deleting StrimziPodSets failed: ${out}" >&2; exit 1 ;; + esac + else + printf '%s\n' "${out}" + fi volumeMounts: - name: tmp mountPath: /tmp @@ -166,6 +189,9 @@ rules: - apiGroups: ["kafka.strimzi.io"] resources: ["kafkanodepools"] verbs: ["list", "delete"] + - apiGroups: ["core.strimzi.io"] + resources: ["strimzipodsets"] + verbs: ["list", "delete"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding diff --git a/packages/apps/kafka/tests/delete_test.yaml b/packages/apps/kafka/tests/delete_test.yaml index d6feced7a..ff09d465f 100644 --- a/packages/apps/kafka/tests/delete_test.yaml +++ b/packages/apps/kafka/tests/delete_test.yaml @@ -154,8 +154,34 @@ tests: documentIndex: 2 - lengthEqual: path: rules - count: 2 - documentIndex: 2 + count: 3 + documentSelector: + path: kind + value: Role + + - it: deletes release-scoped StrimziPodSets after the KafkaNodePools + asserts: + # After an upgrade from Strimzi 0.x, kube-controller-manager's garbage + # collector still tracks each StrimziPodSet under v1beta2, a version no + # longer served, and never deletes it when its pool goes + # (kubernetes/kubernetes#102641). The brokers keep running until + # kube-controller-manager restarts, so the hook deletes the sets itself. + # Same selector and no wait, as for the pools. + - matchRegex: + path: spec.template.spec.containers[0].command[2] + pattern: "(?s)delete kafkanodepools\\.kafka\\.strimzi\\.io .*delete strimzipodsets\\.core\\.strimzi\\.io \\\\\\s+-l 'strimzi\\.io/cluster=kafka-test' \\\\\\s+--ignore-not-found=true \\\\\\s+--wait=false" + documentSelector: + path: kind + value: Job + - contains: + path: rules + content: + apiGroups: ["core.strimzi.io"] + resources: ["strimzipodsets"] + verbs: ["list", "delete"] + documentSelector: + path: kind + value: Role - it: prefixes the hook image with the cluster images registry when set set: From 4c71273ca8cb0058b4f783da40ab502fa08ab1c0 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Thu, 12 Nov 2026 17:46:04 +0300 Subject: [PATCH 7/9] chore(kafka): declare bundled chart dependencies Helm requires bundled subcharts to be declared for chart validation. Keep their source local so validation uses the vendored versions. The application icon remains supplied by the generated dashboard metadata. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- packages/apps/kafka/Chart.yaml | 5 ++++- packages/system/kafka-operator/Chart.yaml | 4 ++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/apps/kafka/Chart.yaml b/packages/apps/kafka/Chart.yaml index 9d88b84fe..f19d3131f 100644 --- a/packages/apps/kafka/Chart.yaml +++ b/packages/apps/kafka/Chart.yaml @@ -1,7 +1,10 @@ apiVersion: v2 name: kafka description: Managed Kafka service -icon: /logos/kafka.svg type: application version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process appVersion: "4.3.1" +dependencies: + - name: cozy-lib + version: "*" + repository: file://charts/cozy-lib diff --git a/packages/system/kafka-operator/Chart.yaml b/packages/system/kafka-operator/Chart.yaml index d669dddc7..3d00fe4af 100644 --- a/packages/system/kafka-operator/Chart.yaml +++ b/packages/system/kafka-operator/Chart.yaml @@ -1,3 +1,7 @@ apiVersion: v2 name: cozy-kafka-operator version: 0.0.0 # Placeholder, the actual version will be automatically set during the build process +dependencies: + - name: strimzi-kafka-operator + version: "*" + repository: file://charts/strimzi-kafka-operator From 0729a6f69162e86bf0431fc3140b8fea3303f454 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Thu, 12 Nov 2026 22:50:17 +0300 Subject: [PATCH 8/9] feat(platform): block the platform upgrade while a Kafka lacks KRaft Only the isp-* variants run the platform migration hook, so on the default variant nothing checked the Kafka clusters before the new kafka-operator chart arrived. helm-controller replaces that chart's CRDs before Helm runs its pre-upgrade hooks, so a guard in the operator chart would come too late. The platform chart now refuses to render the cozystack-packages source while Strimzi still serves v1beta2 and any Kafka lacks KRaft evidence, with the same evidence rule as migration 61. The check stops once only the v1 API is served, so a NotReady tenant Kafka cannot block platform reconciles after the upgrade. Without the migration hook the chart also refuses while any Strimzi CRD still stores v1beta2: the API server would reject the v1-only CRDs of Strimzi 1.2, and the new kafka chart rejects the Kafka 3.x versions that the migration moves. With the hook the check would block the migration that narrows the stored versions in the same upgrade. While v1beta2 is served the chart also refuses while a Kafka application sets a topic config Kafka 4 removed or a segment.bytes below the 1 MiB minimum of Kafka 4. The new kafka chart refuses to render such a topic, which would leave that application on its pre-upgrade spec, and a Kafka 4 broker whose log holds the small segment.bytes does not start. segment.bytes is read the way Kafka reads it: trimmed, with an optional sign, leading zeros decimal. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- packages/core/platform/templates/_helpers.tpl | 145 ++++ .../core/platform/templates/repository.yaml | 1 + .../repository_kafka_kraft_guard_test.yaml | 636 ++++++++++++++++++ 3 files changed, 782 insertions(+) create mode 100644 packages/core/platform/tests/repository_kafka_kraft_guard_test.yaml diff --git a/packages/core/platform/templates/_helpers.tpl b/packages/core/platform/templates/_helpers.tpl index 8c3443a8b..5e92b159c 100644 --- a/packages/core/platform/templates/_helpers.tpl +++ b/packages/core/platform/templates/_helpers.tpl @@ -285,3 +285,148 @@ sliced into /24s ever allowed. {{- define "cozystack.platform.ciliumPoolRange" -}} {{- join ".." (splitList " " (include "cozystack.platform.ciliumPoolBounds" .)) -}} {{- end }} + +{{- /* + Refuse to move cozystack-packages while Strimzi still serves v1beta2 and any + Kafka lacks KRaft evidence. Strimzi 1.2 runs no ZooKeeper, and helm-controller + replaces the kafka-operator CRDs before Helm runs that chart's pre-upgrade + hooks, so this render is the last point before the new operator arrives on a + variant without the migration hook. Once only v1 is served the guard is off: + a NotReady tenant Kafka must not block every platform reconcile. + Without the migration hook, also refuse while any Strimzi CRD still stores a + version other than v1: the API server rejects the v1-only CRDs then, and + the new kafka chart rejects the Kafka 3.x values migration 61 rewrites. + With the hook, migration 61 narrows the stored versions inside this same + upgrade, after this render, so the check would block its own fix. + + The evidence matches migration 61: no ZooKeeper section, a metadata version, + and either the KRaft state, which Strimzi never leaves, or, once the v1 + conversion has deleted that state, Ready=True or the annotation migration 61 + records on every Kafka it accepts. Without the state Ready is required + because Strimzi keeps the previous metadata version on a failed reconcile. +*/ -}} +{{- define "cozystack.platform.kafkaKRaftGuard" -}} +{{- if not .Values.migrations.enabled -}} +{{- /* STRIMZI_CRDS of migration 61. */ -}} +{{- range $name := list "kafkas.kafka.strimzi.io" "kafkanodepools.kafka.strimzi.io" "kafkaconnects.kafka.strimzi.io" "kafkaconnectors.kafka.strimzi.io" "kafkabridges.kafka.strimzi.io" "kafkamirrormaker2s.kafka.strimzi.io" "kafkarebalances.kafka.strimzi.io" "kafkatopics.kafka.strimzi.io" "kafkausers.kafka.strimzi.io" "strimzipodsets.core.strimzi.io" -}} +{{- range (dig "status" "storedVersions" list (lookup "apiextensions.k8s.io/v1" "CustomResourceDefinition" "" $name)) -}} +{{- if ne (printf "%v" .) "v1" -}} +{{- fail (printf "Strimzi still stores %v for %s, and the Strimzi 1.2 CRDs drop it. This platform variant runs no migration hook: run /migrations/61 from this release's platform-migrations image with cluster-admin credentials before upgrading." . $name) -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- $crd := lookup "apiextensions.k8s.io/v1" "CustomResourceDefinition" "" "kafkas.kafka.strimzi.io" -}} +{{- if $crd -}} +{{- $served := list -}} +{{- range (dig "spec" "versions" list $crd) -}} +{{- if and (kindIs "map" .) .served -}} +{{- $served = append $served .name -}} +{{- end -}} +{{- end -}} +{{- if has "v1beta2" $served -}} +{{- $kafkas := lookup "kafka.strimzi.io/v1beta2" "Kafka" "" "" -}} +{{- if not (and (kindIs "map" $kafkas) (kindIs "slice" (get $kafkas "items"))) -}} +{{- fail "Cannot read the Strimzi Kafka list; refusing to upgrade the platform without checking every Kafka for KRaft" -}} +{{- end -}} +{{- $unproven := list -}} +{{- range (get $kafkas "items") -}} +{{- $meta := dig "metadata" dict . -}} +{{- $id := printf "%v/%v" (get $meta "namespace") (get $meta "name") -}} +{{- $release := dig "annotations" "meta.helm.sh/release-name" "" $meta -}} +{{- if $release -}} +{{- $id = printf "%s (Helm release: %v/%v)" $id (get $meta "namespace") $release -}} +{{- end -}} +{{- $status := get . "status" -}} +{{- $reason := "" -}} +{{- if dig "spec" "zookeeper" nil . -}} +{{- $reason = "ZooKeeper section in spec" -}} +{{- else if not (kindIs "map" $status) -}} +{{- $reason = "no status" -}} +{{- else -}} +{{- $state := get $status "kafkaMetadataState" -}} +{{- $hasState := and (hasKey $status "kafkaMetadataState") (not (kindIs "invalid" $state)) -}} +{{- $version := get $status "kafkaMetadataVersion" -}} +{{- if and $hasState (not (and (kindIs "string" $state) (eq $state "KRaft"))) -}} +{{- $reason = printf "metadata state %v" $state -}} +{{- else if not (and (kindIs "string" $version) (ne $version "")) -}} +{{- $reason = "no kafkaMetadataVersion" -}} +{{- else if not $hasState -}} +{{- $ready := false -}} +{{- $conditions := get $status "conditions" -}} +{{- range (kindIs "slice" $conditions | ternary $conditions list) -}} +{{- if and (kindIs "map" .) (eq (printf "%v" .type) "Ready") (eq (printf "%v" .status) "True") -}} +{{- $ready = true -}} +{{- end -}} +{{- end -}} +{{- $verified := dig "annotations" "cozystack.io/kraft-verified" "" $meta -}} +{{- if not (or $ready (and (kindIs "string" $verified) (ne $verified ""))) -}} +{{- $reason = "not Ready" -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- if $reason -}} +{{- $unproven = append $unproven (printf "%s: %s" $id $reason) -}} +{{- end -}} +{{- end -}} +{{- if $unproven -}} +{{- fail (printf "KRaft is not confirmed for these Kafka clusters: %s. Strimzi 1.x cannot run ZooKeeper or an unfinished migration. Upgrade through Cozystack v1.7 and let each Kafka finish its KRaft migration and report Ready. A Kafka created after migration 61 stopped the Strimzi operators has no status until the upgrade finishes: delete that application and create it again afterwards. On platform variants without the migration hook, run /migrations/61 from this release's platform-migrations image before upgrading." (join "; " $unproven)) -}} +{{- end -}} +{{- /* + Kafka 4.2/4.3 reject these topic configs that Kafka 3.x accepted, and the + new kafka chart refuses to render them, which would leave that application + on its pre-upgrade spec; a broker whose log holds a segment.bytes below the + floor does not start. The same limits are in + packages/apps/kafka/templates/topics.yaml, + packages/core/platform/images/migrations/migrations/61 and + packages/system/backupstrategy-controller/files/kafka-backup.sh. + Application values live in the HelmRelease, so this does not depend on the + aggregated apps API being available. +*/ -}} +{{- $segmentBytesMin := 1048576 -}} +{{- $releases := lookup "helm.toolkit.fluxcd.io/v2" "HelmRelease" "" "" -}} +{{- $removed := list -}} +{{- range (and (kindIs "map" $releases) (kindIs "slice" (get $releases "items")) | ternary (get $releases "items") list) -}} +{{- $meta := dig "metadata" dict . -}} +{{- if eq (dig "labels" "apps.cozystack.io/application.kind" "" $meta) "Kafka" -}} +{{- $app := printf "%v/%v" (get $meta "namespace") (dig "labels" "apps.cozystack.io/application.name" (get $meta "name") $meta) -}} +{{- $topics := dig "spec" "values" "topics" list . -}} +{{- range (kindIs "slice" $topics | ternary $topics list) -}} +{{- if kindIs "map" . -}} +{{- $config := get . "config" -}} +{{- $topic := get . "name" -}} +{{- range $key := list "message.downconversion.enable" "message.format.version" "message.timestamp.difference.max.ms" -}} +{{- if and (kindIs "map" $config) (hasKey $config $key) -}} +{{- $removed = append $removed (printf "%s: topic %v sets %s" $app $topic $key) -}} +{{- end -}} +{{- end -}} +{{- $segmentBytes := and (kindIs "map" $config) (get $config "segment.bytes") -}} +{{- if or (kindIs "float64" $segmentBytes) (kindIs "int64" $segmentBytes) (kindIs "int" $segmentBytes) -}} +{{- $segmentBytes = printf "%d" (int64 $segmentBytes) -}} +{{- end -}} +{{- /* + Kafka trims characters up to U+0020 and reads the rest with + Integer.parseInt: a sign is allowed and leading zeros are decimal, where + int64 would read them as octal. The digits are compared as text, so a + string too long for int64 is not misread either. +*/ -}} +{{- $integer := `^[\x00-\x20]*(?:\+|(-))?0*([0-9]+)[\x00-\x20]*$` -}} +{{- if and (kindIs "string" $segmentBytes) (regexMatch $integer $segmentBytes) -}} +{{- $value := regexReplaceAll $integer $segmentBytes "${1}${2}" -}} +{{- $min := toString $segmentBytesMin -}} +{{- if or (hasPrefix "-" $value) (lt (len $value) (len $min)) (and (eq (len $value) (len $min)) (lt $value $min)) -}} +{{- $removed = append $removed (printf "%s: topic %v sets segment.bytes=%s, below the Kafka 4 minimum of %d" $app $topic $value $segmentBytesMin) -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- if $removed -}} +{{- fail (printf "Kafka 4 rejects topic configs these Kafka applications still set: %s. Remove the configs Kafka 4 removed and raise segment.bytes to the minimum in the topics' config before upgrading the platform." (join "; " $removed)) -}} +{{- end -}} +{{- else if not (has "v1" $served) -}} +{{- fail (printf "kafkas.kafka.strimzi.io serves neither v1beta2 nor v1 (served: %v); refusing to upgrade the platform without checking Kafka for KRaft" $served) -}} +{{- end -}} +{{- end -}} +{{- end }} diff --git a/packages/core/platform/templates/repository.yaml b/packages/core/platform/templates/repository.yaml index 555a02f0b..afda66388 100644 --- a/packages/core/platform/templates/repository.yaml +++ b/packages/core/platform/templates/repository.yaml @@ -1,3 +1,4 @@ +{{- include "cozystack.platform.kafkaKRaftGuard" . }} {{- $sourceRef := .Values.sourceRef }} {{- $apiVersion := "source.toolkit.fluxcd.io/v1" }} {{- $kind := $sourceRef.kind }} diff --git a/packages/core/platform/tests/repository_kafka_kraft_guard_test.yaml b/packages/core/platform/tests/repository_kafka_kraft_guard_test.yaml new file mode 100644 index 000000000..8526f169e --- /dev/null +++ b/packages/core/platform/tests/repository_kafka_kraft_guard_test.yaml @@ -0,0 +1,636 @@ +suite: repository refuses to move cozystack-packages while a pre-1.2 Strimzi Kafka lacks KRaft evidence +# The default variant renders no migration hook, and helm-controller applies the +# kafka-operator CRDs before any Helm pre-upgrade hook runs. The root platform +# render is the last point that sees every Kafka before the new chart arrives. +# A pre-1.2 CRD with no Kafka at all is not covered here: the helm-unittest +# fake client registers a list kind only from the objects it is given. The +# empty-list case is covered by the migration 61 tests. +templates: + - templates/repository.yaml +release: + name: cozystack-platform + namespace: cozy-system +scheme: &scheme + "source.toolkit.fluxcd.io/v1/OCIRepository": + gvr: + group: source.toolkit.fluxcd.io + version: v1 + resource: ocirepositories + namespaced: true + "apiextensions.k8s.io/v1/CustomResourceDefinition": + gvr: + group: apiextensions.k8s.io + version: v1 + resource: customresourcedefinitions + namespaced: false + "helm.toolkit.fluxcd.io/v2/HelmRelease": + gvr: + group: helm.toolkit.fluxcd.io + version: v2 + resource: helmreleases + namespaced: true + "kafka.strimzi.io/v1beta2/Kafka": + gvr: + group: kafka.strimzi.io + version: v1beta2 + resource: kafkas + namespaced: true +source: &source + kind: OCIRepository + apiVersion: source.toolkit.fluxcd.io/v1 + metadata: + name: cozystack-platform + namespace: cozy-system + spec: + url: oci://example.test/cozystack-packages +crdPre: &crdPre + kind: CustomResourceDefinition + apiVersion: apiextensions.k8s.io/v1 + metadata: + name: kafkas.kafka.strimzi.io + spec: + versions: + - name: v1 + served: true + storage: true + - name: v1beta2 + served: true + storage: false +crdPost: &crdPost + kind: CustomResourceDefinition + apiVersion: apiextensions.k8s.io/v1 + metadata: + name: kafkas.kafka.strimzi.io + spec: + versions: + - name: v1 + served: true + storage: true +crdPreStoredV1beta2: &crdPreStoredV1beta2 + kind: CustomResourceDefinition + apiVersion: apiextensions.k8s.io/v1 + metadata: + name: kafkas.kafka.strimzi.io + spec: + versions: + - name: v1 + served: true + storage: true + - name: v1beta2 + served: true + storage: false + status: + storedVersions: [v1beta2, v1] +legacyKRaft: &legacyKRaft + kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-legacy + namespace: tenant-a + spec: + kafka: + version: 3.9.0 + status: + kafkaMetadataState: KRaft + kafkaMetadataVersion: 3.9-IV0 + conditions: + - type: Ready + status: "True" +modernKRaft: &modernKRaft + kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-modern + namespace: tenant-b + spec: + kafka: + version: 4.1.1 + status: + kafkaMetadataVersion: 4.1-IV1 + conditions: + - type: Ready + status: "True" +# The test client lists only the kinds it is given an object of. +unrelatedRelease: &unrelatedRelease + kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: unrelated + namespace: tenant-z + spec: {} +kafkaAppRemovedTopicConfig: &kafkaAppRemovedTopicConfig + kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: kafka-events + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Kafka + apps.cozystack.io/application.name: events + spec: + values: + topics: + - name: clicks + - name: orders + config: + retention.ms: "86400000" + message.format.version: 3.0-IV1 +tests: + - it: renders the repository when Strimzi is not installed + kubernetesProvider: + scheme: *scheme + objects: [*source] + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: renders the repository when every Kafka proves KRaft, including modern evidence without legacy state + kubernetesProvider: + scheme: *scheme + objects: [*source, *unrelatedRelease, *crdPre, *legacyKRaft, *modernKRaft] + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + # The kafka-operator release replaces these CRDs with v1-only ones, which the + # API server refuses while v1beta2 is still stored. + - it: fails with migrations disabled while Strimzi still stores v1beta2, even when every Kafka proves KRaft + kubernetesProvider: + scheme: *scheme + objects: [*source, *unrelatedRelease, *crdPreStoredV1beta2, *legacyKRaft, *modernKRaft] + asserts: + - failedTemplate: + errorPattern: "still stores v1beta2" + - failedTemplate: + errorPattern: "/migrations/61" + + # A partly finished conversion can narrow the Kafka CRD and leave another one + # storing v1beta2, which the CRD replace refuses just the same. + - it: fails with migrations disabled while any other Strimzi CRD still stores v1beta2 + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: CustomResourceDefinition + apiVersion: apiextensions.k8s.io/v1 + metadata: + name: kafkatopics.kafka.strimzi.io + spec: + versions: + - name: v1 + served: true + storage: true + status: + storedVersions: [v1beta2, v1] + asserts: + - failedTemplate: + errorPattern: "still stores v1beta2 for kafkatopics.kafka.strimzi.io" + + # Migration 61 runs as this upgrade's pre-upgrade hook and narrows the stored + # versions itself. + - it: renders with migrations enabled while Strimzi still stores v1beta2 + set: + migrations: + enabled: true + kubernetesProvider: + scheme: *scheme + objects: [*source, *unrelatedRelease, *crdPreStoredV1beta2, *legacyKRaft, *modernKRaft] + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: fails with migrations disabled and lists every Kafka without KRaft evidence + set: + migrations: + enabled: false + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-nostatus + namespace: tenant-c + annotations: + meta.helm.sh/release-name: kafka-nostatus + spec: + kafka: + version: 3.9.0 + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-mid + namespace: tenant-d + spec: + kafka: + version: 3.9.0 + status: + kafkaMetadataState: KRaftPostMigration + kafkaMetadataVersion: 3.9-IV0 + conditions: + - type: Ready + status: "True" + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-zk + namespace: tenant-e + spec: + kafka: + version: 3.9.0 + zookeeper: + storage: + type: ephemeral + status: + kafkaMetadataState: KRaft + asserts: + - failedTemplate: + errorPattern: "tenant-c/kafka-nostatus \\(Helm release: tenant-c/kafka-nostatus\\): no status" + - failedTemplate: + errorPattern: "tenant-d/kafka-mid: metadata state KRaftPostMigration" + - failedTemplate: + errorPattern: "tenant-e/kafka-zk: ZooKeeper section" + - failedTemplate: + errorPattern: "created after migration 61 stopped the Strimzi operators" + - failedTemplate: + errorPattern: "/migrations/61" + + - it: fails with migrations enabled on a NotReady Kafka without a KRaft state while v1beta2 is served + set: + migrations: + enabled: true + kubernetesProvider: + scheme: + <<: *scheme + "v1/ConfigMap": + gvr: + version: v1 + resource: configmaps + namespaced: true + objects: + - *source + - *unrelatedRelease + - *crdPre + - kind: ConfigMap + apiVersion: v1 + metadata: + name: cozystack-version + namespace: cozy-system + data: + version: "62" + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-notready + namespace: tenant-f + spec: + kafka: + version: 4.1.1 + status: + kafkaMetadataVersion: 4.1-IV1 + conditions: + - type: NotReady + status: "True" + asserts: + - failedTemplate: + errorPattern: "tenant-f/kafka-notready" + + - it: ignores a NotReady Kafka once only the Strimzi 1.2 v1 API is served + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPost + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-notready + namespace: tenant-f + spec: {} + status: + conditions: + - type: NotReady + status: "True" + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: fails closed on a Kafka CRD that serves neither known version + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - kind: CustomResourceDefinition + apiVersion: apiextensions.k8s.io/v1 + metadata: + name: kafkas.kafka.strimzi.io + spec: + versions: + - name: v1beta1 + served: true + storage: true + asserts: + - failedTemplate: + errorPattern: "kafkas.kafka.strimzi.io serves neither" + + - it: lists a Kafka whose status carries a metadata version but no conditions + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-noconditions + namespace: tenant-g + spec: + kafka: + version: 4.1.1 + status: + kafkaMetadataVersion: 4.1-IV1 + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-emptystatus + namespace: tenant-h + spec: + kafka: + version: 4.1.1 + status: {} + asserts: + - failedTemplate: + errorPattern: "KRaft is not confirmed for these Kafka clusters: .*tenant-g/kafka-noconditions" + - failedTemplate: + errorPattern: "tenant-h/kafka-emptystatus" + + # KRaft is the final Strimzi metadata state, so with a metadata version it + # proves KRaft on its own. The conversion deletes the state; migration 61 + # records the proof in an annotation first, so a retry accepts the same Kafka. + - it: renders when a NotReady Kafka has an explicit KRaft state or the migration's KRaft annotation + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-kraft-notready + namespace: tenant-i + spec: + kafka: + version: 3.9.0 + status: + kafkaMetadataState: KRaft + kafkaMetadataVersion: 3.9-IV0 + conditions: + - type: NotReady + status: "True" + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-converted-notready + namespace: tenant-l + annotations: + cozystack.io/kraft-verified: 3.9-IV0 + spec: + kafka: + version: 3.9.0 + status: + kafkaMetadataVersion: 3.9-IV0 + conditions: + - type: NotReady + status: "True" + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: lists a Kafka without a metadata version, or NotReady without a KRaft state or annotation + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-kraft-noversion + namespace: tenant-j + annotations: + cozystack.io/kraft-verified: 3.9-IV0 + spec: + kafka: + version: 3.9.0 + status: + kafkaMetadataState: KRaft + conditions: + - type: Ready + status: "True" + - kind: Kafka + apiVersion: kafka.strimzi.io/v1beta2 + metadata: + name: kafka-ready-false + namespace: tenant-k + annotations: + cozystack.io/kraft-verified: "" + spec: + kafka: + version: 4.1.1 + status: + kafkaMetadataVersion: 4.1-IV1 + conditions: + - type: Ready + status: "False" + asserts: + - failedTemplate: + errorPattern: "tenant-j/kafka-kraft-noversion: no kafkaMetadataVersion" + - failedTemplate: + errorPattern: "tenant-k/kafka-ready-false: not Ready" + + # Kafka 4 rejects these topic configs, and the kafka chart refuses to render + # them; caught here, they stop the upgrade before any application moves. + - it: fails while v1beta2 is served and a Kafka application sets a topic config Kafka 4 removed + kubernetesProvider: + scheme: *scheme + objects: [*source, *unrelatedRelease, *crdPre, *legacyKRaft, *kafkaAppRemovedTopicConfig] + asserts: + - failedTemplate: + errorPattern: "tenant-a/events: topic orders sets message.format.version" + + - it: fails while a Kafka application sets segment.bytes below the Kafka 4 floor + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: kafka-events + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Kafka + apps.cozystack.io/application.name: events + spec: + values: + topics: + # The fake client deep-copies only JSON number types, so the + # number is written as a float, as decoded JSON arrives. + - name: small + config: + segment.bytes: 524288.0 + - name: floor + config: + segment.bytes: "1048576" + asserts: + - failedTemplate: + errorPattern: "tenant-a/events: topic small sets segment.bytes=524288, below the Kafka 4 minimum of 1048576" + + # Kafka trims the value and reads it with Integer.parseInt, which takes a + # sign and reads leading zeros as decimal; what it cannot parse is left to it. + - it: fails while a Kafka application sets segment.bytes below the floor padded or with a plus sign + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: kafka-events + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Kafka + apps.cozystack.io/application.name: events + spec: + values: + topics: + - name: padded + config: + segment.bytes: " 524288" + - name: plus + config: + segment.bytes: "+100" + - name: zeros + config: + segment.bytes: "02000000" + - name: exp + config: + segment.bytes: "1e3" + - name: negative + config: + segment.bytes: "-2000000" + asserts: + - failedTemplate: + errorPattern: "topic padded sets segment.bytes=524288, below the Kafka 4 minimum of 1048576; tenant-a/events: topic plus sets segment.bytes=100, below the Kafka 4 minimum of 1048576; tenant-a/events: topic negative sets segment.bytes=-2000000, below the Kafka 4 minimum of 1048576\\. Remove" + + - it: renders when no Kafka application sets a removed topic config + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: kafka-events + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Kafka + apps.cozystack.io/application.name: events + spec: + values: + topics: + - name: orders + config: + retention.ms: "86400000" + - kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: other + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Other + spec: + values: + topics: + - name: orders + config: + message.format.version: 3.0-IV1 + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: ignores a removed topic config once only the Strimzi 1.2 v1 API is served + kubernetesProvider: + scheme: *scheme + objects: [*source, *unrelatedRelease, *crdPost, *kafkaAppRemovedTopicConfig] + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system + + - it: skips a topic entry that is not a map instead of failing the render + kubernetesProvider: + scheme: *scheme + objects: + - *source + - *unrelatedRelease + - *crdPre + - *legacyKRaft + - kind: HelmRelease + apiVersion: helm.toolkit.fluxcd.io/v2 + metadata: + name: kafka-events + namespace: tenant-a + labels: + apps.cozystack.io/application.kind: Kafka + apps.cozystack.io/application.name: events + spec: + values: + topics: ["orders"] + asserts: + - containsDocument: + apiVersion: source.toolkit.fluxcd.io/v1 + kind: OCIRepository + name: cozystack-packages + namespace: cozy-system From a48a2f2599474da5c41e718a24654f963b816176 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Fri, 13 Nov 2026 12:53:12 +0300 Subject: [PATCH 9/9] fix(backups): restore Kafka 3.x topic configs that Kafka 4 rejects Every Kafka now runs 4.x, and a topic-metadata backup taken on Kafka 3.8 or 3.9 can record message.format.version, message.downconversion.enable or message.timestamp.difference.max.ms, which Kafka 4 removed, or a segment.bytes below 1 MiB, the minimum Kafka 4 raised it to from 14 bytes. A 4.x broker rejects each of them, so the restore stopped at the first one, leaving that topic half-configured and every later topic unrestored, and a rerun failed on the same line. The restore now skips the removed keys and raises a smaller segment.bytes to the minimum, each with a warning, and applies the rest. segment.bytes is read the way Kafka reads it: trimmed, with an optional sign, leading zeros decimal. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- hack/kafka-backup-script.bats | 89 ++++++++++++++++++- .../files/kafka-backup.sh | 22 +++++ 2 files changed, 110 insertions(+), 1 deletion(-) diff --git a/hack/kafka-backup-script.bats b/hack/kafka-backup-script.bats index dff94c2a9..e0b594ea4 100644 --- a/hack/kafka-backup-script.bats +++ b/hack/kafka-backup-script.bats @@ -106,7 +106,19 @@ if [ "\$mode" = describe ]; then [ -n "\${CFG_FAIL:-}" ] && exit 3 cat "$STATE/cfg.\$name" 2>/dev/null || true; exit 0 fi -if [ "\$mode" = alter ]; then echo "\$name \$add" >> "$STATE/configs_applied"; exit 0; fi +if [ "\$mode" = alter ]; then + # Topic configs a Kafka 4 broker no longer knows. + case "\${add%%=*}" in + message.downconversion.enable|message.format.version|message.timestamp.difference.max.ms) + echo "InvalidConfigurationException: Unknown topic config name: \${add%%=*}" >&2; exit 1 ;; + esac + # Kafka 4 raised the segment.bytes floor to 1 MiB. + case "\$add" in + segment.bytes=*) + [ "\${add#*=}" -ge 1048576 ] || { echo "InvalidConfigurationException: Invalid value \${add#*=} for configuration segment.bytes: Value must be at least 1048576" >&2; exit 1; } ;; + esac + echo "\$name \$add" >> "$STATE/configs_applied"; exit 0 +fi exit 0 STUB @@ -243,6 +255,81 @@ write_backup_object() { grep -qxF "orders cleanup.policy=[compact,delete]" "$STATE/configs_applied" } +@test "restore of a Kafka 3.x backup skips the topic configs Kafka 4 removed and restores the rest" { + init_stubs + { + printf 'T\torders\t3\t1\n' + printf 'C\torders\tmessage.format.version\t3.0-IV1\n' + printf 'C\torders\tmessage.downconversion.enable\ttrue\n' + printf 'C\torders\tmessage.timestamp.difference.max.ms\t60000\n' + printf 'C\torders\tretention.ms\t86400000\n' + printf 'T\tpayments\t2\t1\n' + printf 'C\tpayments\tcleanup.policy\tcompact\n' + } > "$STATE/s3_object" + : > "$STATE/topics" + out=$(MODE=restore S3_ENDPOINT="https://s3.example.org" bash "$SCRIPT" 2>&1) + grep -qxF "orders retention.ms=86400000" "$STATE/configs_applied" + grep -qxF "create payments 2" "$STATE/actions" + grep -qxF "payments cleanup.policy=compact" "$STATE/configs_applied" + for k in message.format.version message.downconversion.enable message.timestamp.difference.max.ms; do + printf '%s\n' "$out" | grep -qF "orders: skipping $k" + done +} + +@test "restore of a Kafka 3.x backup raises segment.bytes below the Kafka 4 floor and restores the rest" { + init_stubs + { + printf 'T\torders\t3\t1\n' + printf 'C\torders\tretention.ms\t86400000\n' + printf 'C\torders\tsegment.bytes\t524288\n' + printf 'T\tpayments\t2\t1\n' + printf 'C\tpayments\tsegment.bytes\t1048576\n' + printf 'T\tpadded\t1\t1\n' + printf 'C\tpadded\tsegment.bytes\t 524288 \n' + printf 'T\tsigned\t1\t1\n' + printf 'C\tsigned\tsegment.bytes\t+524288\n' + printf 'T\tctrl\t1\t1\n' + printf 'C\tctrl\tsegment.bytes\t\013524288\n' + printf 'T\tzeros\t1\t1\n' + printf 'C\tzeros\tsegment.bytes\t0524288\n' + printf 'T\tlarge\t1\t1\n' + printf 'C\tlarge\tsegment.bytes\t02000000\n' + printf 'T\tnegative\t1\t1\n' + printf 'C\tnegative\tsegment.bytes\t-2000000\n' + } > "$STATE/s3_object" + : > "$STATE/topics" + out=$(MODE=restore S3_ENDPOINT="https://s3.example.org" bash "$SCRIPT" 2>&1) + grep -qxF "orders retention.ms=86400000" "$STATE/configs_applied" + grep -qxF "orders segment.bytes=1048576" "$STATE/configs_applied" + grep -qxF "create payments 2" "$STATE/actions" + grep -qxF "payments segment.bytes=1048576" "$STATE/configs_applied" + printf '%s\n' "$out" | grep -qF "orders: raising segment.bytes from 524288 to 1048576" + # Kafka trims the value and reads it with Integer.parseInt, which takes a sign. + grep -qxF "padded segment.bytes=1048576" "$STATE/configs_applied" + grep -qxF "signed segment.bytes=1048576" "$STATE/configs_applied" + # A tab is a field separator in the backup, so a control character pads instead. + grep -qxF "ctrl segment.bytes=1048576" "$STATE/configs_applied" + grep -qxF "zeros segment.bytes=1048576" "$STATE/configs_applied" + grep -qxF "large segment.bytes=02000000" "$STATE/configs_applied" + grep -qxF "negative segment.bytes=1048576" "$STATE/configs_applied" + if printf '%s\n' "$out" | grep -qF "payments: raising"; then echo "warned about a value at the floor" >&2; return 1; fi +} + +# Integer.parseInt rejects it, so the value reaches Kafka as it is and Kafka refuses it. +@test "restore passes a segment.bytes Kafka cannot parse as an integer through unchanged" { + init_stubs + { + printf 'T\texp\t1\t1\n' + printf 'C\texp\tsegment.bytes\t1e3\n' + } > "$STATE/s3_object" + : > "$STATE/topics" + rc=0 + out=$(MODE=restore S3_ENDPOINT="https://s3.example.org" bash "$SCRIPT" 2>&1) || rc=$? + [ "$rc" -ne 0 ] + printf '%s\n' "$out" | grep -qF "Invalid value 1e3 for configuration segment.bytes" + if printf '%s\n' "$out" | grep -qF "raising"; then echo "rewrote a value Kafka cannot parse" >&2; return 1; fi +} + @test "restore fails loudly on a replication-factor mismatch" { init_stubs printf 'T\torders\t3\t2\n' > "$STATE/s3_object" # backup wants RF 2 diff --git a/packages/system/backupstrategy-controller/files/kafka-backup.sh b/packages/system/backupstrategy-controller/files/kafka-backup.sh index 753ed0d2b..21419f0a4 100644 --- a/packages/system/backupstrategy-controller/files/kafka-backup.sh +++ b/packages/system/backupstrategy-controller/files/kafka-backup.sh @@ -5,6 +5,9 @@ set -euo pipefail # kafka-topics.sh / kafka-configs.sh; the Job never sets it. BIN="${BIN:-/opt/kafka/bin}" TAB=$(printf '\t') +# Kafka 4 rejects a smaller topic segment.bytes; Kafka 3.x accepted down to 14. +SEGMENT_BYTES_MIN=1048576 +KAFKA_INTEGER=$'^[\001- ]*([+-]?)0*([0-9]+)[\001- ]*$' # Overridable for the same reason as BIN: hack/kafka-backup-script.bats points it # at a per-test path so runs cannot read back a previous test's records. The Job # never sets it. @@ -197,6 +200,25 @@ else ;; C) t=$a; k=$b; v=$c + # A backup taken on Kafka 3.x can carry topic configs Kafka 4.2/4.3 + # reject, which would stop the restore halfway. The same limits are in + # packages/apps/kafka/templates/topics.yaml, + # packages/core/platform/templates/_helpers.tpl and + # packages/core/platform/images/migrations/migrations/61. + case "${k}" in + message.downconversion.enable|message.format.version|message.timestamp.difference.max.ms) + echo " config ${t}: skipping ${k}, removed in Kafka 4" >&2 + continue + ;; + segment.bytes) + # Kafka trims characters up to U+0020 and reads the rest with + # Integer.parseInt, which allows a sign and leading zeros. + if [[ "${v}" =~ ${KAFKA_INTEGER} ]] && [ "${BASH_REMATCH[1]#+}${BASH_REMATCH[2]}" -lt "${SEGMENT_BYTES_MIN}" ]; then + echo " config ${t}: raising segment.bytes from ${BASH_REMATCH[1]#+}${BASH_REMATCH[2]} to ${SEGMENT_BYTES_MIN}, the Kafka 4 minimum" >&2 + v=${SEGMENT_BYTES_MIN} + fi + ;; + esac # Bracket-group only when the value contains a comma, so kafka-configs # does not split it into bogus key=val tokens. case "${v}" in