From 67b1a960ddb7fbe6ab4757015c8009ac1ca2ff81 Mon Sep 17 00:00:00 2001
From: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Date: Thu, 4 Jun 2026 13:39:36 +0500
Subject: [PATCH] test(e2e): fail install test when HelmReleases are not Ready
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The HR readiness gate in e2e-install-cozystack.bats was toothless in two
layers: the backgrounded kubectl-wait fan-out discarded child exit codes
(POSIX `wait` without args returns 0), and the final check echoed
"Some HelmReleases failed to reconcile" without exiting non-zero — a
regression from 1f240387f, which replaced the unavailable bats `fail`
helper with a bare echo. A permanently-failing platform HelmRelease
(e.g. the backup-controller CRD rejection, #2822) shipped through green
CI for weeks.

Replace the fan-out with a single `kubectl wait hr --all -A` for trace
visibility, gate on an outcome-based re-list (which also covers HRs
created after the snapshot, with a 120s retry to absorb momentary
drift-reconcile flaps), and dump the full Ready condition message per
non-ready HR so the real error is visible in the test output.

Refs #2822

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
---
 hack/e2e-install-cozystack.bats | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/hack/e2e-install-cozystack.bats b/hack/e2e-install-cozystack.bats
index ad915c944..2a58e6715 100644
--- a/hack/e2e-install-cozystack.bats
+++ b/hack/e2e-install-cozystack.bats
@@ -109,7 +109,9 @@ EOF
   # signal for "all platform HRs have been emitted" without hard-coding the
   # expected list, so the 5s pad lets a few late-arrivals join the snapshot.
   sleep 5
-  kubectl get hr -A | awk 'NR>1 {print "kubectl wait --timeout=15m --for=condition=ready -n "$1" hr/"$2" &"} END {print "wait"}' | sh -ex
+  # Pacing only: names every HR that timed out in the trace; the authoritative
+  # gate re-lists below, covering HRs created after this snapshot (#2822).
+  kubectl wait hr --all -A --timeout=15m --for=condition=ready || true
 
   echo "Waiting for post-install-prep to complete"
   if ! wait $POST_PREP_PID; then
@@ -119,10 +121,20 @@ EOF
   fi
   cat /tmp/post-install-prep.log
 
-  # Fail the test if any HelmRelease is not Ready
-  if kubectl get hr -A | grep -v " True " | grep -v NAME; then
-    kubectl get hr -A
+  # Fail the test if any HelmRelease is not Ready. Wait again on a fresh
+  # listing so HelmReleases created after the snapshot above are gated too;
+  # the window absorbs momentary Unknown flaps from drift reconciles.
+  if ! kubectl wait hr --all -A --timeout=15m --for=condition=ready; then
+    kubectl get hr -A || true
+    # kubectl's STATUS column truncates long messages; dump the full Ready
+    # condition per non-ready HR so the real error (e.g. a rejected CRD) is
+    # visible in the test output instead of only inside the cozyreport.
+    kubectl get hr -A --no-headers | awk '$4 != "True"' | while read -r ns name _; do
+      echo "--- Non-ready HelmRelease: $ns/$name" >&2
+      kubectl get hr -n "$ns" "$name" -o jsonpath='{range .status.conditions[*]}{.type}={.status} reason={.reason}: {.message}{"\n"}{end}' >&2 || true
+    done
     echo "Some HelmReleases failed to reconcile" >&2
+    exit 1
   fi
 }