From 8515a702a3e0de733b5ead5382dfa40c49fc68cd Mon Sep 17 00:00:00 2001 From: Myasnikov Daniil Date: Sat, 2 May 2026 18:01:36 +0500 Subject: [PATCH] ci(pull-requests): drop 3x retry on Run E2E + Install Cozystack MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Audit of 30 successful PR runs found that across 5 sampled failure attempts, **25/25 retries** on `Run E2E tests` failed — the retry loop never recovered a flake, only stretched deterministic failures and tripled diagnostic wall-time. Same data shape on `Install Cozystack`. Drop both. `Prepare environment` keeps its 3x retry — that step is pure infrastructure (Talos image download, sandbox VM boot, network) where transient runner hiccups warrant a retry. On failure, the test step now captures `kubectl get hr -A -o wide` and `kubectl get events -A` under a collapsible group so triage starts with the actual broken-state snapshot. Depends on: - #2508 — installer namespace bootstrap (Helm namespace-ownership conflict) - #2509 — operator HelmRelease config knobs (`seaweedfs-system` 2-min wait race) - #2528 — harbor bucket-secret + BucketInfo gating (harbor ValuesError) - #2529 — objectstorage-controller BucketAccess conflict retry - the daniil/split-vminstance PR (vminstance disk race + VM IP/ready timeouts) - the daniil/split-event-driven PR (existence backstops surfacing real errors) Until those land, dropping the retry will fail CI for unrelated PRs that hit the seaweedfs / harbor / installer / vminstance races. Signed-off-by: Myasnikov Daniil Assisted-By: Claude Signed-off-by: Myasnikov Daniil --- .github/workflows/pull-requests.yaml | 38 +++++++++------------------- 1 file changed, 12 insertions(+), 26 deletions(-) diff --git a/.github/workflows/pull-requests.yaml b/.github/workflows/pull-requests.yaml index 3a0008dfd..8ddf81d50 100644 --- a/.github/workflows/pull-requests.yaml +++ b/.github/workflows/pull-requests.yaml @@ -227,27 +227,18 @@ jobs: until make SANDBOX_NAME=$SANDBOX_NAME prepare-env; do attempt=$((attempt + 1)) if [ $attempt -ge 3 ]; then - echo "❌ Attempt $attempt failed, exiting..." + echo "Attempt $attempt failed, exiting..." exit 1 fi - echo "❌ Attempt $attempt failed, retrying..." + echo "Attempt $attempt failed, retrying..." done - echo "✅ The task completed successfully after $attempt attempts" + echo "Prepare environment completed after $attempt attempts" # ▸ Install Cozystack - name: Install Cozystack into sandbox run: | cd /tmp/$SANDBOX_NAME - attempt=0 - until make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME install-cozystack; do - attempt=$((attempt + 1)) - if [ $attempt -ge 3 ]; then - echo "❌ Attempt $attempt failed, exiting..." - exit 1 - fi - echo "❌ Attempt $attempt failed, retrying..." - done - echo "✅ The task completed successfully after $attempt attempts" + make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME install-cozystack - name: Run OpenAPI tests run: | @@ -262,23 +253,18 @@ jobs: failed_tests="" for app in $(ls hack/e2e-apps/*.bats | xargs -n1 basename | cut -d. -f1); do echo "::group::Testing $app" - attempt=0 - success=false - until [ $attempt -ge 3 ]; do - if make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-apps-$app; then - success=true - break - fi - attempt=$((attempt + 1)) - echo "❌ Attempt $attempt failed, retrying..." - done - if [ "$success" = true ]; then + if make -C packages/core/testing SANDBOX_NAME=$SANDBOX_NAME test-apps-$app; then + echo "::endgroup::" echo "✅ Test $app completed successfully" else - echo "❌ Test $app failed after $attempt attempts" + echo "::endgroup::" + echo "❌ Test $app failed (no retry — see diagnostics below)" failed_tests="$failed_tests $app" + echo "::group::Diagnostics for $app" + docker exec "$SANDBOX_NAME" sh -c 'kubectl get hr -A -o wide 2>&1 | tail -50' || true + docker exec "$SANDBOX_NAME" sh -c 'kubectl get events -A --sort-by=.lastTimestamp 2>&1 | tail -30' || true + echo "::endgroup::" fi - echo "::endgroup::" done if [ -n "$failed_tests" ]; then echo "❌ Failed tests:$failed_tests"