admin/cozystack
1
0 Fork 0
mirror of https://github.com/cozystack/cozystack.git synced 2026-10-29 08:07:33 +09:00
Code Issues Packages Projects Releases Wiki Activity

test(foundationdb): widen late-converging status wait budgets

Browse Source 
The reconciled, hasListenIPsForAllPods, fullReplication and healthy gates
polled status fields with a 60s ceiling, while the strictly-earlier
health.available gate had 300s. The operator stamps generations.reconciled
only after every process holds its role for minimumUptimeSecondsForBounce
(60s), making reconciled and the health fields that depend on it the last
status fields to converge. The short ceiling closed the gate seconds before
the field landed on a healthy, fully-reconciled cluster, causing intermittent
jsonpath poll timeouts.

Raise those four gates to 300s to match the health.available budget. The waits
stay event-driven; only the ceilings widen.

Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
This commit is contained in:
Aleksei Sviridkin committed 2026-06-24 22:42:55 +03:00
1 parent 29bf56f550
commit a79f8009f4
1 file changed
+19 -7
Download Patch File Download Diff File Expand all files Collapse all files
hack/e2e-apps/foundationdb.bats
+19 -7
View File
@@ -91,15 +91,27 @@ EOF
# Validate status.desiredProcessGroups field
timeout 60 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.desiredProcessGroups}' | grep -q '^[0-9][0-9]*$'; do sleep 10; done"
# Validate status.generations.reconciled field
timeout 60 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.generations.reconciled}' | grep -q '^[0-9][0-9]*$'; do sleep 10; done"
# Validate status.generations.reconciled field.
# The operator only stamps generations.reconciled after every process has
# held its role for minimumUptimeSecondsForBounce (60s, see
# packages/apps/foundationdb/templates/cluster.yaml), so this is structurally
# the last status field to converge. It needs at least the same budget as the
# earlier health.available gate (300s), not the short 60s ceiling.
timeout 300 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.generations.reconciled}' | grep -q '^[0-9][0-9]*$'; do sleep 10; done"
# Validate status.hasListenIPsForAllPods field
timeout 60 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.hasListenIPsForAllPods}' | grep -q 'true'; do sleep 10; done"
# Validate status.hasListenIPsForAllPods field. A bounce restarts fdbserver
# in-place, so pods and their listen IPs do not churn and this usually settles
# before the reconciled gate. Widened to 300s for a conservative ceiling
# consistent with the surrounding gates; the 300s is headroom, not because
# this field is as late-converging as generations.reconciled.
timeout 300 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.hasListenIPsForAllPods}' | grep -q 'true'; do sleep 10; done"
# Validate comprehensive status.health fields
timeout 60 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.health.fullReplication}' | grep -q 'true'; do sleep 10; done"
timeout 60 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.health.healthy}' | grep -q 'true'; do sleep 10; done"
# Validate comprehensive status.health fields. fullReplication and healthy
# settle only once the cluster is fully reconciled (after the post-bounce
# uptime gate), so they share the same 300s budget rather than the short 60s
# ceiling.
timeout 300 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.health.fullReplication}' | grep -q 'true'; do sleep 10; done"
timeout 300 sh -ec "until kubectl -n tenant-test get foundationdbclusters.apps.foundationdb.org foundationdb-$name -o jsonpath='{.status.health.healthy}' | grep -q 'true'; do sleep 10; done"
# Verify security context is applied correctly (non-root user)
storage_pod=$(kubectl -n tenant-test get pods -l foundationdb.org/fdb-cluster-name=foundationdb-$name,foundationdb.org/fdb-process-class=storage --no-headers | head -n1 | awk '{print $1}')