Files
cozystack/packages
myasnikovdaniil 2642965026 fix(mariadb): widen startup probe budget so bootstrap cannot be killed (#3344)
## What this PR does

The operator builds the MariaDB startup probe without setting
`failureThreshold`, so it falls back to the Kubernetes default of 3. The
kubelet only starts probing after `initialDelaySeconds` and kills the
container on the Nth consecutive failure, so the budget is `initialDelay
+ (failureThreshold - 1) * period` — with the operator's 20s delay and
10s period that is 40 seconds, exactly the budget of the liveness probe
the startup probe exists to defer.

Bootstrapping a fresh datadir does not reliably finish inside 40 seconds
under a constrained CPU limit or slow replicated storage, and
overrunning it is not a slow start that recovers on the next attempt.
The kubelet kills the container after `mariadb-install-db` has created
the system tables but before the entrypoint's temporary server applies
the root grant. The restarted entrypoint keys `DATABASE_ALREADY_EXISTS`
off that directory, skips initialization for good, and the server comes
up with the install-time root account — local only and passwordless —
instead of the credentials the operator holds. Every probe from then on
fails with `Access denied for user 'root'`, the pod crash-loops
permanently, and the install times out on a MariaDB that can never
become ready.

Raising `failureThreshold` to 30 lifts the first-boot budget to 310
seconds. Only the threshold is set, so the operator keeps its own probe
handler — the agent HTTP endpoint when replication is enabled, the exec
check otherwise. Liveness and readiness keep the operator defaults and
stay suppressed until the startup probe first succeeds, so a server that
hangs after a successful start is still caught as quickly as before.

The defect is still present in the latest upstream operator release, so
a version bump does not resolve it.

The e2e suite needed a matching adjustment. The chart hands the operator
a `MariaDB` CR and ships no workload of its own, so the HelmRelease goes
Ready as soon as helm applies the manifests and never waits for a pod —
the gate that actually covers first boot is the Endpoints assert, which
sat at 3m, below the budget the probe now grants. That one moves to 10m;
the HelmRelease ceilings are left as they were.

### Release note

```release-note
fix(mariadb): widen the startup probe budget so a slow first boot cannot leave the datadir permanently unusable
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved MariaDB startup reliability by allowing more time for
initialization before startup checks fail.
* Reduced the risk of containers being terminated during first-time
bootstrap.

* **Tests**
* Added coverage for startup probe timing across single-replica and
replicated deployments.
* Verified that liveness and readiness probe defaults remain unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-20 11:11:22 +05:00
..