Aleksei Sviridkin a40be5e212 fix(e2e): widen tenant-root monitoring bring-up waits to 15m under load (#3160)
## What this PR does

The `Configure Tenant and wait for applications` install test brings up
the entire root-tenant monitoring stack and waits for each component to
become ready/operational. Those waits span the VictoriaMetrics operator
resources (`vmalert`/`vmalertmanager`, `vlclusters/generic`,
`vmcluster/shortterm`+`longterm`) and the Grafana pair (`grafana-db`
CNPG cluster, `grafana-deployment`). Each converges only once its
underlying workloads schedule and turn Ready, and during platform
bring-up they contend for the same node resources as the rest of the
install — so their convergence is load-sensitive.

Only `vmalert`/`vmalertmanager` used a 15m budget; the other four used
10m, so this block carried a 10m/15m split even though all its resources
contend for the same node capacity during install. Under install-time
load (multiple concurrent e2e sandboxes on one runner) the monitoring
bring-up is slow across the whole block rather than on a single
resource, so the shorter budgets risk failing the whole install on PRs
whose branches never touched monitoring — and per-resource bumps are
whack-a-mole: raising one budget just moves the risk to the next
component in the block. On a calm sandbox each of these reaches
ready/operational in under a second, which is what distinguishes the two
cases — slow-not-stuck convergence under contention, not a product
regression.

This lifts the whole tenant-root monitoring wait-set to a single uniform
15m budget and wraps each wait in an on-timeout diagnostic block that
dumps the resource YAML (status conditions / `updateStatus`) plus
tenant-root pod state. A genuinely stuck resource still fails — now at
15m, and with legible diagnostics instead of a bare `timed out` line —
so this does not weaken the gate's ability to catch real breakage. The
extra budget costs near-zero wall-time on the happy path and stays well
inside the E2E job's 180m cap.

The remaining upstream angle is the real driver: how long the
VictoriaMetrics/VictoriaLogs stack and the Grafana bring-up take to
become operational under resource contention during install. This change
addresses the e2e robustness side; reducing that bring-up cost is a
separate, product-side improvement.

### Screenshots

Not applicable — no UI changes.

### Release note

```release-note
NONE
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved and standardized readiness checks during tenant setup so
application bring-up waits behave more consistently.
* Enhanced failure handling to provide clearer diagnostics (including
resource details and pod status) when readiness checks time out.

* **Tests**
* Standardized longer readiness wait timeouts for multiple application
components to reduce flakiness during e2e runs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-07-01 13:12:43 +03:00
2025-10-08 09:43:34 +05:00
2025-04-01 18:48:14 +02:00
2024-02-08 12:04:32 +01:00

Cozystack Cozystack

Open Source Apache-2.0 License Support Active GitHub Release GitHub Commit OpenSSF Best Practices

Cozystack

Cozystack is a free platform and framework for building clouds.

Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.

With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.

Use Cozystack to build your own cloud or provide a cost-effective development environment.

Cozystack user interface

Use-Cases

Documentation

The documentation is located on the cozystack.io website.

Read the Getting Started section for a quick start.

If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.

Versioning

Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.

Contributions

Contributions are highly appreciated and very welcomed!

In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.

You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.

If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.

Community

You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.

License

Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.

Commercial Support

A list of companies providing commercial support for this project can be found on official site.

Languages
Go 46.7%
Shell 37.8%
TypeScript 7.1%
Go Template 5.5%
Makefile 1.5%
Other 1.2%