Aleksei Sviridkin
a53d64d67a
## What this PR does The E2E `Install Cozystack` gate flakes intermittently: a platform HelmRelease/Deployment sticks `InProgress` → `Failed` and its dependents cascade to `DependencyNotReady`, on a **different component each run** (observed: opensearch-operator-controller-manager, monitoring-agents-kube-state-metrics, kamaji readiness "context deadline exceeded"). That non-determinism is the signature of host resource thrash, not a product bug. **Measured envelope.** The sandbox boots three QEMU guests (`hack/e2e-prepare-cluster.bats`), each `-smp 8 -m 24576`: | Resource | Guests demand | 24-vCPU runner (current) | 32-vCPU runner (this PR) | | --- | --- | --- | --- | | vCPU | 3 × 8 = 24 | 24 → 0 host headroom | 32 → 8 (25%) headroom | | RAM | 3 × 24 = 72 GiB | 96 GiB → 24 GiB headroom | 128 GiB → 56 GiB headroom | On the 24-vCPU runner the guests alone pin all 24 host vCPUs, leaving nothing for QEMU's own emulation/IO threads, vhost-net, the host `iptables MASQUERADE` every guest image pull traverses, containerd, the kernel and the runner agent. Under the peak concurrency of the platform install the guest vCPUs contend with those host threads and lose cycles to CPU steal, so in-guest kubelet/apiserver readiness/liveness probes miss their deadlines and the HelmRelease flips. Across sampled failed runs the symptom is always a probe/readiness timeout with `DependencyNotReady`, never `OOMKilled`/`Evicted`/`MemoryPressure` — CPU steal, not memory, is the binding constraint (RAM already keeps 24 GiB of headroom). **Fix.** Move the job to the `oracle-vm-32cpu-128gb-x86-64` class (already referenced in-repo as the commented alternative). Guest sizing is unchanged (24 vCPU / 72 GiB), so the CPU-bound install does not slow down, while the host keeps 8 vCPU + 56 GiB of headroom that absorbs the emulation/network overhead the probes were starving against. This is deliberately **not** a retry or timeout bump (forbidden by `docs/agents/e2e-testing.md`): the install gate keeps its teeth, runs once, and still fails loud on a genuine product bug. It removes the harness oversubscription that manufactured false failures, and is complementary to the per-component probe-hardening work and to the in-sandbox image-mirror change (which relieves the distinct storage-import axis). **Trade-off / caveat.** The 32-vCPU class costs more per E2E run and adds load to that runner pool — an infra/cost decision for maintainers. A zero-cost alternative (drop guest `-smp` to 6 → 18 vCPU / 6 host headroom on the existing runner) was considered and rejected as primary: it cuts guest compute 25% on an already CPU-bound install and could trade steal-flakes for wall-clock-timeout flakes, unverifiable without a CI run. ### Screenshots N/A — CI-only change, no user-facing surface. ### Release note ```release-note NONE ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Increased the default runner size for end-to-end pull request checks. * Added clearer workflow guidance to reduce flaky readiness and liveness failures during installation steps. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.