Aleksei Sviridkin
b60c08f110
Three of the six steps in the pre-delete hook call kubectl delete with neither --wait=false nor --timeout. kubectl defaults --wait to true and reads --timeout=0 as "wait forever", which it implements as a week, so the KamajiControlPlane delete in step 3, the DataVolume delete in step 5 and the LoadBalancer Service delete in step 6 each park the hook for 168 hours on a single stuck finalizer. A DataVolume held by a CDI importer or a PVC finalizer is the reachable case. None of the three waits now. Nothing downstream reads the result of steps 5 and 6, and what matters after step 3 is the TenantControlPlane, which step 4 deletes and waits for directly; blocking on the KamajiControlPlane first only adds a second place for a CAPI finalizer to stall. An expired wait and --wait=false leave the same cluster state either way, with the same objects finalizing in the background once the hook has moved on. Not waiting also keeps these steps under `set -e` with no guard. A guard cannot tell an expired wait from a rejected request, so it would report a failed delete as cleanup that happened; with no timeout to expire, every non-zero exit left is a real failure. Drop the watch verb from the three resources those steps delete, and pin the whole rule set. kubectl opens a watch only on the wait path, so the grant was unexercised for the three; for helmreleases and tenantcontrolplanes its absence would not error but hang, which is the failure mode hardest to read from the outside. Set backoffLimit to 0, against a default of 6 that restartPolicy Never turns into seven pods in series. Zero rather than one because both remaining waits are guarded, so an attempt can spend its whole budget without failing and a retry would start from step 1 with the clock already there; on a nodeless tenant it would also re-pay step 4, since attempt one leaves the TenantControlPlane still terminating. The failure a retry would cover is an eviction, which is not bounded in time and is worst exactly when it arrives late. The price is that such an eviction fails the whole attempt and flux repeats it on its own interval. No activeDeadlineSeconds: the waits and helm-controller's uninstall budget leave no gap a deadline could sit in, once a Docker Hub image pull is counted. That needs the budget raised first, through spec.uninstall.timeout on the generated HelmRelease, and is left out rather than set to a number that would cut runs which were going to finish. These bounds cover the waiting, not the request: --request-timeout is still 0 on every call in the hook, so an unresponsive apiserver hangs it as before. That is a different failure from the stuck finalizer, and it wants one answer across every call rather than a flag on the deletes. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.