Aleksei Sviridkin
ab8b91f39f
Three of the six steps in the pre-delete hook call kubectl delete
with neither --wait=false nor --timeout. kubectl defaults --wait to
true and reads --timeout=0 as "wait forever", which it implements as
a week, so the KamajiControlPlane delete in step 3, the DataVolume
delete in step 5 and the LoadBalancer Service delete in step 6 each
park the hook for 168 hours on a single stuck finalizer. A DataVolume
held by a CDI importer or a PVC finalizer is the reachable case.
None of the three waits now. Nothing downstream reads the result of
steps 5 and 6, and what matters after step 3 is the
TenantControlPlane, which step 4 deletes and waits for directly;
blocking on the KamajiControlPlane first only adds a second place for
a CAPI finalizer to stall. An expired wait and --wait=false leave the
same cluster state either way, with the same objects finalizing in
the background once the hook has moved on.
Not waiting also keeps these steps under `set -e` with no guard. A
guard cannot tell an expired wait from a rejected request, so it
would report a failed delete as cleanup that happened; with no
timeout to expire, every non-zero exit left is a real failure.
Drop the watch verb from the three resources those steps delete, and
pin the whole rule set. kubectl opens a watch only on the wait path,
so the grant was unexercised for the three; for helmreleases and
tenantcontrolplanes its absence would not error but hang, which is
the failure mode hardest to read from the outside.
Set backoffLimit to 0, against a default of 6 that restartPolicy
Never turns into seven pods in series. Zero rather than one because
both remaining waits are guarded, so an attempt can spend its whole
budget without failing and a retry would start from step 1 with the
clock already there; on a nodeless tenant it would also re-pay step
4, since attempt one leaves the TenantControlPlane still terminating.
The failure a retry would cover is an eviction, which is not bounded
in time and is worst exactly when it arrives late. The price is that
such an eviction fails the whole attempt and flux repeats it on its
own interval.
No activeDeadlineSeconds: the waits and helm-controller's uninstall
budget leave no gap a deadline could sit in, once a Docker Hub image
pull is counted. That needs the budget raised first, through
spec.uninstall.timeout on the generated HelmRelease, and is left out
rather than set to a number that would cut runs which were going to
finish.
These bounds cover the waiting, not the request: --request-timeout is
still 0 on every call in the hook, so an unresponsive apiserver hangs
it as before. That is a different failure from the stuck finalizer,
and it wants one answer across every call rather than a flag on the
deletes.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
(cherry picked from commit b60c08f110)
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.