Myasnikov Daniil
660b79401f
A churn-heavy platform install intermittently wedges a pod in
ContainerCreating because the cilium-agent rejects its sandbox with
'[PUT /endpoint/{id}][400] putEndpointIdInvalid "IP ipv4:<X> is already
in use"', repeated until the agent restarts or the install times out.
The disputed IP has no live owner: IPAM and the ipcache released it, but
the agent's in-memory endpointManager still holds a stale Endpoint from a
previously-deleted pod whose CNI DEL was skipped or raced (so
removeReferencesLocked never ran). Neither the operator's CiliumEndpoint
GC nor the agent's veth-based endpoint GC can reap it. This surfaced as
~6 of the post-#2558 e2e failures, each on a different victim pod
(velero node-agent, fluent-bit, fdb-operator, cdi-operator, capi
bootstrap), failing PRs unrelated to the change under test. Tracked
upstream at cilium/cilium#38313 (closed stale); no released or
pre-release Cilium fixes it and no flag disables it.
hack/e2e-cilium-endpoint-leak-healer.sh runs as a file-scoped background
watchdog (setup_file/teardown_file). It detects a wedged pod, confirms
the IP is a true orphan -- an endpoint in the owning node's registry
holds it, that endpoint backs no live Running pod, and a different pod is
wedged requesting the same IP -- then evicts only that endpoint via
'cilium-dbg endpoint disconnect ipv4:<X>' (DELETE /endpoint/{id} ->
unexpose -> removeReferencesLocked). Blast radius is one dead endpoint,
not an agent restart. It is read-only until an orphan is positively
confirmed and refuses (loudly) to touch an endpoint backing a live pod,
so a genuine duplicate-IP bug stays visible rather than masked.
This is an interim CI mitigation, not a product fix; remove the script
and the bats hooks once a fixed Cilium ships.
Assisted-By: Claude <noreply@anthropic.com>
Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
Signed-off-by: Ivan Okhotnikov <ivan.okhotnikov@aenix.io>
Cozystack
Cozystack is a free platform and framework for building clouds.
Cozystack is a CNCF Sandbox Level Project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You can express your intention to on the fix on your own. Commits are used to generate the changelog, and their author will be referenced in it.
If you have Feature Requests please use the Discussion's Feature Request section.
Community
You are welcome to join our Telegram group and come to our weekly community meetings. Add them to your Google Calendar or iCal for convenience.
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.