Aleksei Sviridkin
74c87247ed
## What `drbd_stall_act` in `packages/system/linstor/hack/plunger/plunger-satellite.sh` disconnects a single peer with `drbdadm disconnect res:peer` and then calls `drbdadm connect res`, which walks every connection of the resource. Each peer that is still connected makes `drbdsetup connect` fail with `Device has a net-config (use disconnect first)` (exit code 10), the helper only logs `WARN: action failed for res:peer`, and the peer that was just disconnected is never reconnected. ## Observed cozystack 1.4.6 sandbox, LINSTOR 1.33.2 / DRBD 9.3.2 on Talos 1.13.6, resync of a 20 GiB volume while the Primary keeps writing: ``` STALL detected: res=pvc-9d99d1b9-1efe-4340-bbb3-13421c6bc7f2 sync_peer=sb-cs0203 percent_in_sync=90.4 -> disconnect/connect pvc-9d99d1b9-1efe-4340-bbb3-13421c6bc7f2: Failure: (125) Device has a net-config (use disconnect first) Command 'drbdsetup connect pvc-9d99d1b9-1efe-4340-bbb3-13421c6bc7f2 5' terminated with exit code 10 WARN: action failed for pvc-9d99d1b9-1efe-4340-bbb3-13421c6bc7f2:sb-cs0203 ``` ## Fix Reconnect the same `res:peer` pair that was disconnected. Verified on the sandbox by patching the live `linstor-plunger` ConfigMap: the peer reconnects and the resync completes. ## Note Not part of this change: the stall detector compares `percent-in-sync`, which `drbdsetup status --json` reports with one decimal, so a slow resync (for example one throttled down to `c-min-rate` by application I/O) can look stalled for `STALL_ITERS` iterations and trigger a reconnect every couple of minutes. Happy to follow up on that separately if you think it is worth it. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Improved reconnection behavior by targeting the explicitly selected resource-peer endpoint instead of reconnecting only at the resource level. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Cozystack
Cozystack is a free and open-source platform and framework for building clouds.
Cozystack is a CNCF Sandbox project that was originally built and sponsored by Ænix.
With Cozystack, you can transform a bunch of servers into an intelligent system with a simple REST API for spawning Kubernetes clusters, Database-as-a-Service, virtual machines, load balancers, HTTP caching services, and other services with ease.
Use Cozystack to build your own cloud or provide a cost-effective development environment.
Use-Cases
-
Using Cozystack to build a public cloud
You can use Cozystack as a backend for a public cloud -
Using Cozystack to build a private cloud
You can use Cozystack as a platform to build a private cloud powered by Infrastructure-as-Code approach -
Using Cozystack as a Kubernetes distribution
You can use Cozystack as a Kubernetes distribution for Bare Metal
Documentation
The documentation is located on the cozystack.io website.
Read the Getting Started section for a quick start.
If you encounter any difficulties, start with the troubleshooting guide and work your way through the process that we've outlined.
Versioning
Versioning adheres to the Semantic Versioning principles.
A full list of the available releases is available in the GitHub repository's Release section.
Contributions
Contributions are highly appreciated and very welcomed!
In case of bugs, please check if the issue has already been opened by checking the GitHub Issues section. If it isn't, you can open a new one. A detailed report will help us replicate it, assess it, and work on a fix.
You are welcome to take on the fix yourself. Commits are used to generate the changelog, and their author will be referenced in it.
If you have a concrete feature request, open an issue. For a cross-cutting or architectural change, open a design proposal in the community repo. For questions or early ideas, use GitHub Discussions. See CONTRIBUTING.md for the full routing.
Community
You are welcome to join us in #cozystack, on either CNCF Slack (get an invite) or Kubernetes Slack (get an invite), in our Telegram group, and at our community meetings. Add them to your Google Calendar or iCal for convenience.
Project documents
| Document | What it covers |
|---|---|
| Governance | How decisions are made and who makes them |
| Contributor Ladder | How to grow from contributor to maintainer |
| Maintainers | Who maintains the project |
| Code of Conduct | What we expect of each other |
| Security Policy | Reporting a vulnerability, and which releases get fixes |
| Adopters | Who runs Cozystack in production |
| Roadmap | Where the project is going |
License
Cozystack is licensed under Apache 2.0.
The code is provided as-is with no warranties.
Commercial Support
A list of companies providing commercial support for this project can be found on official site.