Files
cozystack/packages/system
Alexey Artamonov 292189c7fa feat(seaweedfs): alert before an instance stops taking writes
A SeaweedFS instance that runs out of room fails quietly. A volume
server below its minFreeSpacePercent reserve turns every volume it holds
read-only, and one out of slots cannot host the new volume a bucket
needs once its volumes are full; either way uploads fail, backups
archiving into the instance fail in turn, and their WAL piles up on the
Postgres volume. Nothing pages on the way: the generic PVC alert waits
for 3% free, below the 5% reserve and far below a server that ran out of
slots with half its disk free, and its early warning needs the disk to
keep filling, which stops once writes fail.

The rules read what the servers themselves publish, the space left
above the reserve, the slots left and the master's failed volume picks,
so they follow the reserve and the size limit an instance is configured
with instead of restating them. A new bucket's first writes fail a few
picks while its volumes grow, so only failures that go on for 15
minutes page. Every selector is pinned to the release namespace,
because the root vmalert evaluates each instance's rules against every
namespace.

Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
2026-09-29 15:53:44 +03:00
..
…