mirror of
https://github.com/cozystack/cozystack.git
synced 2026-11-21 12:13:55 +09:00
Two etcd members, two Redis Sentinels or both replicas of a storage tier can end up on one node through a soft spread rule, a drain, or a placement made before a spread rule existed. Nothing reported it, and the scheduler does not move running pods, so it stays like that until a pod is recreated. One node failure then costs the quorum or every copy at once. etcd-operator owns each member pod by its own EtcdMember, so members are grouped by the EtcdCluster name their names start with. Owner and node are read from kube_pod_info because kube_pod_owner and kube_pod_status_phase get the node of the kube-state-metrics pod as a target label. Nodes with a NoSchedule or NoExecute taint do not count, which covers cordoned, not ready and unreachable nodes. The alert waits an hour, longer than a replacement pod runs next to its terminating predecessor or a node takes to drain and come back. Assisted-by: LLM Signed-off-by: Yan Bondarenko <202671653+yankawai@users.noreply.github.com>