The per-pod subquery stepped once a minute and took the latest sample
at each step, so with a 30-second scrape it looked at half the samples.
A queue that emptied only in a skipped sample did not count, and the
rule could page on a cluster that was keeping up. While a recreated pod
overlapped its old series, taking the maximum also hid the new
series' zero. The minimum of each raw series, then the minimum per pod
name, sees every sample and still keeps the window across a new
instance label.
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
The scraped series carries the pod IP in its instance label, so a
recreated primary starts a new series, and min_over_time over it saw
only the samples since the restart. The rule could then page ten
minutes after the pod came back with a queue that was still draining,
not after the 30 minutes its description promises. Grouping by pod name
first keeps one series per pod across the restart, and the description
now says what happens to a pod name that is newer than the window.
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
LastFailedArchiveTime compares the last failed archival with the last
successful one, so it resolves as soon as any segment gets through. A
cluster whose archiving fails intermittently, or runs slower than it
writes WAL, clears that alert every time one upload succeeds while the
queue of segments waiting on the Postgres volume keeps growing, until
the volume is full and the database stops. That alert is also only a
warning.
A healthy archiver empties archive_status within seconds of each WAL
switch. A queue that has not been empty once in 30 minutes means point
in time recovery is falling behind whatever the archiver is reporting,
so that is what pages, as critical. Idle clusters and replicas have an
empty queue and stay quiet.
Assisted-by: LLM
Signed-off-by: Alexey Artamonov <aleksei.artamonov@aenix.io>
Remove extra closing parenthesis in the CNPGClusterOffline alert expression
that causes vmalert pods to crash with "bad prometheus expr" error.
Signed-off-by: mattia-eleuteri <mattia@hidora.io>
fix regression introduced in
https://github.com/aenix-io/cozystack/pull/558
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Refactor**
- Updated label formatting in PostgreSQL operator default alerts
configuration
- Enhanced alert template generation to dynamically include multiple
alert configurations from separate files
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Alexey Artamonov