Arsolitt
165e175d70
feat(monitoring): alert on DCGM throttle divisor drift
...
The /1e9 divisor in gpu:{power,thermal}_throttle_fraction:rate5m was
derived empirically against DCGM 3.x on A10 — the counter documents
itself as microseconds but ticks in nanoseconds in practice. If a
future exporter release honors the documented units, pre-clamp values
would exceed 1.0 while clamp_max(..., 1) silently masks the drift,
plateauing every throttle fraction at 100% and making the panels
lie in unison.
Add a validation group that fires when the raw max/1e9 value exceeds
1.0 for 15m, so we notice and rescale to /1e6 before dashboards
silently mislead operators.
Signed-off-by: Arsolitt <arsolitt@gmail.com >
2026-04-18 08:11:57 +03:00
..
2026-01-15 22:35:56 +01:00
2026-04-16 08:50:30 +00:00
2026-04-16 08:50:30 +00:00
2026-01-19 13:59:29 +01:00
2026-03-25 15:57:25 +05:00
2026-04-16 08:50:30 +00:00
2026-03-03 19:32:32 +03:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-02-19 22:12:33 +03:00
2026-01-15 16:06:56 +01:00
2026-02-26 18:32:35 +05:00
2026-02-26 15:28:19 +03:00
2026-02-20 10:43:48 +05:00
2026-04-11 16:39:58 +03:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-02-14 11:42:52 +01:00
2026-02-10 22:47:20 +05:00
2026-02-02 15:32:50 +01:00
2026-01-15 16:06:56 +01:00
2026-04-16 08:50:30 +00:00
2026-03-27 17:48:46 +05:00
2026-04-16 08:50:30 +00:00
2026-04-13 17:15:09 +03:00
2026-04-16 08:50:30 +00:00
2026-03-10 08:30:43 +01:00
2026-03-25 15:57:25 +05:00
2026-03-10 12:48:01 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-01-20 03:24:32 +01:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-01-19 13:59:29 +01:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00
2026-04-18 07:19:05 +03:00
2026-04-16 08:50:30 +00:00
2026-04-10 12:47:46 +05:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-01-19 13:59:29 +01:00
2026-03-02 12:35:14 +05:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-04-16 08:50:30 +00:00
2026-04-10 12:47:46 +05:00
2026-04-02 19:29:54 +03:00
2026-03-11 15:15:09 +03:00
2026-03-23 15:18:23 +03:00
2026-02-10 22:16:11 +01:00
2026-04-16 08:50:30 +00:00
2026-04-16 08:50:30 +00:00
2026-03-25 15:57:25 +05:00
2026-02-21 20:46:31 +01:00
2026-03-10 12:42:01 +01:00
2026-02-20 20:16:33 +01:00
2026-04-16 08:50:30 +00:00
2026-01-15 16:06:56 +01:00
2026-02-20 20:16:34 +01:00
2026-04-16 08:50:30 +00:00
2026-04-16 08:50:30 +00:00
2026-04-17 11:32:44 +05:00
2026-04-13 17:15:09 +03:00
2026-02-10 20:19:03 +01:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 23:32:41 +00:00
2026-01-15 16:06:56 +01:00
2026-01-20 03:24:32 +01:00
2026-03-25 15:57:25 +05:00
2026-04-18 06:35:12 +03:00
2026-04-18 08:11:57 +03:00
2026-03-25 15:57:25 +05:00
2026-04-16 08:50:30 +00:00
2026-02-17 22:54:12 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-04-16 08:50:30 +00:00
2026-02-17 20:03:01 +03:00
2026-03-25 15:57:25 +05:00
2026-01-20 03:24:32 +01:00
2026-03-17 11:20:44 +01:00
2026-03-25 15:57:25 +05:00
2026-03-25 16:11:14 +03:00
2026-02-11 22:54:56 +01:00
2026-03-17 01:59:28 +03:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-02-06 13:30:58 +03:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-04-16 08:50:30 +00:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-03-25 15:57:25 +05:00
2026-02-10 17:49:44 +03:00
2026-01-28 20:08:35 +03:00
2026-01-15 16:06:56 +01:00
2026-03-10 12:03:51 +05:00
2026-03-25 15:59:22 +05:00
2026-04-14 19:59:20 +05:00
2026-04-14 19:59:19 +05:00
2026-03-30 10:55:26 +05:00
2026-03-25 15:57:25 +05:00
2026-01-15 16:06:56 +01:00
2026-01-15 16:06:56 +01:00