mirror of
https://github.com/jhd3197/ServerKit.git
synced 2026-11-09 21:43:05 +09:00
Juan DenisandClaude Opus 5
ade0088777
Three defects, found together because they are the same failure wearing three hats: monitoring that looks configured and is not. Fleet alert thresholds were never evaluated. check_fleet_thresholds was written, was correct, and had no caller anywhere in the backend, so a threshold set on the Fleet page was checked against nothing and a server could sit at 99% CPU for a week in silence. It now runs every minute as a built-in scheduled job alongside every other recurring task, and each alert it opens or resolves goes to admins through the Notification Bus. An evaluation nobody is told about is most of the way to no monitoring. The agent heartbeat was throwing metrics away. It built its own ServerMetrics row and filled five of the model's thirteen columns, dropping memory_used, disk_used, the network totals, both network rates and extra. The two rates are exactly what the threshold checker reads, so a network threshold could never fire for an agent-reported server: the data needed to evaluate it was discarded on the way in. Heartbeats now go through ServerMetricsService, the writer everything else already used. Metric history outlived its own retention. The history and aggregate endpoints offered 7-day and 30-day windows while cleanup deleted every sample after seven days, and two constants described hourly and daily aggregate retention that nothing produced. A 30-day chart came back holding at most seven days and said nothing about it. Both endpoints now return a coverage block — the retention in force, the earliest point they can answer for, and whether the window asked for was truncated — and the two constants that promised retention nobody implemented are gone. Real rolled-up aggregates are the other way to fix this and would need their own tables and writer; until they exist, saying what we have is the honest answer. Two more things surfaced while writing the tests. get_aggregated_metrics grouped by date_trunc, which is PostgreSQL's alone, so the endpoint raised "no such function: date_trunc" on every SQLite deployment — one of the two supported databases. Buckets are now built with the right function for the database in use. And an escalation is not a recovery. warning -> critical closes the warning row, so both a genuine recovery and a worsening breach end up as status='resolved'; announcing the second as "recovered" would tell an operator the problem went away at the moment it got worse. Supersession is tracked separately and is not notified twice, because the new alert already says what happened. The two built-in-job counts in test_jobs.py move from 16 to 17 and 24 to 25, and BASELINE_COUNT is raised for the new tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>