The first pass left routes that hand the body straight to a service
(create_zone(data), MonitorService.create(data), ...). Those services
read it as a dict, so a body of null, [] or a bare string still
crashed them into a 500. Agent long-poll, external webhook receivers
and the routes that already type-check their body are left alone.
328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
33 routes wrapped their body in try/except Exception only to return
unexpected_response(e), which records the crash and answers the same
body the global 500 handler already does. Every site was directly in a
route function, so the handler sees the same exceptions. SAML metadata
relied on its wrapper for an unconfigured IdP; that is now a typed 404
instead of a logged crash on a public, probed route.
Plan 76 milestone B, drift/bug row 4. 43 API handlers answered HTTP 500 from
their own `except Exception`, so the global 500 handler never ran — and that
handler was the ONLY caller of error_log_service.record_error(source='backend').
Every one of those crashes was absent from /monitoring/errors, and the page read
empty precisely because someone had "handled" them.
35 of the 43 also returned str(exc) to the caller. Two returned the full
traceback (templates.py, which also print()ed it), and several logged with a
locally-minted uuid4 'ref' that correlated with nothing but a log line.
The recording side of the 500 handler moves to app/error_reporting.py, so it is
importable rather than reachable only by not-catching:
record_unexpected(exc) -> logs, rolls the session back, records, returns
the request id
unexpected_response(exc) -> the above plus the documented crash body
The 500 handler now calls the same pair, so there is one recorder rather than
one recorder and 43 routes that quietly opted out of it.
Caller-visible change on those endpoints: the body is now the documented
{'error': 'Internal server error', 'status': 500, 'code': 'internal_error',
'request_id': ...} instead of the exception text. That is the point — an
unexpected exception's text is written for whoever reads the log and routinely
carries connection strings, absolute paths, and SQL. The request id is the part
a caller can act on, and it matches the row in the error log. No frontend code
read the 'ref' or 'trace' keys this drops (checked).
Two sites keep their own wording and call record_unexpected instead: sso.py's
login failures should not read as a generic crash on the login page.
docker.py's bulk endpoint keeps its per-item `results` — the caller's half-done
work survives, only the exception text is dropped.
Ratchet: tests/unreported_crash_census.py + its test, held at **0**, not a
legacy baseline to shrink. It counts blanket handlers that answer 500 without
calling either reporter, and is scoped to app/api — a service catching an
exception and returning a value is making a domain decision, not declaring an
HTTP crash. Verified non-vacuous: an injected handler fails it, and it passes
again on revert.
Measured and deliberately still open: 56 API handlers swallow a crash and
answer 200. A failure reported as success is worse, but converging it is
milestone C's response-envelope work, not this row.
Gate: tests/test_unexpected_error_reporting.py (11, including the two that pin
why the old shape lost the crash), test_unreported_crash_ratchet.py (9), and
the 66 test files covering every touched module: 1105 passed, 7 skipped, 1
pre-existing Windows-only failure (test_fleet_doctor's socket.gaierror).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Restore the plan-33 bounce slice lost in the rebuild (migration 069 survived;
the model/route/service did not). EmailBounceState model reconstructed verbatim
from the clean pyc dump. New bounce_service.BounceService maps provider payloads
(generic/SendGrid/Postmark/SES/Mailgun), records bounces correlated to
NotificationDelivery by provider_message_id, auto-mutes a recipient past
MUTE_THRESHOLD (complaints mute immediately), and the email channel skips muted
targets. Inbound webhook POST /api/v1/notifications/inbound/email (HMAC-SHA256,
404 until notify.inbound_secret set, 401 on bad sig) + operator unmute + admin
bouncing list + email_status in GET /preferences.
Route/service names/paths faithful to the salvaged test string-index; logic
reconstructed from the plan-33 doc (the route+test pycs were corrupt). Model
registration in app/models/__init__.py lands with the Phase-2 commit (shared
file). Verified: test_email_bounce.py 16 passed; notification suite no regressions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the email unification: the outbound relay is now one smtp
EmailProviderConnection flagged uses_relay, so a single table drives both the
Notification Bus and the Postfix relay. The legacy single-row email_relay_config
is folded in on first boot and kept only as a read fallback.
- EmailRelayService rewritten over the unified table: get/save/disable/test all
operate on the uses_relay smtp connection; password stays Fernet-encrypted in
credentials_json; masked round-trips preserve the stored secret. Postfix apply
unchanged (no-op when Postfix absent).
- migrate_legacy_config(): idempotent fold of email_relay_config → connection
(password copied verbatim — same Fernet key); wired into the boot-time legacy
migration block.
- API: PUT /notifications/admin/email-providers/:id toggles usage flags
(uses_notifications/uses_relay/relay_priority); uses_relay honored for SMTP only.
Verified: 6 new relay tests + usage-flag + notification-bus suites pass (30 passed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reusable, non-blocking notification layer on top of the Queue Bus:
notify.send(event, to, data) records a Notification + per-channel
NotificationDelivery, queues them, and a daemon consumer delivers via
pluggable channel adapters (in-app, email, chat) inheriting retry,
backoff and dead-letter.
- Jinja2 email system: branded, dark-mode, Outlook-safe base layout +
per-event templates (security/backup/monitoring/welcome/invitation).
The legacy inline f-string email was removed and now renders here.
- In-app center API (inbox/unread-count/mark-read) + per-user Socket.IO push.
- Admin delivery log (stats/filter/retry).
- Email providers (SMTP/SendGrid/Postmark/SES/Mailgun) with encrypted
credentials + test; the email adapter prefers the default provider and
falls back to the legacy SMTP config; surfaced in ConnectionRegistry.
- NotificationService.send_all() is now a non-blocking shim that routes
system alerts through the bus. Migrations 023 + 024.
Tests: backend/tests/test_notification_bus.py (20). Full suite: 391 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add app/utils/formatting.py with format_bytes() and delegate 9 duplicated
human-readable byte formatters across services.
- Import admin_required from app.middleware.rbac in 15 API modules instead of
redefining it locally.
- Extend app/utils/slug.py with validate_app_name() and use it for template
install/validation and apps slug normalization.
Implements backend API endpoints and service logic for executing SQL queries (MySQL, PostgreSQL, SQLite) with readonly enforcement and table structure retrieval. Adds user notification preferences model and API for per-user alert settings. Extends notification service to support email notifications with HTML templates. Introduces a frontend QueryRunner component for interactive SQL execution, table browsing, and result export. Updates related frontend and backend files to support new features.
Introduces backend and frontend support for notification webhooks, including Discord, Slack, Telegram, and generic webhooks. Adds new API endpoints for managing and testing notification channels, updates monitoring service to send alerts to these channels, and provides a new UI for configuring and testing notification integrations in settings. Also adds required styles and updates dependencies to include the requests library.
Juan Denis