328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
33 routes wrapped their body in try/except Exception only to return
unexpected_response(e), which records the crash and answers the same
body the global 500 handler already does. Every site was directly in a
route function, so the handler sees the same exceptions. SAML metadata
relied on its wrapper for an unconfigured IdP; that is now a typed 404
instead of a logged crash on a public, probed route.
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.
The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
The C1/C2 reference conversion: FileService raises typed errors
(PermissionDenied/NotFound/Conflict/Validation) and returns bare data;
files.py stops translating {'success': bool} envelopes and stops choosing
status codes by sniffing error prose. Denials are now 403 regardless of
wording, missing paths 404, already-exists 409. Success bodies keep the
compatibility 'success': True key so consumers are unchanged. The plan's
'eight callers' blocker was a substring-grep artifact — FileService has
exactly one consumer. Sniffing census 23 -> 11.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 76 milestone B, drift/bug row 4. 43 API handlers answered HTTP 500 from
their own `except Exception`, so the global 500 handler never ran — and that
handler was the ONLY caller of error_log_service.record_error(source='backend').
Every one of those crashes was absent from /monitoring/errors, and the page read
empty precisely because someone had "handled" them.
35 of the 43 also returned str(exc) to the caller. Two returned the full
traceback (templates.py, which also print()ed it), and several logged with a
locally-minted uuid4 'ref' that correlated with nothing but a log line.
The recording side of the 500 handler moves to app/error_reporting.py, so it is
importable rather than reachable only by not-catching:
record_unexpected(exc) -> logs, rolls the session back, records, returns
the request id
unexpected_response(exc) -> the above plus the documented crash body
The 500 handler now calls the same pair, so there is one recorder rather than
one recorder and 43 routes that quietly opted out of it.
Caller-visible change on those endpoints: the body is now the documented
{'error': 'Internal server error', 'status': 500, 'code': 'internal_error',
'request_id': ...} instead of the exception text. That is the point — an
unexpected exception's text is written for whoever reads the log and routinely
carries connection strings, absolute paths, and SQL. The request id is the part
a caller can act on, and it matches the row in the error log. No frontend code
read the 'ref' or 'trace' keys this drops (checked).
Two sites keep their own wording and call record_unexpected instead: sso.py's
login failures should not read as a generic crash on the login page.
docker.py's bulk endpoint keeps its per-item `results` — the caller's half-done
work survives, only the exception text is dropped.
Ratchet: tests/unreported_crash_census.py + its test, held at **0**, not a
legacy baseline to shrink. It counts blanket handlers that answer 500 without
calling either reporter, and is scoped to app/api — a service catching an
exception and returning a value is making a domain decision, not declaring an
HTTP crash. Verified non-vacuous: an injected handler fails it, and it passes
again on revert.
Measured and deliberately still open: 56 API handlers swallow a crash and
answer 200. A failure reported as success is worse, but converging it is
milestone C's response-envelope work, not this row.
Gate: tests/test_unexpected_error_reporting.py (11, including the two that pin
why the old shape lost the crash), test_unreported_crash_ratchet.py (9), and
the 66 test files covering every touched module: 1105 passed, 7 skipped, 1
pre-existing Windows-only failure (test_fleet_doctor's socket.gaierror).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 3. The quick-access rail links the local target at /etc/nginx,
but /etc was never in ALLOWED_ROOTS, so the shortcut returned 403 for every
user including admins -- a dead button, and our own regression: it shipped
after the review that flagged the surrounding SSL story.
/etc/nginx is now an allowed root, but browse-only. The panel generates those
vhosts itself, so a hand edit is silently overwritten on the next domain
change, and a writable /etc/nginx would hand anyone holding files.write a
root-owned config that the panel itself reloads -- a much larger grant than
"let me look at the effective config", which is all the shortcut is for.
READ_ONLY_ROOTS is a second tier under ALLOWED_ROOTS rather than a special
case for this one path: is_path_writable() = allowed AND not read-only, and
every mutating entry point now gates on it instead of on is_path_allowed, so
adding a future root to the list is enough to make the whole write surface
refuse it. Both comparisons resolve with realpath first, so a symlink planted
in a writable root cannot launder a write into the subtree.
copy is deliberately asymmetric -- copying a vhost out to keep only reads the
source, so only the destination is gated -- while move removes its source and
therefore checks both ends. The upload endpoint repeats the check because it
writes to disk directly instead of going through FileService's methods.
Only this subtree is opened, not /etc: /etc/passwd and /etc/shadow stay
denied, and the panel-internal PROTECTED_ROOTS exclusion is untouched
(GHSA-rm3m-9mvw-68fh).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-ups from the GHSA-4wqh-7f4f-5qmx / GHSA-mc93-rc3x-fpgq review:
- File Manager read endpoints (browse/info/read/search/disk-*/analyze/
type-breakdown/download + s3/browse, s3/read, s3/download-url) only
required a valid JWT, so a user whose files.read permission is revoked
via custom permissions could still read and download files. They now
require permission_required('files', 'read'), matching the write side.
- LogService._read_syslog() interpolated the unit name into a bash -c
string using subprocess.list2cmdline(), which is cmd.exe-style quoting
and does not stop $(...)/backtick expansion under bash. grep now runs
as a plain argv list (with --) and the tail happens in Python.
Reachable only on non-systemd hosts by admins, but the same bug class
as the Postfix advisory.
Regression tests: revoked files.read 403s every read endpoint; the
syslog filter never invokes a shell and passes payloads verbatim.
Mutating File Manager endpoints (write/create/mkdir/delete/rename/copy/
move/chmod/upload and the S3 equivalents) only required a valid JWT, so a
viewer account (files.write=false) could write and delete files. They now
require the files.write permission via permission_required(). Read
endpoints are unchanged. Adds regression tests covering both advisories.
Co-authored-by: AruvasgaChithan <AruvasgaChithan@users.noreply.github.com>
- backend: StorageProviderService.s3_* (browse/read/write/upload/delete/
presigned-get) + /api/v1/files/s3/* routes, reusing the configured backup
bucket — no new credentials.
- frontend: an "S3 bucket" target in the File Manager. UI paths stay slash-rooted
and map to S3 keys server-side, so breadcrumbs/history work unchanged; virtual
folders from prefixes; presigned-URL download. Folders/rename/permissions are
disabled and the local-only sidebar is hidden on S3. TargetPicker gained a
backward-compatible extraOptions prop.
- docs: mark #14/#15 done (Phase 4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Broad refactor and hardening across backend and frontend:
- Standardize API error responses by removing legacy `success` flags and returning concise error payloads.
- Replace inline admin decorators with centralized middleware import (admin_required) in DB and Docker APIs.
- DatabaseService: add safer MySQL execution helpers and secure user-creation flow (hex-encoding, env-pwd usage) and improve pg drop to validate identifiers.
- Models: add eager-loading (subquery) for relationships and add indexes to frequently queried columns to reduce N+1 and improve performance.
- DockerService: add structured logging and error messages for better observability when Docker commands fail.
- Config: tighten ProductionConfig cookie settings (secure, httponly, samesite).
- Frontend: migrate many .less styles to .scss, add new security UI components and API client modules, update package metadata accordingly.
- Misc: add APP_IMPROVEMENTS.md to .gitignore.
These changes improve security, reliability, observability, and frontend styling consistency.
- Add get_all_disk_mounts() using psutil for multi-mount support
- Add analyze_directory_sizes() for directory size analysis
- Add get_file_type_breakdown() for file categorization by type
- Add recursive helpers with depth limits for performance
- New endpoints: /files/disk-mounts, /files/analyze, /files/type-breakdown
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implemented backend API endpoints and services for file management and FTP server control, including file browsing, editing, upload/download, and FTP user management. Added corresponding frontend pages, components, and API integrations for File Manager and FTP Server features. Updated styles and documentation to reflect new functionality.
Juan Denis