328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.
The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
GHSA-r4q7-x795-vr4j was one forgotten check, but the audit behind it found
the same shape across the surface: `@jwt_required()` authenticates, it does
not authorize, and ServerKit has no default-deny to catch the difference.
This closes the routes that audit left open — fleet-monitor alert
acknowledge/resolve, recycle-bin restore, nginx test/diff, registrar
sync/test, server-template drift checks, uptime tracking on/off, server ping
and remote compose ps/logs, refresh-capabilities, template
test-db-connection, deploy branches, buildpack detect — plus three inline
gaps: a viewer could widen their own API key's scopes, the demo-deploy flag
was doing double duty as an authorization one, and tagging a shared resource
had no write check at all while attaching one did.
`_ensure_group_mutable` only ever blocked *system* queue groups, so every
queue_bus mutation now also calls `_ensure_group_accessible` (owner-or-admin)
— renaming, draining or deleting another user's group was open to any
authenticated caller.
The rbac policy decorators authenticate through `auth_required()`, so they
replace `@jwt_required()` rather than stack under it; stacking rejects
API-key callers before the policy can authorize them
(tests/test_auth_decorator_boundaries.py enforces this). Where a route then
needed the caller it reads `get_current_user()` — `get_jwt_identity()` raises
for an API-key request. The decorators also carry a `_sk_authz` marker so a
static test can see the decision from the source.
`migrations.get_migration_history` moves to `require_admin_user()` for the
same reason its siblings did: the inline `User.ROLE_ADMIN` check it kept was
already reading two names this module no longer imports, so the route raised
NameError for every caller, admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-one imports whose last caller left during the plan-76/77 sweeps:
mostly 'import subprocess' in services now routed through the shared
runners, plus time/requests in the CachedRemoteIndex adopters, threading
in agent_gateway, os in api/system, User in api/bandwidth, run_privileged
in nginx_advanced_service, and the dead SENSITIVE_SETTING_PARTS alias in
audit_service (zero importers). Each removal AST-verified unused before
deletion; all modules compile and the app factory boots.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 76 milestone B, drift/bug row 4. 43 API handlers answered HTTP 500 from
their own `except Exception`, so the global 500 handler never ran — and that
handler was the ONLY caller of error_log_service.record_error(source='backend').
Every one of those crashes was absent from /monitoring/errors, and the page read
empty precisely because someone had "handled" them.
35 of the 43 also returned str(exc) to the caller. Two returned the full
traceback (templates.py, which also print()ed it), and several logged with a
locally-minted uuid4 'ref' that correlated with nothing but a log line.
The recording side of the 500 handler moves to app/error_reporting.py, so it is
importable rather than reachable only by not-catching:
record_unexpected(exc) -> logs, rolls the session back, records, returns
the request id
unexpected_response(exc) -> the above plus the documented crash body
The 500 handler now calls the same pair, so there is one recorder rather than
one recorder and 43 routes that quietly opted out of it.
Caller-visible change on those endpoints: the body is now the documented
{'error': 'Internal server error', 'status': 500, 'code': 'internal_error',
'request_id': ...} instead of the exception text. That is the point — an
unexpected exception's text is written for whoever reads the log and routinely
carries connection strings, absolute paths, and SQL. The request id is the part
a caller can act on, and it matches the row in the error log. No frontend code
read the 'ref' or 'trace' keys this drops (checked).
Two sites keep their own wording and call record_unexpected instead: sso.py's
login failures should not read as a generic crash on the login page.
docker.py's bulk endpoint keeps its per-item `results` — the caller's half-done
work survives, only the exception text is dropped.
Ratchet: tests/unreported_crash_census.py + its test, held at **0**, not a
legacy baseline to shrink. It counts blanket handlers that answer 500 without
calling either reporter, and is scoped to app/api — a service catching an
exception and returning a value is making a domain decision, not declaring an
HTTP crash. Verified non-vacuous: an injected handler fails it, and it passes
again on revert.
Measured and deliberately still open: 56 API handlers swallow a crash and
answer 200. A failure reported as success is worse, but converging it is
milestone C's response-envelope work, not this row.
Gate: tests/test_unexpected_error_reporting.py (11, including the two that pin
why the old shape lost the crash), test_unreported_crash_ratchet.py (9), and
the 66 test files covering every touched module: 1105 passed, 7 skipped, 1
pre-existing Windows-only failure (test_fleet_doctor's socket.gaierror).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 75 §G1, the remaining 33 files. Every raw subprocess call site in
backend/app that can go through a door now does. What is left is what
should be:
8 app/utils/system.py the doors themselves
8 database_service mysqldump|gzip, gunzip|mysql, pg_dump|gzip,
gunzip|psql — a pipe is two processes
7 build/docker/db_sync/ftp streaming output, gzip pipes
1 log_service tail -f
2 serverkit-analytics sibling repo; see below
Extensions were migrated in BOTH trees. builtin-extensions/ is the source
and backend/app/plugins/ the live copy, compared by normalised hash;
test_builtin_extension_drift stays green. serverkit-email has both sides
tracked, serverkit-ftp and serverkit-cloud-provision only the source.
serverkit-analytics (2 calls) is deliberately untouched: it has no source
in builtin-extensions/ at all, and backend/app/plugins/serverkit-analytics
is a gitignored materialised copy. Editing it would be a change that
cannot be committed and is overwritten on the next sync — that fix belongs
in the serverkit-analytics repo.
Exit-code semantics are preserved wherever they carry meaning, with a
comment at each site, because they are the thing a mechanical migration
destroys: clamscan 1 = INFECTED, grype 1 = vulnerabilities FOUND, yara
1 = matches, cron_runner 127 = never launched. Those read
result['returncode'], never result['success'].
run_checked gained two things real call sites needed: it yields when the
caller directs the streams itself (writing straight to a log file), rather
than adding capture_output on top and raising ValueError; and text/errors
pass through, so site_importers/ssh keeps decoding with errors='replace'
instead of gaining a UnicodeDecodeError on a bad byte.
A regression this migration introduced, caught by the suite and fixed:
install_profile_service._docker_usable() is deliberately TRI-STATE —
None means the probe could not run, because inside LXC the docker client
is usually present while the daemon never started, and callers "must not
treat a broken probe as proof of absence". Writing it as
run_checked(...)['success'] turned an exec failure into False, i.e. "docker
is not usable" — a fact nobody established. That is the §A violation this
plan exists to remove, reintroduced by the migration meant to prevent it.
test_probe_honesty_doctor caught it. Every other run_checked(...)['success']
caller was then audited; the rest are genuinely two-state, and their
originals also returned False on a missing binary.
Seven more slips of the same shape were caught before running by an AST
check for `x = run_checked(...)` followed by `x.stdout/.stderr/.returncode`
— a dict read as an object, which raises only on the failure path, so
happy-path tests stay green.
api_controller_boundary_baseline.json: pure deletion. Routing api/docker.py
through DockerService removed the last 6 subprocess crossings, so the
`subprocess` category is gone from the API layer entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Proactive hardening from an internal audit of the same broken-authorization
class as GHSA-6w78-q5vm-rfmh (authentication checked, app/workspace-level
authorization skipped):
- deployment_jobs: job detail/logs/retry resolve the job and gate on
can_access_app / can_operate_app (app-less jobs: requester or admin);
global job listing is admin-only
- secrets_webhooks: every vault/secret/webhook-endpoint/delivery by-id
route resolves to its workspace and 404-seals non-members; secret
reveal and signing-secret regeneration require workspace owner/admin;
dead local admin_required placeholder removed
- shared_resources: variable-group CRUD and attach/detach gate on the
resolved scope (workspace role / can_edit_app / can_operate_app),
closing deploy-time env injection into foreign apps
- queue_bus: message receive/complete/fail/list on system-owned groups
are admin-only; user-owned groups restricted to their owner
- docker: raw container inspect (exposes env vars) and container logs
are admin-only, matching Decision 7; per-app logs stay app-gated
- databases: raw SQL query endpoints, host-level metadata reads, and
process lists are admin-only; per-app managed database flows unchanged
Adds 4 persona-matrix regression suites (74 tests) and bumps the
test-count ratchet.
Block stopping/restarting/removing ServerKit's own Docker containers and surface that in the UI. Backend: add DockerService helpers (is_protected_name / is_protected_container), route guard (_reject_if_protected) to reject destructive actions, annotate list_containers with a protected flag, and update cleanup logic. Add tests (backend/tests/test_docker_protection.py). Frontend: hide/replace lifecycle controls for protected containers (lock badge + styles), add Organization tab group (ORG_TABS) and migrate Projects/Shared Variables/Workspaces into a TabGroupLayout with topbar action hooks; update related styles and sidebar item matching.
Backend: add POST /docker/containers/stats and DockerService.get_containers_stats to fetch resource metrics for multiple containers in a single docker stats call (parses JSON lines and returns a map keyed by ID/Name). Validates input and logs errors.
Frontend: refactor Docker page into a two-column workspace with a left sidebar for targets, resources and maintenance, and a main area with workbar and panel. Add support for multiple targets (default local), fetch available servers, and expose selection UI. Implement bulk stats fetching via api.getContainersStats for local hosts, periodic refresh, per-container stats caching, sorting controls for the container list (status/name/image/cpu/memory/created), and improvements to the container inspector (drawer behavior, loading timer, close action). Update API client with getContainersStats and adjust container stats parsing to handle multiple shapes.
Styles: add comprehensive SCSS for the new workspace, sidebar, inspector drawer, inventory and sort control, and responsive behavior.
Add backend support for listing Docker Compose projects: new /docker/compose/list endpoint and DockerService.compose_list with a JSON-parsing path and a fallback that builds projects from container labels for older compose binaries. Fix RemoteDockerService to use DockerService.get_container for remote/local inspection. Large frontend updates: DashboardLayout supports full-page routes; Databases page received a visual refresh with engine status badges and a summary grid; Docker page refactored extensively — normalize container fields, consistent ID/name/image helpers, table-based resource list, inspector sidebar (ports/mounts/env), resource bars, improved logs/exec handling, and disabled actions for remote targets. Compose tab now uses the new compose list API and streamlines compose actions. File Manager now keeps current path in the URL (search params), tracks previous target changes, and manages selection state more robustly. Miscellaneous UX fixes and icon updates across pages.
Broad refactor and hardening across backend and frontend:
- Standardize API error responses by removing legacy `success` flags and returning concise error payloads.
- Replace inline admin decorators with centralized middleware import (admin_required) in DB and Docker APIs.
- DatabaseService: add safer MySQL execution helpers and secure user-creation flow (hex-encoding, env-pwd usage) and improve pg drop to validate identifiers.
- Models: add eager-loading (subquery) for relationships and add indexes to frequently queried columns to reduce N+1 and improve performance.
- DockerService: add structured logging and error messages for better observability when Docker commands fail.
- Config: tighten ProductionConfig cookie settings (secure, httponly, samesite).
- Frontend: migrate many .less styles to .scss, add new security UI components and API client modules, update package metadata accordingly.
- Misc: add APP_IMPROVEMENTS.md to .gitignore.
These changes improve security, reliability, observability, and frontend styling consistency.
Introduces backend/app/paths.py to centralize all key directory paths, allowing overrides via environment variables for local development. Updates all backend services and APIs to use these centralized paths instead of hardcoded values. Adds dev path override examples to backend/.env.example and updates .gitignore to exclude dev-data. Also changes frontend dev server port to 5274 and updates related scripts and CORS settings for consistency.
Updated log fetching logic in LogsTab to handle both array and string formats for Docker and regular app logs, ensuring logs are displayed correctly regardless of backend response structure.
Add backend and frontend foundations for ServerKit, including Flask API structure, SQLAlchemy models, service layers, and React app with authentication context and service hooks. Implement core API endpoints for authentication, application management, Docker, databases, and system operations. Update README with project overview, tech stack, and setup instructions. Add development roadmap and example environment configuration.
Juan Denis