The first pass left routes that hand the body straight to a service
(create_zone(data), MonitorService.create(data), ...). Those services
read it as a dict, so a body of null, [] or a bare string still
crashed them into a 500. Agent long-poll, external webhook receivers
and the routes that already type-check their body are left alone.
328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.
The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
The explorer showed live sessions but nothing about which queries
cost the most or whether the cache was working. A container's new
Insights tab lists top queries by total time (pg_stat_statements,
performance_schema digests) with connections in use and the buffer
cache hit ratio, and says why a list is empty. PostgreSQL's
pg_stat_statements is switched on through the config tuner, keeping
existing preloads, so its rollback undoes it.
First of the regret-ranked adoptions: a plain delete now tombstones the
row (migration 088; no unique constraints, so no partial-index rewrite).
The database content and its managed BackupPolicy survive the bin window
— restore brings everything back intact; PURGE removes the policy but
NEVER drops database content; the explicit ?drop=true path stays
immediately destructive. Active read paths (list/get/adopt/manifest/env
references/app detail) switch to query_active(); the backup executor
deliberately stays tombstone-blind so bin-parked data keeps backing up.
Server (98 query sites, fleet enrollment + HMAC secrets) is the next
ranked adoption — deliberately its own pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 76 milestone A, drift/bug row 2 — scheduled as a fix, not a refactor,
because it is a behaviour difference shipping today.
rbac.get_current_user() is the only lookup that knows about both auth
mechanisms: it returns g.api_key_user for an X-API-Key request and falls back
to the JWT. 73 call sites answered "who is calling" with
`User.query.get(get_jwt_identity())` instead, behind 19 near-identical private
helpers (12 named get_current_user(), shadowing the real door).
The audit that scheduled this row predicted those return None for API-key
callers. Measured, it is worse: auth_required() deliberately skips
verify_jwt_in_request() once the key middleware has authenticated, so there is
no JWT context and get_jwt_identity() *raises*. Nine handlers reached through
an API-key-capable policy decorator therefore returned 500 to every API-key
caller:
bandwidth.get_app_bandwidth @viewer_required
servers.set_server_workspace @developer_required
servers.create_server @developer_required
sso.get_sso_config / update_provider_config / update_general_settings
source_connections.update_{github,gitlab,bitbucket}_admin_config
source_connections.complete_github_app_manifest
databases.execute_{mysql,pg,sqlite,docker}_query (see below)
Three service helpers had the same lookup inside a blanket `except Exception`,
which converted the raise into a silent user_id=None — the write landed and
the audit trail lost the actor (shared_resource_service, cf_ops_change_service,
secrets_webhooks).
The four databases.execute_*_query sites were a second admin check *inside* an
already-@admin_required route: dead code that always passed for JWT callers and
crashed for API-key ones. They now call require_admin_user() rather than
re-deriving the answer.
Also folds in a sixth _require_admin() found in the cloudflare-ops extension
source (29 gates, the same inverted contract as jobs.py), and fixes
serverkit-gui's blueprint at its sibling-repo source rather than in this repo's
live copy.
Ratchet (second-wave rule 1 — a migration without one regrows):
tests/identity_door_census.py counts both bypass shapes by AST, direct and the
two-line `uid = get_jwt_identity(); User.query.get(uid)` variant. The ceiling is
87, all of them on @jwt_required()-only routes where reading the JWT is correct
today; the ratchet's third assertion holds the API-key-capable population at
exactly zero, so migrating one of those routes to a policy decorator without
fixing its lookup now fails. Verified non-vacuous: an injected bypass fails both
assertions and passes again on revert.
The first wave's controller-boundary baseline is regenerated — this removes 74
persistence crossings (593 -> 519), a debt reduction with no new entries.
Gate: tests/test_api_key_identity.py (7, each red before the fix),
test_identity_door_ratchet.py (10), test_api_controller_boundaries.py (4), plus
the 65 test files covering every touched module: 1024 passed, 10 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Proactive hardening from an internal audit of the same broken-authorization
class as GHSA-6w78-q5vm-rfmh (authentication checked, app/workspace-level
authorization skipped):
- deployment_jobs: job detail/logs/retry resolve the job and gate on
can_access_app / can_operate_app (app-less jobs: requester or admin);
global job listing is admin-only
- secrets_webhooks: every vault/secret/webhook-endpoint/delivery by-id
route resolves to its workspace and 404-seals non-members; secret
reveal and signing-secret regeneration require workspace owner/admin;
dead local admin_required placeholder removed
- shared_resources: variable-group CRUD and attach/detach gate on the
resolved scope (workspace role / can_edit_app / can_operate_app),
closing deploy-time env injection into foreign apps
- queue_bus: message receive/complete/fail/list on system-owned groups
are admin-only; user-owned groups restricted to their owner
- docker: raw container inspect (exposes env vars) and container logs
are admin-only, matching Decision 7; per-app logs stay app-gated
- databases: raw SQL query endpoints, host-level metadata reads, and
process lists are admin-only; per-app managed database flows unchanged
Adds 4 persona-matrix regression suites (74 tests) and bumps the
test-count ratchet.
SHOW FULL PROCESSLIST / pg_stat_activity normalized per server or
container, with admin-only kill/terminate and 5s auto-refresh tabs in
the Database Explorer.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persist the databases ServerKit provisions (and let you adopt existing ones) so
backups, connection strings, and the UI have durable state -- beside, not
replacing, the live-introspection explorer. Not a DBaaS (no pooling/replicas).
- ManagedDatabase model (engine/name/host/port/origin/owner; admin secret
Fernet-encrypted, unique per engine+host+name) + migration 044.
- managed_database_service: record_provisioned (additive on the create routes),
adopt (idempotent), build_connection_uri (real URI, masked by default),
sync_state (drift detection), delete (optional drop), protect.
- Backup FK fix (the headline): a managed-DB BackupPolicy uses
target_id = managed_databases.id plus a `managed` marker, and _resolve_target
resolves the descriptor (incl. decrypted secret) from the row. Legacy
JSON-descriptor policies keep resolving unchanged (backward compatible).
- API under /api/v1/databases/managed (list/create/get+sync/delete?drop=/adopt/
connection-uri/protect); the connection-uri reveal is admin-only and audited.
- Frontend: a "Managed" panel on the Database Explorer (connection-string
reveal/copy, one-click Protect, Untrack/Drop).
- Tests: provisioning persists a row, adopt idempotent, connection URI per
engine + masking, sync_state drift, protect resolves via the real FK, legacy
policy still resolves, API masking/admin-gating/audited reveal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finish #33's two named workspace-scoping slices, backend-only (no schema
migration, no frontend changes):
- Domains: get_domains scopes through the parent Application via scope_query
(own + granted, workspace-narrowed); also surfaces granted apps' domains,
closing a gap vs the per-domain routes. Admin-no-context path unchanged.
- Databases: the app-linked /docker/databases enumeration scopes via
scope_query; host-level MySQL/PG/SQLite stay global system resources
(documented inline).
- Role reconciliation: WorkspaceService.effective_role + can_write_in_workspace
intersect global User.role with per-workspace WorkspaceMember.role,
narrow-only (a workspace role can only reduce capability; system admin never
capped). Enforced at apps/servers create + reassign seams (viewer -> 403,
only when a workspace context is active). Reconciled role surfaced on
GET /workspaces (my_role/my_effective_role).
6 new tests; full backend suite 193 passed / 12 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker_mysql_get_tables returned [] on both a genuinely empty database and a
container/auth failure, so the explorer rendered "Empty" either way and silently
hid broken connections. It now returns {connected, tables, error}; the API
passes that through, and the source tree shows "No tables yet" for a connected
but empty database vs the real reason when the container exec/auth fails.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expose and surface containerised databases and introduce managed app base port control. Backend: add endpoint to list databases across all Docker apps, robustly resolve compose container names, include docker DB user when querying, compute migration needs from live heads and self-heal orphaned alembic_version entries by stamping to head, and add the managed_app_base_port system setting. Template service now honours the managed_app_base_port and WordPress template default HTTP port updated to 8300. Frontend: add react-icons brand icons and DatabaseBrands component, show Docker-hosted DBs under their engine nodes with a docker badge, platform-aware console shortcut label, pass DB user to docker table/query API calls, add API client header merge fix, UI for managing base port in settings, enhanced migration history UI (backup/apply/resync flows, confirm dialog, toasts), and various style tweaks and color additions.
A grant now reaches every app-related blueprint, not just apps.py + WordPress, so
a shared application is fully manageable by its grantees.
- Promoted the access helpers to a shared seam: ResourceGrantService.can_access_app
(owner/admin/any grant -> reads) and can_edit_app (owner/admin/editor grant ->
mutations); apps.py now delegates to them.
- Swept 33 ownership checks across 6 blueprints (categorized read vs mutate via a
parallel mapping workflow): builds.py (12), domains.py (7, incl. SSL + domain
CRUD), deploy.py (6), private_urls.py (5), templates.py (2), databases.py (1).
Reads -> any grant; mutates -> editor grant.
- Fixed the same latent int-vs-str ownership bug in all 33 (now user.id).
Deferred: python.py's single get_app_or_404 helper gates read+mutate+delete
together and needs per-method splitting before it's safe to make grant-aware.
Verified: cross-blueprint integration test (grantee reaches builds deployments;
stranger 403), full suite 131 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Broad refactor and hardening across backend and frontend:
- Standardize API error responses by removing legacy `success` flags and returning concise error payloads.
- Replace inline admin decorators with centralized middleware import (admin_required) in DB and Docker APIs.
- DatabaseService: add safer MySQL execution helpers and secure user-creation flow (hex-encoding, env-pwd usage) and improve pg drop to validate identifiers.
- Models: add eager-loading (subquery) for relationships and add indexes to frequently queried columns to reduce N+1 and improve performance.
- DockerService: add structured logging and error messages for better observability when Docker commands fail.
- Config: tighten ProductionConfig cookie settings (secure, httponly, samesite).
- Frontend: migrate many .less styles to .scss, add new security UI components and API client modules, update package metadata accordingly.
- Misc: add APP_IMPROVEMENTS.md to .gitignore.
These changes improve security, reliability, observability, and frontend styling consistency.
Implements endpoints for managing and querying MySQL/PostgreSQL databases running in Docker containers, including listing containers, databases, tables, and executing queries. Adds endpoints for retrieving app logs and real-time status for Docker apps. Updates template variable processing to flag auto-generated and hidden variables. Extends DatabaseService with Docker MySQL/PostgreSQL discovery and query utilities.
Implements backend API endpoints and service logic for executing SQL queries (MySQL, PostgreSQL, SQLite) with readonly enforcement and table structure retrieval. Adds user notification preferences model and API for per-user alert settings. Extends notification service to support email notifications with HTML templates. Introduces a frontend QueryRunner component for interactive SQL execution, table browsing, and result export. Updates related frontend and backend files to support new features.
Introduces new API endpoints and services for backups, monitoring, and Git deployment. Adds security middleware, account lockout on failed logins, and improves domain/SSL management. Updates Dockerfiles, deployment configs, and frontend to support new features and UI components.
Add backend and frontend foundations for ServerKit, including Flask API structure, SQLAlchemy models, service layers, and React app with authentication context and service hooks. Implement core API endpoints for authentication, application management, Docker, databases, and system operations. Update README with project overview, tech stack, and setup instructions. Add development roadmap and example environment configuration.
Juan Denis