328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
Extensions can now contribute walkthroughs via contributions.walkthroughs
in plugin.json. The backend validates definitions against a shared
contract (route, signal, manual, and check completion types), transports
them through the dynamic contribution envelope, and persists custom
walkthroughs from the Studio. Includes the JSON Schema, SDK version gate,
and extension scaffold support.
The 1791-line security omnibus splits along tool lines and the install-gated
surfaces leave core for five standalone extensions - serverkit-clamav
(ClamAV+YARA scanning, quarantine, scan jobs), serverkit-fail2ban
(management UI; probes stay core on Fail2banJailService), serverkit-lynis,
serverkit-auto-updates, and serverkit-image-scan (headless CVE/SBOM
scanning; models stay core per the plan 52 D1 data seam). Each mounts its
routes back at the same /api/v1/security prefix when installed, so route
shapes never change; absent extensions mean plain 404s, no dead UI.
Core keeps the zero-host-package baseline: status/config, legacy integrity,
suspicious activity/events, SSH keys, IP lists, the audit (fail2ban check
now probes via the jail service; updates check was already self-contained),
scoped FIM, and the scan/alert logs + notification helpers extensions write
through. Job handlers for security.malware_scan/lynis_scan/image_scan now
come exclusively from the extensions' manifest jobs key.
Security page becomes core-tabs-only; extension tabs contribute into the
security tab group (3a groupId). Overview posture drops the hardcoded
ClamAV rows and offers the Marketplace when no security extension is
installed. Upgrade parity: run_registry_auto_install() installs each
extension from the registry once, gated on the host tool actually being
present, fail-soft until the registry entry is published.
Wizard round 2 (Ph5): a security-posture chooser (minimal/recommended/
hardened) on the Summary step, catalog-resolved so unpublished slugs never
render dead installs; persisted as onboarding_security_posture.
Moved test suites re-home to the extension repos; panel keeps
test_security_suite_extraction.py (18 proving tests: absent-shape 404s,
lean summary, jail-service probes, migration gates, posture wiring).
docs/API_SURFACE.md regenerated (34 operations moved out).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SDK 1.2.0 -> 1.3.0 (additive).
Auditing the extension surface for plan 79 found the localization gap I
expected and two pre-existing bugs I did not.
The gap: extensions had no way to translate anything. The SDK now re-exports
the HOST's translation surface -- useTranslation, Trans, t, useLocale,
useLabel, useFormat and the intl door -- and i18next/react-i18next join the
shared vendor singletons. That sharing is the load-bearing part: an extension
bundling its own i18next would get a separate, uninitialised instance, so
every t() in it would render its English default forever and nothing would
report it. Exactly the "two React copies" hazard vendorShare already documents,
and exactly the failure mode this plan hit twice in core. They also join
dedupe/optimizeDeps for the same reason React is there.
The bugs, both caused by the same duplication:
- The import map listed `react` and `react-router-dom`, but no shim existed
for either. A runtime extension importing bare `react` 404'd at load.
- The serverkit-sdk shim re-exported 26 of the SDK's 71 exports. An extension
importing Button, Modal, EmptyState, formatBytes or timeAgo from the SDK got
`undefined`. The shim's own comment claimed parity was "asserted by the
frontend build + backend test_sdk_contract"; nothing asserted it.
Three files independently restated the same module list -- the import map, the
namespace capture, and one hand-written shim per module -- so they drifted.
Now vendorManifest.js is the list, generate-vendor-shims.mjs derives every
shim's exports (from the real package's namespace, or from the SDK entry's own
export statements -- never typed by hand), the import map is built from the
same manifest, and `--check` fails the build on drift. 26 -> 85 exports on the
SDK shim, 6 -> 8 shims, all present.
Manifest contributions gain an optional `labelKey` beside `label` for nav, tabs
and command-palette entries, resolved at render by the same translateLabel core
uses -- data cannot call t(), and a label resolved at import would translate
once and never follow a switch. Omitting it renders `label` verbatim, which is
what every existing extension does, so nothing breaks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 76 milestone B, drift/bug row 4. 43 API handlers answered HTTP 500 from
their own `except Exception`, so the global 500 handler never ran — and that
handler was the ONLY caller of error_log_service.record_error(source='backend').
Every one of those crashes was absent from /monitoring/errors, and the page read
empty precisely because someone had "handled" them.
35 of the 43 also returned str(exc) to the caller. Two returned the full
traceback (templates.py, which also print()ed it), and several logged with a
locally-minted uuid4 'ref' that correlated with nothing but a log line.
The recording side of the 500 handler moves to app/error_reporting.py, so it is
importable rather than reachable only by not-catching:
record_unexpected(exc) -> logs, rolls the session back, records, returns
the request id
unexpected_response(exc) -> the above plus the documented crash body
The 500 handler now calls the same pair, so there is one recorder rather than
one recorder and 43 routes that quietly opted out of it.
Caller-visible change on those endpoints: the body is now the documented
{'error': 'Internal server error', 'status': 500, 'code': 'internal_error',
'request_id': ...} instead of the exception text. That is the point — an
unexpected exception's text is written for whoever reads the log and routinely
carries connection strings, absolute paths, and SQL. The request id is the part
a caller can act on, and it matches the row in the error log. No frontend code
read the 'ref' or 'trace' keys this drops (checked).
Two sites keep their own wording and call record_unexpected instead: sso.py's
login failures should not read as a generic crash on the login page.
docker.py's bulk endpoint keeps its per-item `results` — the caller's half-done
work survives, only the exception text is dropped.
Ratchet: tests/unreported_crash_census.py + its test, held at **0**, not a
legacy baseline to shrink. It counts blanket handlers that answer 500 without
calling either reporter, and is scoped to app/api — a service catching an
exception and returning a value is making a domain decision, not declaring an
HTTP crash. Verified non-vacuous: an injected handler fails it, and it passes
again on revert.
Measured and deliberately still open: 56 API handlers swallow a crash and
answer 200. A failure reported as success is worse, but converging it is
milestone C's response-envelope work, not this row.
Gate: tests/test_unexpected_error_reporting.py (11, including the two that pin
why the old shape lost the crash), test_unreported_crash_ratchet.py (9), and
the 66 test files covering every touched module: 1105 passed, 7 skipped, 1
pre-existing Windows-only failure (test_fleet_doctor's socket.gaierror).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 76 milestone A, drift/bug row 2 — scheduled as a fix, not a refactor,
because it is a behaviour difference shipping today.
rbac.get_current_user() is the only lookup that knows about both auth
mechanisms: it returns g.api_key_user for an X-API-Key request and falls back
to the JWT. 73 call sites answered "who is calling" with
`User.query.get(get_jwt_identity())` instead, behind 19 near-identical private
helpers (12 named get_current_user(), shadowing the real door).
The audit that scheduled this row predicted those return None for API-key
callers. Measured, it is worse: auth_required() deliberately skips
verify_jwt_in_request() once the key middleware has authenticated, so there is
no JWT context and get_jwt_identity() *raises*. Nine handlers reached through
an API-key-capable policy decorator therefore returned 500 to every API-key
caller:
bandwidth.get_app_bandwidth @viewer_required
servers.set_server_workspace @developer_required
servers.create_server @developer_required
sso.get_sso_config / update_provider_config / update_general_settings
source_connections.update_{github,gitlab,bitbucket}_admin_config
source_connections.complete_github_app_manifest
databases.execute_{mysql,pg,sqlite,docker}_query (see below)
Three service helpers had the same lookup inside a blanket `except Exception`,
which converted the raise into a silent user_id=None — the write landed and
the audit trail lost the actor (shared_resource_service, cf_ops_change_service,
secrets_webhooks).
The four databases.execute_*_query sites were a second admin check *inside* an
already-@admin_required route: dead code that always passed for JWT callers and
crashed for API-key ones. They now call require_admin_user() rather than
re-deriving the answer.
Also folds in a sixth _require_admin() found in the cloudflare-ops extension
source (29 gates, the same inverted contract as jobs.py), and fixes
serverkit-gui's blueprint at its sibling-repo source rather than in this repo's
live copy.
Ratchet (second-wave rule 1 — a migration without one regrows):
tests/identity_door_census.py counts both bypass shapes by AST, direct and the
two-line `uid = get_jwt_identity(); User.query.get(uid)` variant. The ceiling is
87, all of them on @jwt_required()-only routes where reading the JWT is correct
today; the ratchet's third assertion holds the API-key-capable population at
exactly zero, so migrating one of those routes to a policy decorator without
fixing its lookup now fails. Verified non-vacuous: an injected bypass fails both
assertions and passes again on revert.
The first wave's controller-boundary baseline is regenerated — this removes 74
persistence crossings (593 -> 519), a debt reduction with no new entries.
Gate: tests/test_api_key_identity.py (7, each red before the fix),
test_identity_door_ratchet.py (10), test_api_controller_boundaries.py (4), plus
the 65 test files covering every touched module: 1024 passed, 10 skipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 55 task 6, backend half. Installing a plugin's requirements runs pip with
the backend's privileges, and a setup.py hook is arbitrary code, so it is
opt-in and off by default -- correct, and the part that was missing is that
nothing ever said so. The skip was a logger.warning: the requirements.txt was
written next to the extension and the panel showed nothing, so an extension
could sit there running with imports it does not have and the operator had no
way to find out short of reading the install log.
pending_requirements() answers it: whether a file is pending, its contents, its
path, the parsed package lines, and the exact env var that changes the
behaviour. GET /plugins/<id>/requirements exposes it, admin-only.
Strictly read-only. Nothing here installs anything -- the plan is explicit that
this surface introduces no auto-install behaviour change -- and a test pins
that by failing if _install_requirements is reached.
The env check the installer inlined is now plugin_pip_enabled(), shared by both
sides. If the surface and the installer ever disagreed about what "enabled"
means, the page would describe a behaviour the installer does not have.
Two honest edges: the content is capped at 64KB before being echoed into a
browser, and pip_enabled is reported alongside pending, because a file written
while the opt-in was off survives turning it on -- the surface reports both
facts rather than guessing which applies.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 55 task 7, backend half. A manifest's `permissions` array was
declaration-only: the install dialog showed it and nothing ever compared it
against what the extension went on to do.
The capability gate now records every call through it -- including the
refusals, which are the interesting ones -- so `usage_report(slug)` can answer
declared-versus-observed per extension. Observations are in-memory and
per-process by design: the question is "what has this extension done since the
panel started", and persisting it would outlive the evidence and start reading
like an audit trail, which this is not. `has()` deliberately does not record; it
is a question, not a use, and counting it would fabricate activity for callers
that only checked.
The larger half of this change is admitting what the panel cannot see. Task 7
was specified as hooks at "the shell/docker/db helper entry points", and those
do not exist: the SDK exposes no shell, docker, filesystem or network helper at
all, and hands out `db` as raw SQLAlchemy. An extension wanting any of them
imports the host module directly, where no in-process gate is involved. The one
genuinely gated capability is `agent.command:<action>`, because the SDK is the
only way to reach an agent.
So observability is data, not prose: OBSERVABLE_PREFIXES marks what the gate
truly mediates, and the report marks every other declared permission
`observable: false` rather than showing it as "never used". Absence of evidence
is not evidence of absence here, and a UI claiming otherwise would be inventing
enforcement the panel does not have -- exactly the honesty the plan's D2
decision asks for.
GET /plugins/<id>/permissions returns the report, admin-only: it describes what
third-party code has been reaching for.
The UI surface and the docs follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`contributions.widgets` mounts a component at a host-chosen slot. That is the
wrong shape for the new dashboard, where the user decides what goes on a board
and where. So this adds `dashboard_widgets` alongside it: a registry of
placeable widget TYPES that surface in the widget library, each instance
positioned, sized and configured by the person looking at it.
The two are easy to confuse, so the docstring, the manifest spec and
EXTENSIONS.md all now say plainly which is which, and a test asserts a
dashboard widget never leaks into the fixed-slot list.
SDK_VERSION 1.0.0 -> 1.1.0 (added surface, older extensions unaffected),
bumped in lock-step across the JS export and the backend mirror.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wire the server half of runtime-frontend delivery: get_active_contributions()
now emits panel sdk_version and a 'frontends' descriptor map {slug: {entry,
hashes, sdk_version}} for every active extension whose frontend_entry is a .mjs
ESM bundle. The bundle sha256 is pinned at install time into the plugin config
under the reserved _frontend_hashes key (no migration — reuses config_json);
update_plugin_config() and the config API preserve/hide reserved keys. Add the
extensions.runtime_frontend kill switch (default on). The client loader already
consumed this shape, so an installed extension's UI now renders on a pre-built
panel with no Vite rebuild. Un-skips the restored plan-25 recovery suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Plan 47 Phase 1. The onboarding "Recommended for you" step now installs real
extensions matched to the selected use cases instead of rendering decorative
strings:
- Backend use-case -> slug map + resolver (RECOMMENDED_EXTENSIONS_BY_USE_CASE,
recommend_extensions_for_use_cases) single-sourced and test-guarded against
dead slugs; exposed via GET /api/v1/plugins/recommendations.
- SetupStepSummary renders default-checked install checkboxes with each
extension's real display_name/description and installs source-aware on Finish
(per-item progress, fail-soft: an install failure never blocks setup).
- complete-onboarding persists onboarding_installed_extensions.
- WordPress flagship seeding is now gated on needs_setup(): fresh installs skip
it (the wizard installs it when the WordPress use case is picked); completed
-setup/upgraded installs keep it (zero regression). TESTING bypass keeps the
WordPress suite green; the real gate is covered by dedicated proving tests.
Proving tests: tests/test_onboarding_recommendations.py (12).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the deferred half of #30. InstalledPlugin gains config_json
(migration 045); admin-gated GET/PUT /api/v1/plugins/<id>/config
persists values; plugins read them via the new plugins_sdk.config(slug)
accessor. The Marketplace Installed tab shows a Configure button for
plugins declaring a config_schema and renders a real form
(string/number/integer/boolean/enum/secret fields, defaults, hints).
Values may hold secrets so they are excluded from to_dict() and only
served by the admin endpoint. Documented in docs/EXTENSIONS.md.
2 new tests (config round-trip + SDK accessor + secret non-leak;
non-object rejection); pipeline suite 18 green; lint 0 errors; build
clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extensions can now contribute a real tab to a core-owned tab group:
a new `tabs` contribution kind ({group, to, label, icon?, end?, order?},
group id == sidebar item id) is merged into the strip by TabGroupLayout
(new groupId prop; core wins `to` collisions), and route contributions
gained a `group` field so ExtensionRoutes collects them per group and
App.jsx mounts them INSIDE that group''s TabGroupLayout — the shared
PageTopbar chrome stays and the Sidebar keeps the group item lit via
contributed matchPrefixes. Wired for files/servers/monitoring.
First consumer: the FTP Server page moves to the serverkit-ftp builtin
extension (tab in the Files group; backend /api/v1/ftp stays core per
D2). Added to CONVERTED_BUILTIN_SLUGS so upgraded panels auto-install
it once; fresh installs find it in the Marketplace.
Proven by the extended contributions-envelope test (26 plugin/platform
tests green; full backend suite 1215 passed), lint 0 errors, build clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
backend/app/services/plugin_service.py:
- Zip Slip defense: new _safe_extract_path() validates every archive
member against the destination root (rejects absolute paths, ../
segments, and entries that resolve outside backend/frontend dest).
All zip extract paths route through it.
- Plugin requirements.txt no longer auto-installs. Running pip on an
untrusted plugin gives setup.py hooks code-exec at the backend's
privilege level. Operators must opt in with SERVERKIT_ALLOW_PLUGIN_PIP=1;
otherwise the file is saved to the plugin dir for manual review.
- New _attach_status_guard() registers a per-blueprint before_request
that returns 503 if the plugin's DB status is not 'active'. Flask
can't unregister blueprints from a running app, so the in-DB status
is now authoritative at request time — disabling a plugin actually
stops serving its routes.
- New _update_plugin_metadata() syncs all mutable manifest fields
(display_name, description, author, homepage, repository, license,
category) on reinstall. Previously these went stale after the first
install.
backend/app/api/plugins.py:
- /install endpoints no longer echo raw exception strings to clients
(could leak paths, lib versions, SQL fragments). Log full traceback
server-side with a short correlation ref returned to the client.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote the serverkit-git extension to own the canonical /git route and migrate legacy /git-ext usage. Adds an Alembic migration that updates the installed_plugins manifest for serverkit-git to the canonical /git contributions. Frontend changes: plugin manifests and build-time plugin list use the new display name and /git routes; plugin entry exports now expose GitExtensionPage (and keep GitExtPage as a compatibility alias); contribution normalization rewrites older plugin contributions that referenced /git-ext to the canonical /git and provides fallback normalization for build-time contributions; App routing adds redirects for legacy git-ext routes to /git and removes the host's hardcoded /git page from core routes; Command Palette and Sidebar code updated to remove the static core Git item and to respect plugin-contributed nav items (including new hidden-item handling and Set-based sidebar helpers). Small backend docstring wording tweak.
Several changes to plugin handling, API client behavior, and the marketplace UI:
- Backend: added explicit '' route decorators for marketplace and plugins endpoints (GET/POST) to ensure routes mount at both '' and '/'.
- Frontend: make Git page accept a basePath prop and update serverkit-git plugin export to render GitPage at /git-ext.
- Auth/API: normalize API base URL, update token-expiry handling to dispatch a global serverkit:auth-expired event (client) and listen for it in AuthContext to clear user state; refresh token logic preserved on JWT 401s.
- Plugins/contributions: add build-time plugin manifest support and fallback for contributions when backend endpoint is unavailable; parse local plugin manifests and tag contributions with plugin slugs.
- Plugin loader: internalize getInstalledPlugins function signature.
- Marketplace UI: remove the submit-extension modal and form, replace with an Import ZIP flow and make tabs controlled via state; remove related SCSS styles.
- API & sockets: fix plugin API URL encoding/format and compute socket URL robustly for dev/prod.
These changes improve offline/dev experience (Vite dev server), make plugin contributions resilient when the backend is absent, and simplify the marketplace extension import workflow.
Introduce fallback audit logging and enrich audit tooling; add builtin extension discovery/install; and update default launcher ports and related docs/config.
Key changes:
- New middleware app.middleware.audit registers an after-request fallback that records authenticated mutating API requests when routes don't emit explicit audit logs.
- Audit model/action constants extended and AuditService enhanced (request context handling, redaction for sensitive settings, optional commit flag, sets g.audit_logged, emits events).
- Routes/services updated to create explicit audit entries for marketplace and plugin operations (create/update/publish/install/uninstall/enable/disable) and SSO/settings flows switched to use AuditService.
- Added tests for audit logging behavior and redaction (backend/tests/test_audit_log.py).
- Plugin service: added BUILTIN_EXTENSIONS_DIR, list_builtin_extensions and install_builtin_extension; plugin API endpoints to list/install builtin extensions.
- Contribution system extended to support layouts and route layout metadata; frontend wiring updated to surface standalone/custom layouts and extension routes.
- Launcher port defaults changed (backend -> 47927, frontend -> 41921). Updated backend .env.example, config CORS defaults, Dockerfiles and docker-compose to use SERVERKIT_BACKEND_PORT, and many docs/README files and frontend envs to reflect new ports.
- Minor frontend cleanups and new builtin extension example (serverkit-git) plus associated manifest and frontend re-export.
These changes improve observability/security for mutating APIs, enable opt-in builtin plugins, and align local dev/run ports with the launcher defaults.
Introduce a first-class plugin contribution system and SDK, plus frontend integration and installer enhancements.
Backend:
- Add /plugins/manifest-spec and /plugins/contributions endpoints to expose the plugin.json contract and merged active-plugin UI contributions.
- Add app/plugins_sdk for plugin authors (current_user, logger, audit, jwt helpers).
- Add ContributionService to aggregate active plugin contributions (nav, routes, page_titles, command_palette, widgets).
- Marketplace now hands off to plugin installer when an extension has a source URL and enforces admin-only installs in the marketplace API.
- PluginService: run lifecycle hooks (install/uninstall) and install declared template dependencies (best-effort, recorded/logged) during plugin installs; surface template/manifest fields on InstalledPlugin.to_dict; surface linked InstalledPlugin status on ExtensionInstall.to_dict.
Frontend:
- Add runtime contributions loader (plugins/contributions.js) with useContributions hook and resolveComponent/getPluginModule helpers.
- Add SDK re-exports for plugin frontends (plugins/sdk/index.js) and Vite alias 'serverkit-sdk'.
- Integrate contributions across UI: Sidebar, CommandPalette, PluginLoader (declared widgets + legacy fallback), PageTitleUpdater, DashboardLayout (refreshContributions), and route injection via new ExtensionRoutes hook loaded in App.jsx.
- Add API client getPluginContributions and adjust PluginLoader discovery patterns.
Other:
- Add .extension-platform-plan/ to .gitignore.
Overall this enables declarative plugin-driven UI integration (sidebar items, SPA routes, page titles, command palette entries, and widgets), lifecycle hooks, and template dependency installs while keeping legacy behavior for older plugins.
Introduce a full log viewer feature and add backend/frontend support for installing plugins from local folders or uploaded zips. Backend: add /plugins/install-local and /plugins/install-upload endpoints (admin-only, 50MB cap for uploads) that reuse existing install pipeline and emit audit logs. Frontend: add LogContent, LogFileList, LogToolbar and logHelpers, integrate into Terminal.jsx with target picker, search, auto-refresh, wrap/line-number prefs and remote-target handling; add styles for the viewer and marketplace install tabs. API: detect FormData in the client to avoid forced JSON Content-Type and add installPluginFromPath/installPluginFromZip helpers. Misc: marketplace UI now supports URL / path / upload install modes and minor SCSS fixes (font-size variable adjustments).
Introduce a plugin installation and management system.
Backend: add InstalledPlugin model, plugin_service with download/install/enable/disable/uninstall, dynamic blueprint registration and frontend manifest generation, and plugins API endpoints (list/get/install/enable/disable/uninstall). Register plugins blueprint and attempt to hot-load installed plugin blueprints at app startup.
Frontend: add PluginLoader component (Vite import.meta.glob) to render plugin widgets, extend Marketplace UI to install/manage plugins, add API client methods for plugins, and update styles for the marketplace plugin UI. This enables installing plugins from GitHub/repos/zips and managing them from the app.
Juan Denis