Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
Migrate all 20 direct agent_registry.send_command call sites (api/servers,
terminal, fleet doctor/repair/fleet incl. the upgrade Thread target,
deployment_runner, survey, tunnel broker/publish, plugins SDK) onto the
shared dispatcher, and widen the ratchet from remote_*_service.py to a
tree-wide AST scan covering builtin-extensions/. The survey test's
transport stub gets the transport's real keyword signature.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 55 task 7, backend half. A manifest's `permissions` array was
declaration-only: the install dialog showed it and nothing ever compared it
against what the extension went on to do.
The capability gate now records every call through it -- including the
refusals, which are the interesting ones -- so `usage_report(slug)` can answer
declared-versus-observed per extension. Observations are in-memory and
per-process by design: the question is "what has this extension done since the
panel started", and persisting it would outlive the evidence and start reading
like an audit trail, which this is not. `has()` deliberately does not record; it
is a question, not a use, and counting it would fabricate activity for callers
that only checked.
The larger half of this change is admitting what the panel cannot see. Task 7
was specified as hooks at "the shell/docker/db helper entry points", and those
do not exist: the SDK exposes no shell, docker, filesystem or network helper at
all, and hands out `db` as raw SQLAlchemy. An extension wanting any of them
imports the host module directly, where no in-process gate is involved. The one
genuinely gated capability is `agent.command:<action>`, because the SDK is the
only way to reach an agent.
So observability is data, not prose: OBSERVABLE_PREFIXES marks what the gate
truly mediates, and the report marks every other declared permission
`observable: false` rather than showing it as "never used". Absence of evidence
is not evidence of absence here, and a UI claiming otherwise would be inventing
enforcement the panel does not have -- exactly the honesty the plan's D2
decision asks for.
GET /plugins/<id>/permissions returns the report, admin-only: it describes what
third-party code has been reaching for.
The UI surface and the docs follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The module docstring described only the plumbing a plugin imports, not the six
surfaces it can now contribute to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Everything an extension could do stopped at the machine the panel runs on,
which is at odds with a product built around a fleet of servers. A plugin can
now run an agent command and get its result back.
This is also where agent.command:<action> starts meaning something. The
manifest permission has existed as a declaration — collected in the manifest,
surfaced at install time for consent — with nothing enforcing it, because
plugins called the agent registry directly. Dispatching through the SDK checks
it first, matched verbatim, so a plugin can only run the actions it asked the
operator for by name. The one bundled extension that uses agent commands still
calls the registry directly; retrofitting it onto this is a follow-up.
Nothing underneath is re-implemented: the per-server ACL still decides whether
that server permits the action, and the ServerCommand audit row is still
written — pass user_id so it names a person rather than "the panel".
Three things the wrapper adds over a bare send_command:
- it unwraps the double envelope. A transport-successful reply can carry a
handler failure inside, which reads as success to anyone checking only the
outer flag.
- it fails before dispatch on an offline agent or a missing capability, so the
caller gets a reason instead of an unknown-action error from the far side.
- the default timeout is 45s rather than the registry's 30s. An agent on the
HTTP long-poll fallback can sit ~25s before it even sees the command, which
leaves a 30s budget almost no room to actually run anything.
Failures raise with a .code, so a plugin can tell an offline agent from a
refused one from a timeout, and map them to its own HTTP responses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An extension with something worth backing up had to hand-roll it — no policy,
no cron mirror, no run history, no retention, no offsite copy, no verification,
no restore, and no place in the Protection panel. A registered kind inherits
every one of those, because everything downstream of the target_type switches
is already generic: the run lifecycle, retention (count, age, incremental-chain
ancestry), the API blueprint, and the panel itself, which threads target_type
through without ever switching on its value.
No migration: target_type is an unconstrained String, and the allowed set lived
purely in a Python tuple. A kind supplies resolve() and execute(), plus an
optional restore().
Four switches had to learn about the registry, and the order matters in three
of them:
- validate_target_type is the keystone — every policy route funnels through it,
so accepting a registered kind opens the whole blueprint at once.
- _resolve_target and _execute_backup check the registry BEFORE their core
branches, because both end in an application fallthrough that swallows any
unrecognised type.
- run_restore_job likewise. This is the sharp one: without it a plugin's
archive would be unpacked over an app directory by core's else-branch.
A kind that didn't implement restore is refused at request time with a reason,
rather than queueing a job that takes the policy lock and runs a safety backup
before failing.
The registry normalises what plugins hand back — target_type is forced rather
than defaulted, so a kind cannot describe itself as an 'application' and be
restored as one; execute() must return (path, size, meta) with a real path; and
the metadata retention and the panel read is filled in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sweep was a fixed sequence of core producers, so an extension had nowhere
to say "the thing I manage is unhealthy, and here is what to do about it" — on
the page an operator opens to ask exactly that. A plugin now registers a
provider and its checks appear beside the core ones, with a working Repair
button, and no frontend change: the panel's status pill, repair button and diff
toggle all key off fields rather than a known list of check ids.
Because these run third-party code, the registry gives them two things core's
own producers don't have:
- an error boundary. run() calls its producers bare — each hand-rolls its own
try/except and anything that escapes 500s the whole sweep. A provider that
raises becomes one warning attributed to the plugin instead.
- a time budget. Nothing bounds a core producer today, so a hung one hangs the
synchronous POST /doctor/run for whoever asked. A provider that overruns is
reported as a warning and the sweep moves on. Note the executor is shut down
with wait=False on purpose: its context manager joins the worker on exit,
which would have made the timeout worthless — a test pins that down.
Providers run in a worker thread with the app context carried across, so a
check can query the database the way a core one does.
Contributed keys are namespaced, statuses outside ok/warn/fail are reported as
warnings rather than rendering as a blank pill, and a check claiming to be
repairable without a registered handler is demoted — showing a button that
dispatches nowhere is the state two core checks are already in, and it isn't
worth reproducing. Repairs route back through the registry, which is itself the
allowlist: a crafted ref for an unregistered namespace reaches nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An extension that needed to remember anything had to add a model to core AND an
Alembic migration for it — which is why extension tables like
ext_serverkit_minecraft_* sit in the panel's own migration history. That is a
steep tax on a small extension and it puts plugin schema in the panel's upgrade
path. A plugin now writes to a key/value store instead:
mine = store.for_plugin('serverkit-minecraft')
mine.set('world:1:seed', 8675309)
Values are JSON, so dicts and lists round-trip without hand-serialising, and
rows are namespaced by slug so two plugins cannot collide on a key. Reach for a
real table when you need to query BY the data — this is a key/value store, not
a database, and the docstring says so.
Because the table is shared by every installed plugin, three things are refused
rather than trusted: a value that isn't JSON-serializable, a value over 256 KB,
and an empty or overlong key. Validation happens at set() rather than at commit
— by commit time the error is opaque and the session is already poisoned, so
the next unrelated write fails too.
has() exists because get() cannot distinguish a stored None from a missing key,
and a plugin storing None deliberately would otherwise have no way to tell.
Concurrent writers race on insert; the unique constraint makes that safe and
the loser applies its write to the winner's row instead of failing the caller.
Store rows follow the same rule as extension-declared tables: an ordinary
uninstall leaves them, so a reinstall finds its state where it left it, and
"uninstall and delete data" removes them. The purge runs after the uninstall
lifecycle hook, since a hook may legitimately read its own store on the way out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The omnisearch fan-out was a hardcoded sequence of core entity blocks, so an
extension's objects were absent from the command palette — the one place the
UI tells you to look. A plugin now registers a provider and its rows appear
alongside services, servers and domains, with no frontend change: the palette
already renders unknown types, bucketing them under "Results".
Scoping stays the provider's job, because search invents no ACL — every core
block scopes its own query and the provider is handed the user and workspace
to do the same. The shape, though, is not trusted:
- rows are always tagged with the registered type, so a plugin cannot mint a
row that claims to be a 'service' or a 'vault' (which are trusted to have
been scoped by core);
- the per-type cap is re-applied host-side rather than trusted from the
provider, so one contributed type can't drown the core ones;
- a path must be an in-app router path. The palette hands it straight to
navigate(), so a protocol-relative '//host' would walk the user off the
panel entirely.
A malformed row is dropped on its own, a broken provider degrades to "no rows
of that type" — the same isolation each core block already has.
The lazy accessor in plugins_sdk now carries a table of surfaces rather than a
single special case for deploys, since the other SDK surfaces land the same way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Promoting between WordPress environments ran in the request thread: no
console, no log stream, no retry, and a progress bar the pipeline page had
to keep alive itself. It now runs as a registered deployment kind, so it
inherits the Deploy Console, live logs, retry and a row in Deploy Activity.
POST /promote returns 202 with a job_id instead of blocking, and the UI
opens the console with it.
PromotionJob is kept, not replaced. It carries pre_promotion_snapshot_id,
the rollback pointer, which is WordPress domain a generic DeploymentJob has
no business holding — so the deployment records which promotion rows it
produced instead. They are collected by id watermark rather than from the
service's return value, because promote_* only reports the job on success
and a failed promotion is exactly when someone needs the snapshot.
A full promotion runs two five-step halves and restarts its own numbering
at 1 for the second, so the progress bridge carries an offset across the
reset — otherwise the console's step strip walks backwards halfway through.
Two gaps this exposed in the SDK, both fixed here:
- Plugin kinds never closed their log stream, so they got no per-step
timings, no failure tail and no hint, and no terminal status push. The
handler and the runner also built separate streams, so the timings one
recorded were dropped by the other's close; they now share one per job.
- A handler that reported failure by return value logged nothing, leaving
a failed run with no reason visible in the console.
The console now titles a run from plan.title, so a contributed kind names
itself without a branch per kind in the frontend.
A locked environment is refused by the endpoint rather than by the job:
queueing first would send the user to a console that fails a second later.
The pipeline service still re-checks, since a lock can be taken in between.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DeploymentJobService.run_job dispatched on a hardcoded switch over three
core kinds, so a plugin could not own work that behaves like a
deployment. Its only option was the generic job system, which gives no
step plan, no streaming console and no place in Deploy Activity — the
reason promoting a WordPress environment, which is deploy-shaped in every
respect, is invisible in the one page you open to see what ran.
plugins_sdk.deploys lets a plugin register a kind and start one. The job
is a real DeploymentJob, so it inherits the Deploy Console, live logs,
retry and a row in the feed with no frontend work at all — those surfaces
key off the job, not off any particular kind, and the feed already
renders an unknown kind by its id.
Three things the SDK guarantees, each pinned by a test:
- Core kinds cannot be hijacked. A plugin registering app_deploy would
silently take over real deployments.
- A handler that raises still ends the job visibly failed. Third-party
code must not be able to leave a job spinning at 'running' with no
explanation — the same promise _reconcile_interrupted_jobs makes for a
lost process, extended to a lost handler.
- A handler that finalises the job itself keeps that outcome, even if its
return value contradicts it.
deploys.steps() advances the job through its plan as the handler works,
so the console's pipeline strip and the feed's tick bar fill in, and a
failure records which step failed rather than only a stack trace.
The implementation is deploys_sdk.py, not deploys.py: a submodule of that
name is bound as a package attribute on import and would shadow the lazy
accessor, handing callers the module instead of the SDK.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Completes the deferred half of #30. InstalledPlugin gains config_json
(migration 045); admin-gated GET/PUT /api/v1/plugins/<id>/config
persists values; plugins read them via the new plugins_sdk.config(slug)
accessor. The Marketplace Installed tab shows a Configure button for
plugins declaring a config_schema and renders a real form
(string/number/integer/boolean/enum/secret fields, defaults, hints).
Values may hold secrets so they are excluded from to_dict() and only
served by the admin endpoint. Documented in docs/EXTENSIONS.md.
2 new tests (config round-trip + SDK accessor + secret non-leak;
non-object rejection); pipeline suite 18 green; lint 0 errors; build
clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalize the Notification Bus pattern (thin {id} message -> fat DB row
-> single consumer) into a Job/WorkUnit layer on top of the Queue Bus, so
background work flows through one pipe instead of many ad-hoc threads.
- app/jobs: Job + ScheduledJob models, handler registry, JobService +
ScheduledJobService, one generic JobConsumer, one JobScheduler ticker,
and a plugin SDK surface (app.plugins_sdk.jobs).
- Migrate the 8 per-domain daemon scheduler threads (auto-sync,
snapshot-retention, workflow, health-check, wp-update, api-background,
pairing-prune, registrar-expiry) to ScheduledJob rows + built-in
handlers that reuse the exact prior check functions: cadence and
per-task de-dup unchanged, only the trigger moved.
- Retire the orphaned in-memory BackgroundJobService; repoint
/api/v1/performance/jobs at JobService.
- Add /api/v1/jobs (list/get/cancel/retry/stats; scheduled list/run/
toggle) as a single pane over all background work.
- tests/test_jobs.py: enqueue->run->succeed, dead-letter->failed,
unknown-kind, cancel, retry, scheduler tick/advance, built-ins.
No Alembic migration (tables created via create_all), avoiding a
multi-head clash with the in-flight telemetry work. Plan lives in
docs/ORCHESTRATION_ROADMAP.md (local/git-ignored).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reusable, non-blocking notification layer on top of the Queue Bus:
notify.send(event, to, data) records a Notification + per-channel
NotificationDelivery, queues them, and a daemon consumer delivers via
pluggable channel adapters (in-app, email, chat) inheriting retry,
backoff and dead-letter.
- Jinja2 email system: branded, dark-mode, Outlook-safe base layout +
per-event templates (security/backup/monitoring/welcome/invitation).
The legacy inline f-string email was removed and now renders here.
- In-app center API (inbox/unread-count/mark-read) + per-user Socket.IO push.
- Admin delivery log (stats/filter/retry).
- Email providers (SMTP/SendGrid/Postmark/SES/Mailgun) with encrypted
credentials + test; the email adapter prefers the default provider and
falls back to the legacy SMTP config; surfaced in ConnectionRegistry.
- NotificationService.send_all() is now a non-blocking shim that routes
system alerts through the bus. Migrations 023 + 024.
Tests: backend/tests/test_notification_bus.py (20). Full suite: 391 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add pluggable queue bus with SQLAlchemy backend (Redis-ready interface)
- Models: QueueGroup, Queue, QueueMessage with retries, visibility timeout,
delayed delivery, priority ordering, and dead-letter handling
- REST API at /api/v1/queue for groups, queues, messages, and stats
- Plugin SDK: from app.plugins_sdk import queue
- Operations UI page with group/queue/message management and send-message form
- Migrate webhook deliveries from raw threading.Thread to the queue bus
- Add webhook consumer with retry/dead-letter support
- Migration 022_queue_bus and backend tests
AI is a core ServerKit primitive (not a plugin): an Intercom-style bubble + slide-in drawer on every dashboard page, with a tools+page-context assistant mode and a plain simple chat mode, SSE streaming, conversation persistence, and guarded write actions (the model proposes; the user approves).
Plugins EXTEND the assistant rather than rebuilding it: backend via app.plugins_sdk.ai (reg.tool + register_context_provider, discovered from a plugin.json ai block with install/enable/disable hooks); frontend via useServerkitAI() and a contributions.ai category (suggested_prompts, tool_renderers). The marketplace gains an ai category for discovery.
Backend: ai_service (Prompture Conversation + ProviderEnvironment + guardrails), ai_tool_registry singleton, ai_tools_builtin (core.* tools), the /api/v1/ai blueprint (SSE via a worker-thread + queue bridge, single-gevent-worker safe; in-thread confirmation gate), AiConversation/AiMessage/AiPendingAction models + migration 013, encrypted ai_* settings. Frontend: AIContext + components/ai/*, AI settings tab. Powered by Prompture (installed editable; requirements pinned).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce a first-class plugin contribution system and SDK, plus frontend integration and installer enhancements.
Backend:
- Add /plugins/manifest-spec and /plugins/contributions endpoints to expose the plugin.json contract and merged active-plugin UI contributions.
- Add app/plugins_sdk for plugin authors (current_user, logger, audit, jwt helpers).
- Add ContributionService to aggregate active plugin contributions (nav, routes, page_titles, command_palette, widgets).
- Marketplace now hands off to plugin installer when an extension has a source URL and enforces admin-only installs in the marketplace API.
- PluginService: run lifecycle hooks (install/uninstall) and install declared template dependencies (best-effort, recorded/logged) during plugin installs; surface template/manifest fields on InstalledPlugin.to_dict; surface linked InstalledPlugin status on ExtensionInstall.to_dict.
Frontend:
- Add runtime contributions loader (plugins/contributions.js) with useContributions hook and resolveComponent/getPluginModule helpers.
- Add SDK re-exports for plugin frontends (plugins/sdk/index.js) and Vite alias 'serverkit-sdk'.
- Integrate contributions across UI: Sidebar, CommandPalette, PluginLoader (declared widgets + legacy fallback), PageTitleUpdater, DashboardLayout (refreshContributions), and route injection via new ExtensionRoutes hook loaded in App.jsx.
- Add API client getPluginContributions and adjust PluginLoader discovery patterns.
Other:
- Add .extension-platform-plan/ to .gitignore.
Overall this enables declarative plugin-driven UI integration (sidebar items, SPA routes, page titles, command palette entries, and widgets), lifecycle hooks, and template dependency installs while keeping legacy behavior for older plugins.
Juan Denis