557 Commits
Author SHA1 Message Date
Juan DenisandClaude Opus 5 20f1659b51 test(monitoring): pin that monitors stay core
Walks the AST of monitor_service and the monitors blueprint to assert neither
imports the plugin loader nor reaches for StatusPageService, and that the routes
register without the extension. This is the property the promotion exists to
provide — watching a site must keep working on a lean panel with
serverkit-status uninstalled — so it should fail loudly if it regresses.

AST rather than a source grep: the modules' own comments discuss the extension,
which a text match would trip over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 00:28:19 -04:00
Juan DenisandClaude Opus 5 cceb444d27 fix(monitoring): give the TLS certificate read its own clock
_maybe_attach_certificate throttled on last_check_at, which is the probe clock.
An active monitor is checked every 30s, so that condition was never older than
the 6h refresh window: the certificate would be read once and then never
refreshed, leaving a stale expiry date on the detail page forever.

Adds cert_checked_at (migration 081) and throttles on that instead. The stamp
is written even when the read fails, so an unreachable TLS endpoint is retried
on the same cadence rather than on every single probe.

Also serves the Monitors list sparkline from one bulk query per page instead of
one per row, bounded by a time window so a busy monitor's history cannot crowd
out a quiet one's the way a single global LIMIT would.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 00:22:56 -04:00
Juan DenisandClaude Opus 5 9512975ebc feat(monitoring): promote status components to first-class monitors
A StatusComponent was already a full synthetic check (http/tcp/dns/smtp/ping,
target, interval, timeout, HealthCheck history, uptime windows, auto-incidents)
— it was just framed as "a component of a status page" and its engine lived
inside the serverkit-status extension, so watching a site required publishing a
status page. Promote it:

- migration 081 makes status_components.page_id and status_incidents.page_id
  nullable, and adds the columns backing the monitor UI (is_paused,
  check_method, expected_status, keyword, follow_redirects, verify_tls, retries,
  consecutive_failures, cert_issuer, cert_expires_at)
- new core app/services/monitor_service.py owns the check engine; the extension
  keeps page publishing and delegates
- new /api/v1/monitors (CRUD, check, pause, history, uptime buckets, incidents)
- new builtin.monitor_check sweep on a 30s tick — until now checks only ran when
  someone pressed "Check now", so a configured monitor never noticed an outage

Check-engine defects fixed while moving it:

- ping ignored the process return code and could only ever report 'up'
- ping used Linux-only flags with no guard, breaking the Windows dev server
- http ignored its own method/expected_status/follow_redirects/verify_tls
- smtp was documented on the column but unimplemented; keyword did not exist
- run_check never recomputed uptime and never opened or resolved an incident —
  only the WordPress health-sync path did, so network monitors never paged
- TLS issuer/expiry are now captured (throttled to one read per 6h)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:29:02 -04:00
Juan DenisandClaude Opus 5 b83cb5ca82 feat(cron): rebuild /cron on the Domains list pattern
The page kept a shape the rest of the app moved off: a four-card KPI
strip, its own in-body search input, a hand-rolled table, four icon
buttons per row, and an edit modal on row click.

It now matches /domains: the segmented filter carries the counts (no
KPI strip), SearchField lives in the top bar, the list is the shared
DataTable, and a row opens the DS Drawer. The top bar keeps its title
— /cron has no tab group, and the reference mock shows a titled bar
there; the body was what was stale.

Last run / Next run / Status columns need data list_jobs() never
returned: it skipped the schedule_human / next_run / run-tracking joins
that jobs_for_application() has done since plan 34, so the admin list
could only ever show enabled-vs-disabled. Apply the same enrichment,
with tests covering the schedule join, the last-run join, and the
no-history case.

The drawer's run history uses GET /cron/jobs/<id>/runs, which already
existed and only ScheduledTasksCard consumed. It's admin-only and
empty until a job carries the tracking shim, so both cases are stated
in the drawer (with an enable-tracking action) rather than rendering a
blank table.

Also fixes styles that never applied: .run-output-*, .spinner-inline
and the modal field overrides were nested under .cron-page, but Radix
Dialog and Sheet portal to <body>. Hoisted to top level.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 22:44:40 -04:00
Juan Denis e3c4fa778d fix(files): address PR review — literal grep match, tail clamp, extend RBAC coverage
- log_service: grep the service/unit filter with -F so names like
  'nginx.service' are matched literally instead of as a regex.
- log_service: clamp the in-Python tail like 'tail -n' — lines=0 sliced
  as [-0:] == all lines, negatives kept nearly everything.
- test_files_rbac: add files/upload and files/s3/upload to WRITE_CASES,
  and disk-usage/disk-mounts/analyze/type-breakdown to READ_CASES, so the
  permission gates on the full /api/v1/files surface are covered.
2026-07-26 16:25:12 -04:00
Juan Denis 88295ab000 fix(update): harden migration step against boot side-effects and slot venv mismatch
- create_app(): skip background daemons (queue consumers, metrics, job
  system, analytics flush, linked-panel client) when loaded by a Flask CLI
  one-shot such as 'flask db upgrade', or when SERVERKIT_SKIP_BACKGROUND=1.
  They query the DB before migrations run, so any error there (corrupt DB,
  pre-migration schema) aborted the migration and sank the whole update.
- update.sh: set SERVERKIT_SKIP_BACKGROUND=1 for both flask db upgrade
  invocations as an explicit contract.
- update.sh: fail fast with an actionable message when the SQLite slot copy
  fails PRAGMA integrity_check, instead of dying mid-boot with 'database
  disk image is malformed'.
- update.sh: rebuild the release tarball's prebuilt venv when its baked
  VIRTUAL_ENV path resolves to the old slot (it is baked as
  /opt/serverkit/venv), which previously ran new code on the old slot's
  dependencies during migration. Resolved-path comparison keeps the fast
  prebuilt path for fresh installs.
2026-07-26 16:18:37 -04:00
Juan Denis ddf2e948d1 fix(security): enforce files.read on reads; de-shell syslog unit filter
Follow-ups from the GHSA-4wqh-7f4f-5qmx / GHSA-mc93-rc3x-fpgq review:

- File Manager read endpoints (browse/info/read/search/disk-*/analyze/
  type-breakdown/download + s3/browse, s3/read, s3/download-url) only
  required a valid JWT, so a user whose files.read permission is revoked
  via custom permissions could still read and download files. They now
  require permission_required('files', 'read'), matching the write side.

- LogService._read_syslog() interpolated the unit name into a bash -c
  string using subprocess.list2cmdline(), which is cmd.exe-style quoting
  and does not stop $(...)/backtick expansion under bash. grep now runs
  as a plain argv list (with --) and the tail happens in Python.
  Reachable only on non-systemd hosts by admins, but the same bug class
  as the Postfix advisory.

Regression tests: revoked files.read 403s every read endpoint; the
syslog filter never invokes a shell and passes payloads verbatim.
2026-07-26 15:40:14 -04:00
Juan DenisandAruvasgaChithan 696226002b fix(security): enforce files.write permission on File Manager mutations (GHSA-4wqh-7f4f-5qmx)
Mutating File Manager endpoints (write/create/mkdir/delete/rename/copy/
move/chmod/upload and the S3 equivalents) only required a valid JWT, so a
viewer account (files.write=false) could write and delete files. They now
require the files.write permission via permission_required(). Read
endpoints are unchanged. Adds regression tests covering both advisories.

Co-authored-by: AruvasgaChithan <AruvasgaChithan@users.noreply.github.com>
2026-07-26 15:19:22 -04:00
Juan Denisandtonghuaroot 20e79ff363 fix(security): prevent command injection via hostname in Postfix install (GHSA-mc93-rc3x-fpgq)
The hostname parameter was concatenated into a bash -c string executed as
root via run_privileged(). Validate the hostname against a strict charset
and pipe debconf selections via stdin so no user input ever reaches a
shell command line.

Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
2026-07-26 15:19:08 -04:00
Juan DenisandClaude Opus 5 640e7e827e docs(screenshots): recapture, and fix two bugs the capture exposed
Six new sections cover the pages rebuilt this round — the dashboard as it ships,
its edit mode, the widget library, the widget editor, the databases explorer and
the engine catalog drawer. 31/31 sections render.

The capture is a render smoke test, not just a picture, and it earned that here
by surfacing two real bugs that no test could see:

`SearchField` merges its `className` onto its OWN root, so `.skw-lib__search`
IS the `.sk-searchfield` element rather than a wrapper around one. The rule
styling it as a parent matched nothing, leaving the shared `flex: 1 1 240px`
in force — and inside a column-direction filter bar that 240px basis is a
HEIGHT. The widget library's search box was rendering 240px tall.

`GET /auth/me` answers `{user: {...}}`, but Databases read `.role` straight off
the envelope, so `isAdmin` was always false. Every admin got the read-only
console — and with the DDL builder now shipped, would never have seen the button
that runs a CREATE TABLE.

Also carries `postgresql.yaml`, which I left out of the extensions commit.
Without `engine.extensions: true` stock PostgreSQL is not a host for pgvector at
all; the extension tests passed only because the working tree had the change the
commit did not.

Fixture note for whoever edits these next: `/metrics/history` points are NESTED
(`cpu.percent`, `load['1m']`) and carry no network data, `/servers/available` is
a bare array, and `DeployPill` queries `/deployment-jobs` twice and sums — a
single unfiltered payload makes it claim eight running deploys.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 09:59:06 -04:00
Juan DenisandClaude Opus 5 6edeba2ff5 fix(databases): scope the extension listing to apps the caller can see
`GET /databases/engines/<app_id>/extensions` carried only `@jwt_required()`, so
any signed-in user could enumerate any instance on the panel — the image it
runs, its databases, and what it can load. All facts about someone else's
application.

It gates on `can_access_app` now, mirroring the per-app docker DB read next to
it: owner, admin, or an explicit grant. Read access rather than admin-only is
deliberate — a developer who can already see an app should be able to see what
that app can load. The POST was already admin-gated and is unchanged.

The file also declares its `# Bucket: PER-APP` header (plan 29 #9), which is
what the structural sweeps read to know a per-app route file has been thought
about at all.

Found by the repo's own authz sweeps in a full-suite run — four failures, one
cause. Worth noting they fired on a route that has no feature test capable of
catching this: every functional test authenticates as someone entitled to the
data. That is precisely what a structural sweep is for.

Four tests pin the behaviour: denied without access, still visible to the owner,
404 rather than 403 for an app that does not exist (conflating them would make
the endpoint useless to its own caller), and 401 anonymous.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 00:19:46 -04:00
Juan DenisandClaude Opus 5 679082bf16 feat(databases): engine extensions, without the silent failure
The third deferred item. An extension installs INTO a running engine rather than
getting its own container, so it lives in its own namespace:
`backend/templates/extensions/<id>.yaml` with a top-level `extension:` block,
discovered by the same operator-wins two-directory scan the app templates use.
Dropping a YAML in makes it available with no code change, and a test pins that.

Not nested inside the engine template, because pgvector applies to any
PostgreSQL-speaking engine rather than to `postgresql.yaml` — nesting would force
duplication per host. Not a third `kind:` in the app catalog either: an
extension has no compose, port or volume and produces no Application, so every
consumer of `list_all_templates` would have had to learn to skip it.

The hard part is not the SQL. `CREATE EXTENSION vector` is valid and fails on
stock `postgres:16` with "could not open extension control file", because that
image does not ship the library. Two layers, and the statement is never sent
speculatively: an extension declares which images provide it, so a listing marks
an instance unsupported BEFORE anything is clicked; then at install
`pg_available_extensions` is queried first and a negative answer is a 409 with a
remedy, not an attempt. The live probe overrules the image name in both
directions — a private `registry.internal/ops/pg-with-vector` is allowed, and a
`pgvector/pgvector` tag that does not actually carry the library is still
refused.

ServerKit does not recreate a container behind the operator's back. The remedy
instead names an engine template that IS capable, found by matching the
extension's images against each engine template's compose — hence
`postgresql-pgvector.yaml`. TimescaleDB deliberately ships no bundled host and
says so rather than implying one exists.

Compatibility also needed correcting mid-build: matching on wire protocol alone
claimed CockroachDB could host pgvector. It answers on pgwire and cannot load a
single extension. Engines opt in with `engine.extensions: true`, which excludes
it without any Python naming it.

Two fixes fell out. `DatabaseService` had no psql twin of `docker_mysql_execute`
— `managed_db_user_service` carried a private copy with a comment saying as
much; there is now one implementation and it delegates. And that twin sets
`ON_ERROR_STOP=1`, without which psql reports SUCCESS for a statement the server
rejected.

57 tests, covering the zero-code-change property, compatibility, the image
problem in both probe directions, admin gating, and refusals that must not
execute anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 23:44:34 -04:00
Juan DenisandClaude Opus 5 5927e44487 fix(databases): run all twelve engines before claiming they work
The engine templates were written from upstream docs and reviewed carefully, but
only ever read. Every one has now been brought up for real, authenticated
against, and torn down.

CouchDB was broken. A fresh node never created its system databases: `/_all_dbs`
answered `[]`, `_users` did not exist so no non-admin account could ever be
created, and it error-looped on "Missing system database _users". The image
reads only COUCHDB_USER/PASSWORD/SECRET/ERLANG_COOKIE — both
COUCHDB_COUCHDB__SINGLE_NODE and ERL_FLAGS are rejected as unknown_config_value
— so `single_node = true` ships as a local.d config file instead.

Meilisearch and Valkey advertised versions they do not install (1.0 vs the 1.12
that boots; 8.0 vs 8.1.9), caught by comparing the catalog header against what
actually reported for itself.

The method mattered more than the count. Probing auth from INSIDE the container
is not a test and would have passed two engines that were never checked:
postgres initdb writes `host all all 127.0.0.1/32 trust`, so a deliberately
wrong password succeeds on loopback, and the ClickHouse client reads
CLICKHOUSE_PASSWORD out of the container's own environment. Re-run from a remote
source address, both correctly refuse. Every auth assertion here was made from
off-host.

Nine tests cover what can be checked without Docker, led by the one that matters
most: the generated admin password must actually REACH the image. Mongo,
ClickHouse and CouchDB all start happily with authentication disabled when the
password is generated but never passed — silent, and indistinguishable from
working. Also: every ${VAR} resolves, no port binds all interfaces, the catalog
port matches the published one, the declared version is the tag installed, and a
config file can never be rendered over the data volume's mount point (which
would delete persistence without a word).

Verified by reverting each fix and confirming the tests fail with the right
message, then restoring. 67 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 22:51:04 -04:00
Juan DenisandClaude Opus 5 aa30c859ed fix(databases): the panel could not read the MariaDB it installs
MariaDB 11 dropped the `mysql` symlink in favour of `mariadb`. Every place that
execs a client inside a database container hardcoded `mysql`, so introspecting
any modern MariaDB image failed outright:

    OCI runtime exec failed: exec: "mysql": executable file not found in $PATH

Which is not a hypothetical image. `backend/templates/mariadb.yaml` — the engine
template shipped earlier in this same plan — pins `mariadb:11.4`. Installing
MariaDB through ServerKit produced a database ServerKit could not open, and the
failure surfaced in the tree as a generic "couldn't load".

The client is now resolved per container by probing `--version`, preferring
`mariadb` and falling back to `mysql`, memoised because the answer cannot change
while a container runs. When neither answers it keeps returning `mysql`, so an
unrelated failure still reports the error operators already recognise.

Found by running a real mariadb:11.4 container against the service rather than
trusting the code path; verified end to end (SHOW DATABASES, list databases,
list tables with row counts) before writing the fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 22:01:15 -04:00
Juan DenisandClaude Opus 5 500a217618 feat(dashboard): one control row, and the clock becomes a widget
The board carried two headers: a host identity strip (hostname, live dot, IP,
kernel, uptime, clock) and, under it, the board tabs and controls. Two rows of
chrome before any actual dashboard, and the strip was fixed — you could not
move it, size it, or point it at a different machine.

It is one row now: board tabs left, then the server picker, range, refresh,
TV and Edit. The server switcher moved into that row as a labelled `server`
pill, and still collapses to a plain readout when there is only one host.

Nothing was lost, it just became placeable. IP, kernel and uptime are the Host
details widget — which can be resized and pointed at any server, unlike the
strip. And the time is a new Clock widget: add it if you want it, put it on a
second timezone if that is what you need, leave it off if you do not. It ticks
locally rather than polling, because re-fetching every second to print a number
the browser already knows would be silly.

Also fixes a regression from the previous commit: making the spark row flexible
let it compete with the value for height, and centring then pushed the number up
out of the tile. The reading is the point of a stat widget, so it is pinned
first and the chart takes what is left.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 21:23:14 -04:00
Juan DenisandClaude Opus 5 563e6c980d feat(dashboard): a real editor, and widgets that say what they are
The widget editor was a cramped fixed panel wedged against the right edge, with
inputs that rendered as bare white boxes because the field class sat on a
wrapper div while the stylesheet styled it as though it were the input. It is
now a wide drawer: settings on the right, a LIVE PREVIEW of the actual widget
on the left, sized from its real grid geometry so the size steppers visibly
move it. Stacks below lg. Built on the shared Drawer, which also makes the
floating AI bubble hide itself — it already hides for any open dialog, and the
old bare <aside> was exactly why it did not.

Every tile read "Stat". Seeded widgets carry no title, so the frame fell back
to the widget TYPE and a board of four different stats said "Stat" four times.
Titles are now derived at render time from what the widget points at — CPU
usage, Memory, Applications, "CPU usage & Memory" for a two-series chart —
falling back to the type name only when nothing better is known. Deliberately
derived rather than written into stored config: a title the user never set has
no business appearing in their saved board.

Boards already in the database are repaired on read. Fixing the shipped
defaults only ever helped users who had never opened the dashboard, because
seeding never runs again for anyone else — so `metrics` becomes `series`,
`actions` becomes `items`, dead shortcut keys are aliased or dropped, and an
empty shortcut list falls back to the shipped four instead of rendering "No
actions selected". The repair works on a deep copy: a JSON column mutated in
place looks unchanged to SQLAlchemy, so the commit would expire the instance
and silently reload the originals.

The log widget stopped guessing. With nothing configured it grabbed whatever
the log listing returned first — typically a system file the panel cannot read
— so a fresh dashboard greeted people with "sudo: a password is required". It
now asks you to pick a source, and phrases a permission failure as a setup
fact rather than quoting the shell.

Also deletes ~300 lines of stylesheet for the inspector that no longer exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 21:14:50 -04:00
Juan DenisandClaude Opus 5 ac4d8761b2 fix(dashboard): the widgets were speaking a dialect the stylesheet did not
Thirty-five of the fifty-two class names the widget renderers emit had no rule
anywhere. The renderers kept the design mock's flat names (`skw-stat-v`,
`skw-feed-item`, `skw-bar-row`) while the stylesheet was written in BEM
(`skw-stat__value`, `skw-topn__row`). Both files compiled, linted and built
clean the whole time — the styles simply never reached the markup.

So the activity feed rendered as raw text, host details ran their labels into
their values ("HostnameDESKTOP-8ELA366"), stat values wrapped mid-tile, and the
time-series chart lost its height constraint and drew as stray vertical spikes
against an empty box. Renamed the JSX to match, and wrote the blocks that were
missing outright: the feed, quick-action rows, alert body, and distinct
loading-vs-error empty states.

The seeded boards were wrong in the same way, one layer down. `timeseries`
seeded `metrics` while the renderer reads `series`; `actions` seeded `actions`
while the renderer reads `items`, with shortcut keys (`new-app`, `deploy`) that
no route resolves. Both fail silently — the widget just renders empty, which is
what every new user saw on their first dashboard. `logs` and `status` were
seeding keys nothing reads either.

Two tests now pin that coupling: seeded configs must use keys the renderers
actually read, and every seeded shortcut must resolve to a real route. The
table in the test is an explicit mirror of renderers.jsx, and says so.

Also: seeded widgets carry real titles again — "CPU usage", "Applications",
"Hardware specs", following the old dashboard's panel names — instead of every
tile falling back to its widget type and reading "Stat". And "Local (this
server)" is now "This server", which fits a 3-column tile on one line.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:57:32 -04:00
Juan DenisandClaude Opus 5 c55390e5f4 test(migrate): pin the three states the update screen can be in
`MigrationService.get_status()` had no tests, and it had just grown a third
state: a database stamped at a revision this build does not contain. That state
previously rendered as "80 pending updates" — every revision in history — behind
a Continue button that could only fail.

Covers up-to-date, genuinely-behind, and orphaned, plus the two edges worth
nailing down: the orphan flag has to survive the HTTP response because the
screen keys its warning off it, and a database with no revision at all is a
fresh install rather than a stranded one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:35:19 -04:00
Juan DenisandClaude Opus 5 69b98f7149 fix(migrate): stop burying Continue under every migration ever written
An install a long way behind head listed all 80 pending migrations at full
size, so reaching the Continue button meant scrolling past a page of revisions
nobody reads. Show the newest four — the ones that say what you're actually
getting — with the rest behind "Show all N", and let the expanded list scroll
inside itself so the button stays put no matter how far behind you are. Panel
and row padding tightened along the way.

The 80 was also a lie, and that turned out to be a real bug. Pending migrations
are computed by walking history backwards and breaking when the walk reaches the
database's current revision. If that revision isn't in the chain at all — a
branch that added a migration and was then reverted, leaving the database
stamped at a file that no longer exists — the walk never breaks and reports
every revision as pending. `upgrade` would then fail with "Can't locate
revision" anyway.

So detect it: an unknown current revision is reported as `orphaned_revision`
instead of a fabricated backlog, the screen explains what happened and prints
the `flask db stamp` that fixes it, and Continue is disabled because there is
nothing it could successfully do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:33:34 -04:00
Juan DenisandClaude Opus 5 3070353631 feat(databases): install any engine, because an engine is just a template
You could not install a database from ServerKit. The tree's four roots were
literals, only MySQL and PostgreSQL reported real state, and no install path
existed anywhere in the backend.

Rather than add one, this makes a database engine an ordinary app template that
declares a top-level `engine:` block — family, wire protocol, admin user, which
variable holds the password, what it calls a "table". Presence of that block is
the only thing that lists a template in the Databases catalog, and installing it
is the template pipeline that already existed: same deployment job, same
streaming console.

The consequence is the point. Adding CockroachDB is adding a YAML file. So is
adding something we have never heard of — author it locally or sync a template
repo — and the install form still works, because it is generated from the
template's own `variables[]` rather than hand-written per engine. There is no
engine model, no engine table, no engine install mechanism to keep in step.
`test_dropping_a_new_engine_yaml_lists_it` is the design; if it ever needs code
to pass, this went wrong.

Twelve engines ship: postgresql, mysql, mariadb, mongodb, redis, valkey,
cockroachdb, clickhouse, influxdb, neo4j, meilisearch, couchdb. Each pins its
image, binds to loopback, names a data volume, and was checked against the
upstream image's actual environment contract — a wrong variable name is a
container that boots with authentication silently disabled.

Two safety notes worth reading. Engines bind 127.0.0.1 unless explicitly
exposed, and the flag is applied after caller variables so a handcrafted payload
cannot quietly publish one. And CockroachDB single-node runs `--insecure`
because password auth needs a certificate-backed cluster: for engines with no
credentials at all the loopback bind is the only control, so public exposure is
refused outright rather than warned about. A test pins the passwordless list to
exactly cockroachdb, so a future template that drops its password variable
fails loudly instead of opening a hole.

Also surfaced at last: `ManagedDatabase.owner_application_id` has been accepted
by the create endpoints for ages with no UI to set it. Creating a database now
offers "Attach to application".

Deferred deliberately: new-table builder, SQL dump import, engine extensions
(pgvector, TimescaleDB) — each a data-mutation surface deserving its own round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:09:05 -04:00
Juan DenisandClaude Opus 5 c4878f392f fix(tests): stop the compose-flag test depending on the host having Docker
`test_streams_lines_in_order_with_plain_flags` asserted `--progress plain`
reaches the command, but left `_is_compose_v2()` real. That probe shells out to
`docker compose version`, so on any machine without a working Docker it fell
back to v1, the flag was legitimately omitted, and the test failed for a reason
that had nothing to do with what it was testing.

Pin the compose flavour instead, and add the case the production comment
actually warns about: v1 answers an unknown flag with its usage text and exit 1,
which is how template installs on a v1 host once died at "Start Docker Compose
stack" with a wall of help text. Nothing asserted that `--progress` stays off
the v1 command until now.

No production code changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 20:04:39 -04:00
Juan DenisandClaude Opus 5 010b632530 feat(dashboard): boards that survive the browser they were built in
The dashboard's layout lived in localStorage as `{id, visible}` — a visibility
toggle wearing a layout's clothes. It carried no geometry, so widget width was
baked into SCSS (`.dash-kpi { grid-column: span 3 }`) and "make the chart wider"
was a code change. And it stayed on one machine.

`DashboardBoard` gives a user real boards: named, ordered, each holding a JSON
array of placed widgets with x/y/w/h and per-instance config. Three defaults
ship — overview, infrastructure, apps & deploys — seeded on a user's first GET
and restorable per board.

Validation is deliberate about the boring cases: geometry must fit the 12-column
grid, ids must be unique within a board, bools are rejected as coordinates
(`bool` subclasses `int`), and a rejected PUT leaves the stored board untouched
rather than half-applied. A board belonging to another user is a 404, not a 403
— there is no reason to confirm it exists.

The shipped defaults are checked by rebuilding their occupancy map cell by cell,
so editing DEFAULT_BOARDS into an overlap fails the suite instead of shipping a
broken first-run layout.

31 tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:30:51 -04:00
Juan DenisandClaude Opus 5 92e0750262 feat(sdk): a second widget surface — types users place, not just slots
`contributions.widgets` mounts a component at a host-chosen slot. That is the
wrong shape for the new dashboard, where the user decides what goes on a board
and where. So this adds `dashboard_widgets` alongside it: a registry of
placeable widget TYPES that surface in the widget library, each instance
positioned, sized and configured by the person looking at it.

The two are easy to confuse, so the docstring, the manifest spec and
EXTENSIONS.md all now say plainly which is which, and a test asserts a
dashboard widget never leaks into the fixed-slot list.

SDK_VERSION 1.0.0 -> 1.1.0 (added surface, older extensions unaffected),
bumped in lock-step across the JS export and the backend mirror.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:24:41 -04:00
Juan DenisandClaude Opus 5 ef3712f97a fix(services): real numbers in Resource Usage, and a Related card that finishes
Two more of the same fault as the logs: the frontend reading fields the backend
does not send at that level.

Resource Usage showed 0.0% CPU, 0.0% memory and N/A for everything else on a
healthy container. The stats route answers {stats: {...}} carrying docker's own
CLI keys, and the caller read the top level of the envelope — so every field was
undefined and fell through to its zero. It unwraps `stats` now, which is where
CPUPerc, MemUsage, NetIO and PIDs have been all along.

Related Resources sat on "Loading…" permanently because GET /apps/<id>/related
was never implemented. The client has always called it, the 404 was swallowed by
an empty catch, and the card reads null as "still loading". The endpoint now
exists — domains, managed databases, backup policy and deploy count, scoped by
the same access check as the rest of the app routes, each section gathered
independently so a missing table degrades that row instead of failing the card.
The catch also settles to an empty object now, so a future failure shows "no
related resources" rather than spinning forever.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 17:13:36 -04:00
Juan DenisandClaude Opus 5 658f51dbbd fix(deploy): let a failed install be retried from the panel
An install writes its directory before it starts containers, so failing at
"Start Docker Compose stack" leaves the directory behind. The guard against
clobbering an existing app then refused the retry with "App directory already
exists" — and the only way forward was a shell. A dead end in exactly the flow
where someone is most likely to try again, and the one this happened in:
Uptime Kuma failed on the compose flag bug, then could not be reinstalled.

A directory with no Application row pointing at it was left by a failure — that
row is only written once the plan completes. Those are reusable, since the plan
rewrites every file it owns. Anything else is somebody's data: an unowned
directory still has to be empty or carry the .serverkit-template.json marker
before it can be touched, and an unreadable one counts as occupied.

When it genuinely is a conflict, the message now names a way out instead of
just stating the fact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 16:51:44 -04:00
Juan DenisandClaude Opus 5 cde4595764 fix(docker): stop sending compose v1 a v2-only flag
Every template install on a docker-compose v1 host failed at "Start Docker
Compose stack". The streaming path passed `--progress plain`, which v1 has no
global flag for, so it printed its entire usage text and exited 1 — the deploy
console filled with help output and nothing anywhere said the argument was the
problem. Reproduced deploying Uptime Kuma locally.

The service already detects v1 vs v2 and falls back to docker-compose; the flag
simply wasn't gated on it. `--ansi never` stays for both, since v1 does accept
that one.

Tests pin the shape rather than the symptom: v1 never receives --progress, v2
still gets plain progress, both keep their global flags before the subcommand,
and detection falls back to v1 when docker is missing entirely instead of
raising into the deploy path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 16:19:33 -04:00
Juan DenisandClaude Opus 5 da2b77f3d2 feat(templates): tell people whether a template actually fits the server
Deploying checked three things — the name is free, the server row exists, the
directory isn't taken. Nothing asked whether the box could carry the thing, so
a 2 GB app landed on a 1 GB VPS at full speed and the first sign of trouble was
the OOM killer.

Every piece was already here and unconnected: templates describe what they cost
(`requirements: memory / storage`), servers report what they have
(Server.total_memory / total_disk) and what they are using (the newest
ServerMetrics sample). This is the join, answering in the shape a person asks
it — does it fit, will it be tight, or can't we tell?

Advisory on purpose. It reports; it never refuses. An estimate is an estimate,
real usage depends on configuration and traffic, so the operator keeps the
final say and "tight" is a sentence to read rather than a wall to argue with.
A full server still returns valid:true, and a test asserts that so this cannot
quietly become a gate.

Three judgements worth stating:

- The reserve is a floor-and-fraction pair (10% of the box, at least 256 MB RAM
  / 2 GB disk). A bare percentage is wrong at both ends: 10% of 1 GB is
  nothing, 10% of 64 GB is more than most apps need.
- `unknown` outranks `ok`. No recent sample, or a template that declares
  nothing, says so — a confident wrong answer is worse than no answer.
- Free memory comes from psutil's `available`, not `free`: cache and buffers
  are reclaimable, and counting them as used calls every healthy Linux box
  full.

Catalog coverage goes from 52/106 to 106/106. The 54 estimates are each app's
typical idle-to-light-use footprint, taken from the project's own documented
minimum where it publishes one and rounded to the values the existing 52
already use — sonarqube 4 GB because that is its documented heap floor, immich
4 GB for the ML models, prometheus 20 GB because retention dominates, traefik
256 MB.

Surfaced two ways: a capacity block alongside (never inside) `errors` on
validate-install, and a read-only GET the deploy drawer can poll while the
operator is still choosing a target, so the answer updates as they switch
servers instead of waiting for a submit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 15:48:36 -04:00
Juan DenisandClaude Opus 5 cc72494d59 refactor(gui): route agent commands through the SDK, and declare them all
serverkit-gui called the agent registry directly, so its manifest permissions
were decoration — the install dialog said "screenshot and capabilities" while
/synthetic also asked for system:info and system:processes, neither declared.
Going through plugins_sdk.agents enforces the declaration, so the first thing
the retrofit needed was for the manifest to tell the truth: all four actions
are now declared, in both permissions and requires_agent_actions.

Per-endpoint behaviour is deliberately unchanged. /capabilities still answers
200 with the synthetic fallback when the agent can't do it, /frame still 502s a
failed capture and 503s an offline agent.

One case needed care. declared_permissions() reads the manifest captured in the
database at install time, not plugin.json on disk, so an install predating this
lists only the two gui actions and would have started 403-ing on /synthetic.
That endpoint exists to render something for a host with no display, and it
already degrades when the agent won't answer — so a permission this plugin
hasn't been granted now lands in the same place, with a log line naming the
fix. An older install keeps working with less detail instead of breaking, and
regains it on update. There is a test for exactly that install.

Two things improve for free: a handler failure arriving inside a
transport-successful reply now reports the agent's real error instead of the
vague "agent returned no frame", and a non-dict payload can no longer crash the
capabilities handler.

The registry index still advertises the published 0.1.0-beta.1 zip and its
sha256, so it is left alone — permissions there move when the release is cut.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:19:06 -04:00
Juan DenisandClaude Opus 5 b1f002c09d perf(ai): keep Prompture out of the panel's boot path (-24 MB RSS)
The AI assistant is a core primitive, so app/api/ai.py is registered by
create_app() like every other blueprint. It reached ai_service, which imported
prompture at module scope -- and `import prompture` costs ~69 MB of RSS, pulling
~300 prompture modules plus rich, pydantic, httpx, pygments and jsonschema.

Every panel paid that at startup whether or not anyone ever opened the
assistant. Measured in the production container (gunicorn --workers 1
--threads 100):

    before   worker 177.8 MB   total 204.9 MB
    after    worker 153.1 MB   total 180.5 MB

The saving is smaller than 69 MB because pydantic and httpx are still pulled by
other subsystems; only the prompture/rich/pygments-exclusive share is recovered.

`from __future__ import annotations` was already in both files, so every
annotation is a string and nothing needed a signature change:

- ai_service: prompture imports moved under TYPE_CHECKING, with real imports
  pushed into the four functions that construct a Conversation, ToolRegistry or
  ProviderEnvironment.
- ai_service: the two module-level guardrail singletons (PromptInjectionDetector,
  PIIRedactor) were the only objects forcing the import at load time. They are
  now built on first use behind the same double-checked lock that
  ensure_initialized() already uses.
- ai_tool_registry: tool_from_function deferred into register(). This one counts
  twice -- plugins_sdk/ai.py imports that module at load time too.

No behaviour change: the first assistant request pays the import instead of
every boot. Prompture is unmodified.

test_ai_lazy_import.py guards it. The checks run in a subprocess on purpose: by
the time pytest reaches that file another test has usually imported prompture
already, so an in-process sys.modules assertion would pass no matter what the
boot path does. Verified the guard actually bites by reintroducing a
module-level `from prompture import Conversation` -- 2 of 3 tests fail, and pass
again once removed.

Suite: 2586 passed, 47 skipped, run in a clean worktree with a private
TEST_DATABASE_URL. That isolation was needed because tests/conftest.py hardcodes
one shared %TEMP%/serverkit_test.db for every pytest process on the machine, so
a concurrent run stamps it to another branch's migration and mass-errors an
unrelated suite. Left alone here; it deserves its own fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:15:29 -04:00
Juan DenisandClaude Opus 5 0ae70f8761 docs(plugins): point the SDK docstring at the contribution surfaces
The module docstring described only the plumbing a plugin imports, not the six
surfaces it can now contribute to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:05:58 -04:00
Juan DenisandClaude Opus 5 047e69501a feat(plugins): let extensions run commands on the fleet
Everything an extension could do stopped at the machine the panel runs on,
which is at odds with a product built around a fleet of servers. A plugin can
now run an agent command and get its result back.

This is also where agent.command:<action> starts meaning something. The
manifest permission has existed as a declaration — collected in the manifest,
surfaced at install time for consent — with nothing enforcing it, because
plugins called the agent registry directly. Dispatching through the SDK checks
it first, matched verbatim, so a plugin can only run the actions it asked the
operator for by name. The one bundled extension that uses agent commands still
calls the registry directly; retrofitting it onto this is a follow-up.

Nothing underneath is re-implemented: the per-server ACL still decides whether
that server permits the action, and the ServerCommand audit row is still
written — pass user_id so it names a person rather than "the panel".

Three things the wrapper adds over a bare send_command:

- it unwraps the double envelope. A transport-successful reply can carry a
  handler failure inside, which reads as success to anyone checking only the
  outer flag.
- it fails before dispatch on an offline agent or a missing capability, so the
  caller gets a reason instead of an unknown-action error from the far side.
- the default timeout is 45s rather than the registry's 30s. An agent on the
  HTTP long-poll fallback can sit ~25s before it even sees the command, which
  leaves a 30s budget almost no room to actually run anything.

Failures raise with a .code, so a plugin can tell an offline agent from a
refused one from a timeout, and map them to its own HTTP responses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 14:02:08 -04:00
Juan DenisandClaude Opus 5 e8e94a126f feat(plugins): let extensions contribute backup kinds
An extension with something worth backing up had to hand-roll it — no policy,
no cron mirror, no run history, no retention, no offsite copy, no verification,
no restore, and no place in the Protection panel. A registered kind inherits
every one of those, because everything downstream of the target_type switches
is already generic: the run lifecycle, retention (count, age, incremental-chain
ancestry), the API blueprint, and the panel itself, which threads target_type
through without ever switching on its value.

No migration: target_type is an unconstrained String, and the allowed set lived
purely in a Python tuple. A kind supplies resolve() and execute(), plus an
optional restore().

Four switches had to learn about the registry, and the order matters in three
of them:

- validate_target_type is the keystone — every policy route funnels through it,
  so accepting a registered kind opens the whole blueprint at once.
- _resolve_target and _execute_backup check the registry BEFORE their core
  branches, because both end in an application fallthrough that swallows any
  unrecognised type.
- run_restore_job likewise. This is the sharp one: without it a plugin's
  archive would be unpacked over an app directory by core's else-branch.

A kind that didn't implement restore is refused at request time with a reason,
rather than queueing a job that takes the policy lock and runs a safety backup
before failing.

The registry normalises what plugins hand back — target_type is forced rather
than defaulted, so a kind cannot describe itself as an 'application' and be
restored as one; execute() must return (path, size, meta) with a real path; and
the metadata retention and the panel read is filled in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:59:51 -04:00
Juan DenisandClaude Opus 5 7e051ddac6 feat(plugins): let extensions contribute doctor checks
The sweep was a fixed sequence of core producers, so an extension had nowhere
to say "the thing I manage is unhealthy, and here is what to do about it" — on
the page an operator opens to ask exactly that. A plugin now registers a
provider and its checks appear beside the core ones, with a working Repair
button, and no frontend change: the panel's status pill, repair button and diff
toggle all key off fields rather than a known list of check ids.

Because these run third-party code, the registry gives them two things core's
own producers don't have:

- an error boundary. run() calls its producers bare — each hand-rolls its own
  try/except and anything that escapes 500s the whole sweep. A provider that
  raises becomes one warning attributed to the plugin instead.
- a time budget. Nothing bounds a core producer today, so a hung one hangs the
  synchronous POST /doctor/run for whoever asked. A provider that overruns is
  reported as a warning and the sweep moves on. Note the executor is shut down
  with wait=False on purpose: its context manager joins the worker on exit,
  which would have made the timeout worthless — a test pins that down.

Providers run in a worker thread with the app context carried across, so a
check can query the database the way a core one does.

Contributed keys are namespaced, statuses outside ok/warn/fail are reported as
warnings rather than rendering as a blank pill, and a check claiming to be
repairable without a registered handler is demoted — showing a button that
dispatches nowhere is the state two core checks are already in, and it isn't
worth reproducing. Repairs route back through the registry, which is itself the
allowlist: a crafted ref for an unregistered namespace reaches nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:56:09 -04:00
Juan DenisandClaude Opus 5 caa511064d feat(plugins): give extensions a place to keep their own state
An extension that needed to remember anything had to add a model to core AND an
Alembic migration for it — which is why extension tables like
ext_serverkit_minecraft_* sit in the panel's own migration history. That is a
steep tax on a small extension and it puts plugin schema in the panel's upgrade
path. A plugin now writes to a key/value store instead:

    mine = store.for_plugin('serverkit-minecraft')
    mine.set('world:1:seed', 8675309)

Values are JSON, so dicts and lists round-trip without hand-serialising, and
rows are namespaced by slug so two plugins cannot collide on a key. Reach for a
real table when you need to query BY the data — this is a key/value store, not
a database, and the docstring says so.

Because the table is shared by every installed plugin, three things are refused
rather than trusted: a value that isn't JSON-serializable, a value over 256 KB,
and an empty or overlong key. Validation happens at set() rather than at commit
— by commit time the error is opaque and the session is already poisoned, so
the next unrelated write fails too.

has() exists because get() cannot distinguish a stored None from a missing key,
and a plugin storing None deliberately would otherwise have no way to tell.
Concurrent writers race on insert; the unique constraint makes that safe and
the loser applies its write to the winner's row instead of failing the caller.

Store rows follow the same rule as extension-declared tables: an ordinary
uninstall leaves them, so a reinstall finds its state where it left it, and
"uninstall and delete data" removes them. The purge runs after the uninstall
lifecycle hook, since a hook may legitimately read its own store on the way out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:51:11 -04:00
Juan DenisandClaude Opus 5 4a5c90df62 fix(tests): give each pytest process its own database
The suite pointed every run at one fixed file, %TEMP%/serverkit_test.db,
deleted once at conftest import — and the delete swallowed OSError. On Windows
that is not a rare edge: a finished pytest whose app threads outlive it still
holds the file, the delete fails, the error is discarded, and the next run
quietly inherits the previous run's rows. create_all then collides with the
already-seeded settings row, and tests error at fixture setup — in a different
place every run, never in the test that caused it. Two suites running at once
share one database and truncate each other's tables mid-test, which is the same
symptom from the other direction.

That is the "flaky tests" folklore in this repo, and it is why a full-suite run
reported four errors that all passed when re-run alone.

Each process now uses its own serverkit_test_<pid>.db and sweeps stale ones at
startup, so a lingering process can no longer poison the next run and two
suites can run side by side. Session finish disposes the engine and removes the
file; if a thread still holds it, the next run's sweep gets it.

Verified by running two suites concurrently — both green, where the shared file
previously corrupted whichever one started second.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:50:53 -04:00
Juan DenisandClaude Opus 5 2a0b4ca029 feat(plugins): let extensions put their objects in global search
The omnisearch fan-out was a hardcoded sequence of core entity blocks, so an
extension's objects were absent from the command palette — the one place the
UI tells you to look. A plugin now registers a provider and its rows appear
alongside services, servers and domains, with no frontend change: the palette
already renders unknown types, bucketing them under "Results".

Scoping stays the provider's job, because search invents no ACL — every core
block scopes its own query and the provider is handed the user and workspace
to do the same. The shape, though, is not trusted:

- rows are always tagged with the registered type, so a plugin cannot mint a
  row that claims to be a 'service' or a 'vault' (which are trusted to have
  been scoped by core);
- the per-type cap is re-applied host-side rather than trusted from the
  provider, so one contributed type can't drown the core ones;
- a path must be an in-app router path. The palette hands it straight to
  navigate(), so a protocol-relative '//host' would walk the user off the
  panel entirely.

A malformed row is dropped on its own, a broken provider degrades to "no rows
of that type" — the same isolation each core block already has.

The lazy accessor in plugins_sdk now carries a table of surfaces rather than a
single special case for deploys, since the other SDK surfaces land the same way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:40:00 -04:00
Juan DenisandClaude Opus 5 3c84d152ea feat(wordpress): run environment promotions as watchable deployments
Promoting between WordPress environments ran in the request thread: no
console, no log stream, no retry, and a progress bar the pipeline page had
to keep alive itself. It now runs as a registered deployment kind, so it
inherits the Deploy Console, live logs, retry and a row in Deploy Activity.
POST /promote returns 202 with a job_id instead of blocking, and the UI
opens the console with it.

PromotionJob is kept, not replaced. It carries pre_promotion_snapshot_id,
the rollback pointer, which is WordPress domain a generic DeploymentJob has
no business holding — so the deployment records which promotion rows it
produced instead. They are collected by id watermark rather than from the
service's return value, because promote_* only reports the job on success
and a failed promotion is exactly when someone needs the snapshot.

A full promotion runs two five-step halves and restarts its own numbering
at 1 for the second, so the progress bridge carries an offset across the
reset — otherwise the console's step strip walks backwards halfway through.

Two gaps this exposed in the SDK, both fixed here:

- Plugin kinds never closed their log stream, so they got no per-step
  timings, no failure tail and no hint, and no terminal status push. The
  handler and the runner also built separate streams, so the timings one
  recorded were dropped by the other's close; they now share one per job.
- A handler that reported failure by return value logged nothing, leaving
  a failed run with no reason visible in the console.

The console now titles a run from plan.title, so a contributed kind names
itself without a branch per kind in the frontend.

A locked environment is refused by the endpoint rather than by the job:
queueing first would send the user to a console that fails a second later.
The pipeline service still re-checks, since a lock can be taken in between.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:29:12 -04:00
Juan DenisandClaude Opus 5 6be60249f0 feat(plugins): let plugins own deployment-shaped work
DeploymentJobService.run_job dispatched on a hardcoded switch over three
core kinds, so a plugin could not own work that behaves like a
deployment. Its only option was the generic job system, which gives no
step plan, no streaming console and no place in Deploy Activity — the
reason promoting a WordPress environment, which is deploy-shaped in every
respect, is invisible in the one page you open to see what ran.

plugins_sdk.deploys lets a plugin register a kind and start one. The job
is a real DeploymentJob, so it inherits the Deploy Console, live logs,
retry and a row in the feed with no frontend work at all — those surfaces
key off the job, not off any particular kind, and the feed already
renders an unknown kind by its id.

Three things the SDK guarantees, each pinned by a test:

- Core kinds cannot be hijacked. A plugin registering app_deploy would
  silently take over real deployments.
- A handler that raises still ends the job visibly failed. Third-party
  code must not be able to leave a job spinning at 'running' with no
  explanation — the same promise _reconcile_interrupted_jobs makes for a
  lost process, extended to a lost handler.
- A handler that finalises the job itself keeps that outcome, even if its
  return value contradicts it.

deploys.steps() advances the job through its plan as the handler works,
so the console's pipeline strip and the feed's tick bar fill in, and a
failure records which step failed rather than only a stack trace.

The implementation is deploys_sdk.py, not deploys.py: a submodule of that
name is bound as a package attribute on import and would shadow the lazy
accessor, handing callers the module instead of the SDK.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 12:46:59 -04:00
Juan DenisandClaude Opus 5 9feaaed278 build(docker): halve the panel image, 1.07 GB -> 501 MB
The single-container image shipped its own build toolchain and an optional
antivirus in every running panel. Measured on linux/amd64:

    before                1.07 GB
    multi-stage venv        553 MB
    + venv prune            501 MB   (-53%)

Four changes:

1. Split a python-builder stage. gcc and the *-dev headers build wheels into
   /opt/venv; the runtime stage copies the venv and leaves the toolchain behind.
   The apt layer goes from 284 MB to 5 MB. No runtime -dev packages are needed:
   the xmlsec, lxml and cryptography wheels are self-contained, verified by
   importing all of them in the built image.

2. ClamAV behind ARG INSTALL_CLAMAV=false (153 MB). Safe because
   malware_scan_service already treats clamscan as best-effort -- it reports
   clamav_available and falls back to the builtin YARA matcher -- which is what
   the old Dockerfile comment claimed but did not do. Build with
   --build-arg INSTALL_CLAMAV=true to bundle it; that path is verified too.

3. COPY --chown instead of a trailing `chown -R /app`, which rewrote every
   file's metadata and duplicated the whole tree into a 22 MB layer.

4. Prune the venv (~50 MB): third-party bundled test suites, and botocore's API
   models for 400+ AWS services the panel never talks to. ServerKit constructs
   exactly three clients -- s3, route53, ses -- plus sts for signing.

pip stays in the runtime image on purpose: PluginService installs an extension's
Python requirements with `sys.executable -m pip install -r`, so PATH points at
the venv and the venv is owned by the serverkit user.

The botocore prune fails open in dev and closed in the image -- a new
boto3.client('sns') would work locally and raise UnknownServiceError only in the
built image, when a user hits the feature. test_botocore_prune_contract.py
parses ARG BOTOCORE_KEEP and every call site and fails the suite on drift.

Verified in the built image: 14 optional deps import, all four boto3 clients
construct, sns is gone, no gcc, no __pycache__ or venv leaked into /app,
/api/v1/system/health and the SPA index both return 200. Suite: 2546 passed,
47 skipped. Idle RSS is unchanged at ~208 MB -- image size and resident memory
are separate problems.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 12:38:56 -04:00
Juan DenisandClaude Opus 5 dddf0906e7 fix(plugins): write manifests with a trailing newline
json.dump emits no trailing newline, but every other writer of these same
tracked files — scripts/new-extension.mjs and
scripts/sync-builtin-frontends.mjs — appends one. The two disagreed, so
each plugin install and manifest regeneration rewrote
frontend/src/plugins/**/plugin.json and plugins-manifest.json with the
newline stripped.

The result was a working tree that went dirty on its own during a normal
backend boot, showing five modified files whose diff contained no actual
change. Easy to dismiss as line-ending noise; it is a writer disagreement,
and it would have kept coming back.

111 plugin/extension tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 11:48:15 -04:00
Juan DenisandClaude Opus 5 54a9b2d391 fix(system): put a ceiling on every privileged subprocess call
210 run_privileged() call sites passed no timeout. With output captured
and no terminal attached, a command that never returns takes its caller
with it — silently and forever. That is exactly how the metadata guard's
iptables probe wedged backend startup; the sudo -n fix removed that one
trigger, but the class of bug was still one hung command away.

run_privileged now applies a default timeout when the caller does not
name one, so the fix lands on all 210 sites at once rather than being
reapplied by hand and forgotten on the 211th. TimeoutExpired is a far
better outcome than a wedged worker, and callers that genuinely need
longer already pass their own; `timeout=None` opts out deliberately.

The four direct read-only probes in this module (dpkg -s, rpm -q,
systemctl is-active/is-enabled) bypass that helper, so they get their own
much tighter ceiling — they answer immediately or not at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 05:41:00 -04:00
Juan DenisandClaude Opus 5 0d768ca377 feat(services): port the deploy/services/templates design
Brings the Services tab group in line with the design prototype.

Deploy Activity is a feed rather than a table: runs group into Queued /
Today / Earlier, each row showing what was deployed (commit, image, or
template), why (push, webhook, manual), a per-step pipeline bar, target,
duration, and the error inline on failures. Step states are derived
client-side from current_step/total_steps/status, so the feed stays one
request. Status filters carry live counts via the shared SegControl.

Deploy Console trades the vertical step rail for a horizontal pipeline
strip above the log, each segment's bar sized to its share of the run —
so where the time went is visible at a glance, and the log pane gets the
full page width. A mono run-identity line sits between them.

Templates leads with the two routes no template covers (import a repo,
drop a ZIP), gains a Featured / A-Z sort, and deploys through a drawer
that shows the subdomain the service will be published at and a server
picker listing each target's CPU and memory — placing a service on a
second box is now a visible choice rather than a hidden default. That
needs capacity in the payload, so get_available_servers() now reports
cpu_cores/total_memory/total_disk (remote from the Server row, local via
a psutil probe).

Services gains an opt-in card view; ResourceListPage grew renderCard /
viewStorageKey props, so table-only pages are untouched.

Not ported: the mock's Resources card (CPU/memory limits, auto-restart,
backups). installTemplate does not carry limits through to
Application.cpu_limit/memory_limit, so the card would have claimed
limits the deploy never applies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 04:40:10 -04:00
Juan DenisandClaude Opus 5 3fb4242736 fix(system): make generated sudo invocations non-interactive
privileged_cmd prepended a bare `sudo`. Nothing here runs attached to a
terminal and run_privileged captures output, so on a non-root host a
password prompt is both invisible and unanswerable — sudo blocks forever
and takes its caller with it. This hung backend startup outright: the
metadata guard's `iptables -C DOCKER-USER` probe sat unanswered and the
server never bound its port.

Emit `sudo -n` so the call fails immediately with a non-zero exit, which
every caller already handles. Production runs as root (_needs_sudo() is
false) which is why this stayed hidden, but any non-root install without
passwordless sudo would hang on boot.

An explicitly-sudo command passed in by a caller is still forwarded
untouched; the regression test pins non-interactive sudo across all four
invocation shapes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 04:39:55 -04:00
Juan DenisandClaude Fable 5 3cc708a1e2 feat(deployments): simulated demo deploys through the real console pipeline
Dev-gated (DEMO_DEPLOYS_ENABLED) scripted DeploymentJobs — success/fail-build/
long/ansi/slow scenarios driven through RunLogStream so the Deploy Console is
fully exercisable locally with no Docker or server. GET/POST
/api/v1/deployment-jobs/simulate; retry + unified-job wiring; 12 proving tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 16:58:46 -04:00
Juan DenisandClaude Fable 5 7d4543312c feat(themes): add 11 bundled Venezuelan seed themes
Turpial, Araguaney, Orquídea, Apamate, Ceiba, Azulejo, Guacamaya, Cacao,
Papelón, Onoto (dual dark+light) and Tepuy (radius override) — named after
Venezuelan national symbols and nature. Authored in
frontend/src/data/themes/, synced to backend via sync-bundled-themes.mjs
(17 bundled themes total). check-theme-tokens + backend theme suites pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 10:22:23 -04:00
Juan Denis ad96423966 feat(themes): hardening — preview validation, registry caching, token wiring (plan 61)
- Validate preview swatches as colors server-side (closes url() smuggling
  via POST /import); import tolerates unknown metadata (license, image, modes)
- theme_registry_service: negative caching on fetch failure (TTL-stamped
  last-good + 60s failure backoff — no more 15s gallery stalls offline);
  drop dead base_url cache key; refresh stale comment
- .env.example: document SERVERKIT_THEMES_REGISTRY_URL/_TTL
- SCSS: route $success/$warning/$danger/$info through --green/--amber/
  --red/--cyan (raw twins for color-function call sites), $font-main/$font-mono
  through --sans/--mono, scrollbar track through --scrollbar-track
- Fix 5 undefined CSS var consumers (--radius-md/--card-bg/--danger/
  --accent-subtle/--accent-soft -> existing tokens)
- Nits: drop dead ThemeStudioTrigger export, dedupe lucide import,
  run check-theme-tokens in pre-commit hook
- docs/plans/61_THEMES_HARDENING_PLAN.md

23 backend theme tests pass; check-theme-tokens, lint (0 errors), build green
2026-07-23 09:25:01 -04:00
Juan DenisandClaude Opus 4.8 c20a9f14f2 feat(themes): backend model, API, and registry pipeline (plan 60 Phase 2-3)
Themes are data, not code — so the backend is a simple validated-JSON store,
no zips/checksums/permissions.

- Theme model + migration 078_themes
- theme_tokens.py: server-side mirror of the token whitelist + validators
  (the gate: a malicious/malformed theme never reaches the DB)
- theme_service: bundled-seed loading, import/list/delete, panel default,
  pre-auth public/active
- /api/v1/themes: /installed, /import (admin), DELETE (admin), /default
  get+set (admin), unauthenticated /public/active for login/setup, plus
  /registry + /registry/<slug>/install
- theme_registry_service: offline-tolerant registry fetch (TTL + last-good +
  bundled fallback), mirrors the extension registry pattern
- bundled seed themes synced to app/data/themes + themes_index.json
- 18 proving tests (server-side validation sanitizes url()/;/} smuggling,
  admin gate, panel-default → public/active, offline fallback, install flow)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 00:24:00 -04:00
Juan Denis e8d84297f2 Marketplace trust: split 409 reasons, hide unreviewed outside dev mode
Review feedback + follow-up on the review-stamps feature:

- The 409 trust gate now distinguishes reason 'unreviewed' (community
  code not vetted) from 'unverified' (no pinned checksum, possible even
  for first_party), each with an accurate message; the risk dialog
  carries the reason and matches its title/body to the actual case
  instead of always claiming 'unreviewed community extension'
- Unreviewed registry entries are now developer-stage content: hidden
  from the Marketplace catalog and refused at the install endpoint
  (404) unless the panel runs in a development context (Flask debug /
  testing config, or the dev_mode setting). Hidden means not
  installable; the 409 acknowledgment flow only exists where they list
- dev_mode setting description mentions the Marketplace effect
- tests: reason assertions on both 409 cases; catalog visibility and
  404-on-hidden-install coverage (38 passing)
2026-07-22 23:18:33 -04:00
Juan Denis 02bb7666e2 Marketplace: hash-bound review stamps and trust levels for registry extensions
The registry index (schema v3) can now carry a review object per entry
(reviewer, date, sha256, optional commit/notes) asserting that someone
inspected the exact artifact being listed. The stamp is bound to the
artifact hash: review.sha256 must equal the entry sha256, so any change
to the released zip invalidates the review.

- registry_service: keep review through normalization (validated),
  derive trust per entry: first_party, reviewed (stamp matches the
  pinned hash), or unreviewed
- marketplace install route: unreviewed (or sha256-less) entries return
  409 requires_acknowledgment unless the admin posts
  acknowledge_risk: true; acknowledged installs record trust in the
  audit log. Consent gate, consistent with ADR 0002
- Marketplace UI: Reviewed badge (reviewer + date tooltip) and
  Unreviewed warning badge on registry cards, plus a risk confirmation
  modal that retries the install with acknowledge_risk
- tests: trust derivation and the install gate (409 + acknowledge path)
- docs: EXTENSIONS_REGISTRY.md documents the field and the flow
2026-07-22 22:45:57 -04:00
Juan DenisandClaude Opus 4.8 3f3bff3132 test: bump BASELINE_COUNT ratchet 2507 -> 2534 for the batch's new suites
Locks in the proving tests added across plans 52/53/55 (runtime-frontend recovery
suites, db-snapshot relocation, generic fail2ban jail, gamekit framework, minecraft
install wiring, html-sink sweep) so they can't silently disappear.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:22:43 -04:00