Full VM installs and Test Sandbox full mode now record a short
post-install runtime observation (60s settle + 120s sampling) with an
explicit measured/unavailable/failed/interrupted status that never
rewrites install success. Adds the standalone scripts/measure-runtime.py
sampler, the measure-installed-runtime.py harness wrapper, E2E report
rendering, results-API summaries, CI coverage and METRICS.md accounting
boundaries.
Flask refuses register_blueprint once the app has handled its first
request, so on any running panel — where login and dashboard requests
long precede the first extension install — the hot-load in
_register_plugin_blueprint always raised, the installer only logged a
'will load on restart' warning, and the extension's routes stayed
missing. Its pages rendered (frontend is static) while every API call
failed: GETs were swallowed by the tabs' console.error handlers, and the
first POST fell through to the static-file route, surfacing as the 405
reported for the Malware Scanner custom-path scan.
_register_blueprint_live waives Flask's setup guard for the duration of
the registration call. That is safe in this deployment shape: the panel
runs a single gunicorn worker with threads, so the process installing
the plugin is the process serving its routes, and werkzeug rebuilds its
URL matcher lazily on the next bind while in-flight requests keep the
matcher they already bound.
PR #145 closed the prefix-sibling hole in LogService.is_path_allowed but
left the gap its author named: unlike FileService, the log viewer had no
PROTECTED_ROOTS exclusion. On the install.sh layout the install dir sits
under the allowed root /opt, so the Log Files tab could read — and,
through clear_log, truncate — the backend .env, the SQLite instance dir
and the generated deploy config, the same hole the file manager closed
for GHSA-rm3m-9mvw-68fh.
Share FileService.PROTECTED_ROOTS rather than copying the list, so the
two surfaces cannot drift apart, and reject protected paths before the
allowed-root check. Panel logs live under /var/log and SERVERKIT_DIR,
which are untouched, so legitimate viewing is unaffected.
LogService.is_path_allowed resolved the requested path with realpath and
then asked whether the result merely started with one of
ALLOWED_LOG_DIRECTORIES. That is a text comparison rather than a
containment test, so a directory whose name only begins with an allowed
root was accepted as if it sat inside it. With the roots shipped at
backend/app/services/log_service.py:30-36, both /opt-private and
/homestead pass on the strength of /opt and /home, which is not what the
comment above the list at line 29 says the list does.
FileService guards the same roots and gets this right at
backend/app/services/file_service.py:96-100, comparing real_path == root
or real_path.startswith(root + os.sep). The same form appears at
backend/app/api/deploy.py:197, backend/app/services/upload_service.py:66
and backend/app/services/linked_panel_agent.py:251. LogService does not,
and its guard also gates truncation through clear_log at
backend/app/services/log_service.py:367.
The loop now compares against the separator-terminated root, so a
sibling sharing a name prefix is rejected while everything genuinely
inside a root still resolves, including paths reached through .. because
realpath has already collapsed them. A regression test covering the
shipped /opt constant, a sandboxed root and the clear_log door ships
with it.
Panel side of the managed profile: the panel receives a presentation
profile from Cloud over the existing command channel, stores it,
acknowledges it, and hides or simplifies fleet-owner-only surfaces
until the profile is lifted or the link is revoked.
- ui.managed_profile command handler with managed-profile.json
storage, audit entries, and ack-by-output; profile is cleared on
disconnect and on consent revoke
- GET /api/v1/connect/managed-profile for the local UI
- ManagedProfileContext with capability-driven filtering of sidebar
items, command palette actions, and sidebar settings; ManagedCard
explains the managed state
- In-page collapses for Templates deploy, About self-update, and
Backup schedules/storage (restore stays available)
- Settings gains a ServerKit Cloud tab showing link and profile state
Local verification: 153 backend tests pass, 198 frontend node tests
pass, lint clean, build ok. No live policy has been pushed to a real
panel yet.
The catalog certification test now reads SERVERKIT_TEMPLATES_DIR and
SERVERKIT_TEMPLATES_INDEX, so the official template registry's CI can
run the installer's own rules against its tree before a merge publishes
it to every panel. The build_repo_index() check stays bundled-only and
skips under the override.
The changelog had stopped at "1.11.1 - Unreleased" while 1.11.2 through
1.11.4 shipped: the Connect revocation, JWKS refetch, metrics-warning and
sidebar fixes are now recorded under the published 1.11.4 release.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JtgcyJ75YEozymcrWDyedu
_poll_session reports 'revoked' when ServerKit Cloud refuses the poll
with the revoked reason, but the loop only honored 'stopped' and fell
through to re-probe the websocket: ws refused, poll revoked, ws
refused — forever, and the panel never wrote the revoked state. This
is the path a panel behind a close-code-stripping edge depends on, old
builds unconditionally and current ones whenever they are in limited
mode. The poll outcome now gets the same terminal handling as the
websocket one.
The production edge strips non-standard WebSocket close codes, so the
relay's 4009 never reached the panel: a revoked panel kept reconnecting
forever instead of stopping. ServerKit Cloud's relay now sends the
reason as a text frame before closing, and the panel honors it both
during the hello exchange and mid-session, ending the session as
revoked either way. The numeric code still serves direct connections,
and the long-poll fallback's HTTP refusal is unchanged.
ServerKit Cloud mints each signing key the first time it is used and
rotates keys on its own schedule, so a command can legitimately arrive
signed with a key the connection's cached JWKS predates. The panel used
to refuse it as untrusted; it now refetches the JWKS once on an unknown
kid and only refuses a key that is still unknown afterwards.
A metrics build that keeps raising, and a sampler that keeps producing
nothing, both used to log at debug level: ServerKit Cloud showed no
telemetry and the panel had nothing to act on locally. A failing build
now warns on the first failure of a streak and logs its recovery, and
five consecutive empty collects warn that the panel sampler may not be
running. The healthy path stays quiet.
Raises the test-count floor to 5271 (check_test_count.py --update): the
three new publisher tests plus the floor never having been bumped for
the 63 cases added since the last update.
connect_inventory builds inventory/1 (applications keyed on the row id so a
rename is not a new application; slug, branch and commit instead of a repo
URL; a WordPress admin URL only where this panel installed it; databases as
names, engines and states) and diagnostics/1 (busiest processes and mounts,
never a command line). connect_logs tails the panel and web error logs by
offset with a generation per rotation, reads application output and build
logs through the existing services, applies every Cloud bound locally,
redacts before sending, and stops on collect: false.
The relay client publishes inventory on connect and every five minutes,
diagnostics each minute while Cloud allows it with a fifteen-minute back-off
after a refusal, and logs every five seconds while sources are wanted.
Older Clouds keep answering the facts stream as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JtgcyJ75YEozymcrWDyedu
Keep connection persistence behind the service boundary, use typed errors and shared JSON parsing, and refresh reviewed inventories. The new GET connection catalog accepts API keys so API clients managing connections can discover IDs without exposing credentials; existing JWT-only routes stay unchanged. Add regression coverage and release notes.
The tree backup rsync copied backend/instance/serverkit.db even though the
same database had just been written as the pre-upgrade copy, and rollback
only ever restores from that copy. On a 450 MB database that made every
update cost ~900 MB of backups, which is how small VPSes kept filling up.
Exclude the SQLite files from the tree snapshot. Retention is unchanged
(SERVERKIT_BACKUP_RETENTION, default 3, plus the 15% disk budget).
Add a test that every shipped nginx vhost forces a JavaScript MIME type for
/serverkit-vendor/*.mjs and that the regex cannot capture the
/api/v1/plugins/<slug>/assets/... route.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWc84WDsWR7M55Mn7qtpie
generate_bitbucket_authorize_url() takes a redirect_uri and leaves it
out of the query string, so Bitbucket returns the user to whatever
callback is registered on the OAuth consumer rather than the one the
panel asked for. complete_bitbucket_callback() then posts that same
redirect_uri to the token endpoint, so the two halves of one flow name
different callbacks.
Both sibling generators in the same file send it
(generate_github_authorize_url line 207, generate_gitlab_authorize_url
line 446), as does generate_auth_url in sso_service.py; Bitbucket was
the only one that dropped it. GET /source-connections/bitbucket/authorize
already rejects a request without redirect_uri, so the value was always
present and always discarded.
Adds the parameter, plus a regression test parametrised over all three
providers so the next one added is held to the same contract.
Three defects, found together because they are the same failure wearing
three hats: monitoring that looks configured and is not.
Fleet alert thresholds were never evaluated. check_fleet_thresholds was
written, was correct, and had no caller anywhere in the backend, so a
threshold set on the Fleet page was checked against nothing and a server
could sit at 99% CPU for a week in silence. It now runs every minute as a
built-in scheduled job alongside every other recurring task, and each
alert it opens or resolves goes to admins through the Notification Bus.
An evaluation nobody is told about is most of the way to no monitoring.
The agent heartbeat was throwing metrics away. It built its own
ServerMetrics row and filled five of the model's thirteen columns,
dropping memory_used, disk_used, the network totals, both network rates
and extra. The two rates are exactly what the threshold checker reads, so
a network threshold could never fire for an agent-reported server: the
data needed to evaluate it was discarded on the way in. Heartbeats now go
through ServerMetricsService, the writer everything else already used.
Metric history outlived its own retention. The history and aggregate
endpoints offered 7-day and 30-day windows while cleanup deleted every
sample after seven days, and two constants described hourly and daily
aggregate retention that nothing produced. A 30-day chart came back
holding at most seven days and said nothing about it. Both endpoints now
return a coverage block — the retention in force, the earliest point they
can answer for, and whether the window asked for was truncated — and the
two constants that promised retention nobody implemented are gone. Real
rolled-up aggregates are the other way to fix this and would need their
own tables and writer; until they exist, saying what we have is the
honest answer.
Two more things surfaced while writing the tests.
get_aggregated_metrics grouped by date_trunc, which is PostgreSQL's alone,
so the endpoint raised "no such function: date_trunc" on every SQLite
deployment — one of the two supported databases. Buckets are now built
with the right function for the database in use.
And an escalation is not a recovery. warning -> critical closes the
warning row, so both a genuine recovery and a worsening breach end up as
status='resolved'; announcing the second as "recovered" would tell an
operator the problem went away at the moment it got worse. Supersession is
tracked separately and is not notified twice, because the new alert
already says what happened.
The two built-in-job counts in test_jobs.py move from 16 to 17 and 24 to
25, and BASELINE_COUNT is raised for the new tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
On an existing install the startup schema sync runs before alembic and
added users.auth_version as a bare nullable TEXT, so migration 097 saw the
column and skipped its '0' default. Every pre-existing user then carried
a NULL auth_version that no JWT claim could match: 2FA verification and
plain logins both failed with "Invalid or expired token" (reproduced on
the builditdesign test box after `update.sh --branch dev`).
- _fix_missing_columns now renders a literal server_default as a SQLite
DEFAULT, so pre-alembic column adds backfill existing rows.
- migration 097 always backfills NULL/empty auth_version to '0'.
- tests pin both paths plus the default rendering.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVRnRXqNMZagqQ3RTqG5pz
Centralize backup schedule calculations and shared request/form flows, encode query values, and move repository finalization into its service. Add browser CI coverage, repair stale test fixtures, and provide opt-in request and build measurement tools.
Scope built-in tools to the live caller, recheck write authorization after confirmation, redact structured results, and fail closed when protections fail. Bound chat inputs, concurrent turns, and cancellation-aware streaming queues.
Require valid access-token sessions, scope application and server rooms, and enforce run visibility consistently across sockets and polling. Add positive-delivery and cross-user isolation regressions.
Copilot review on #135:
- `_repo_steps()` now falls back to port 3000 when `params.port` is not
numeric instead of raising ValueError mid-stream.
- `params.pace` must be a positive number; zero, negative and non-numeric
values are ignored so `line_delay` can never go negative.
CI: the ubuntu integration job sat in "Install Python & pytest" for ~2 h on
a wedged apt mirror, and Version Bump waits on every sibling. Cap the job at
15 minutes and give apt retries, a fetch timeout and non-interactive dpkg
conffile options.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LVRnRXqNMZagqQ3RTqG5pz
Add a `repo` scenario to the simulated deployments so a demo run reads like a
real deploy of a named service: clone, validate, build, route, start and
health-check lines carry the app's own name, repository, branch, port and
health path. The simulate endpoint also accepts a `title` (shown verbatim by
the Deploy Console) and `params.pace` to slow the stream for recordings.
Used by the walkthrough videos to show the Deploy Console streaming live
inside the read-only demo image, which has no git or docker.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SiGEPH8S231SMwFWTd76b6
ServerKit Cloud counts a host once per organisation however many panels
report it, and it can only do that if it can recognise the same machine
twice. This panel was reporting each agent's own id and nothing else, so an
agent managed by two panels — or re-reported after a re-pair — was counted as
two servers and took two slots off the customer's plan.
The agent's machine id and key fingerprint are already on the enrolment row it
paired with, keyed by the server it was claimed onto. Both now go out with
every host entry. An agent with no enrolment row is still reported, with the
ids we have: a host left out of the list is a server Cloud cannot show at all.
Files that matter:
- backend/app/services/connect_client.py _gather_hosts reads the enrolment
- backend/tests/test_connect_hosts.py
Verified: backend/venv/Scripts/python -m pytest tests/test_connect_hosts.py -q
→ 2 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013S1pQEJFU8P7US5MRpJFmi
Four things the guards and the review caught, all of them real:
- app/api/connect.py: GET /connect/status carried a bare @jwt_required(),
which the jwt-only census forbids for a new route (609 vs a ceiling of
608). It is an admin read of pairing state, so it takes @admin_required
— the same door the rest of the admin surface uses, API-key capable by
way of auth_required() like its neighbours.
- connect_policy._systemctl ran systemctl through raw subprocess, one over
the raw-subprocess ceiling. run_privileged() adds sudo when the panel is
not root, so the fail2ban repair now works off a packaged install
instead of reporting a permission error nobody can act on.
- connect_client.hello_headers assumed the end-to-end key fields were
always there. build_hello() drops them rather than fail the connection,
so a panel that could not read its X25519 key would KeyError on the
long-poll fallback — exactly the path taken when things are already bad.
- status() called resolve_cloud_url(), which raises on a URL with no
scheme. A typo in SERVERKIT_CLOUD_URL should surface when you pair, not
turn `serverkit connect status` and the Settings panel into an error.
Both connect threads are now classified in the thread-ownership registry:
the relay loop is a process loop, and a signed command is bounded delivery
whose record lives in the control plane.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
A second keypair, X25519, published in the signed hello with an Ed25519
signature over it so ServerKit Cloud can tell it is really ours — and so a
relay cannot substitute one of its own, because it does not have this
panel's identity key.
Command arguments then arrive encrypted to that key. The relay carries a
blob. That is what makes it safe for Cloud to send an object-storage
credential through it, which until now crossed the relay in the clear.
A payload this panel cannot open is a refused command with a sentence
saying what to do, never a handler running on arguments we guessed at.
- backend/app/services/connect_keys.py — the key (a raw 32-byte file, not
PEM: it is a key with no certificate story around it), the signed hello
fields, and open_sealed.
- backend/app/services/connect_client.py — publishes it on WebSocket and
on the long-poll fallback, so limited mode is not quietly less private.
- backend/app/services/connect_commands.py — unseal before the handler.
- backend/tests/test_connect_sealing.py — 13 tests, including the control plane's
half of the scheme written independently from the specification, so the two
implementations cannot drift apart silently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
A facts document — firewall, SSH port, brute-force protection, the panel's
two-factor policy, its version, and when its backups last succeeded and were
verified — goes up the policy stream on connect and every six hours.
`policy.report` sends one now, which is what "Check now" presses.
The important part is what it does not claim. Each bundle is advertised in
`capabilities` only when its probe actually answered, so a rule Cloud cannot
get facts for reads "not checkable here" rather than marking a healthy
server non-compliant. Pending OS security updates have no probe here, so
that bundle is never advertised and its handler refuses honestly instead of
pretending.
Four repairs, each the panel's own path with its own refusals:
FirewallService.enable (which declines to lock SSH out), the require-2FA
policy with its grace window, systemctl start fail2ban, and the offsite
verification.
- backend/app/services/connect_policy.py
- backend/app/services/connect_commands.py — the three consent bundles
- backend/app/services/connect_client.py — the policy stream
- backend/tests/test_connect_policy.py — 17 tests, nothing touches the host
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
The panel can now run the signed commands Cloud sends, and the first four
of them are the Storage Hub's. A destination assigned in Cloud is tested
here before it is saved, written as `s3.path_prefix` (which every upload,
list and verify path already honoured), and reported back on a `storage`
stream. Unassigning hands it back and deletes nothing in the bucket.
- backend/app/services/connect_commands.py — verify a command JWT against
ServerKit Cloud's JWKS (signature, audience, exp, nonce, consent scope), run the
handler on its own thread, report running/succeeded/failed. An action
this version does not know fails with a sentence saying so.
- backend/app/services/connect_storage.py — storage.assign/unassign/
test/report, and the snapshot the storage stream carries.
- backend/app/services/connect_client.py — handle `command` opens, fetch
the JWKS per connection, publish a storage snapshot on connect and
every five minutes.
- backend/app/api/backups.py — GET /storage says when Cloud owns the
destination; PUT answers 409 rather than letting two writers race.
- frontend/src/pages/Backups.jsx — the local form is read-only while
Cloud manages it, with the sentence that says how to take it back.
- backend/tests/test_connect_storage.py — 23 tests, no network.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
On reconnect the client asks ServerKit Cloud what the current release is and
verifies the answer against the JWKS it already trusts before believing it —
the artefact lives on a CDN we do not control, so the signature is the only
reason to trust the URL in it.
- services/connect_updates.py fetch, verify, compare, and act: an install
the panel can update runs its own update path; one it cannot (Docker,
non-root) says exactly what to run on the host instead
- services/connect_client.py ask once per reconnect, at most hourly, and
never let a failed check take the connection down with it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
The panel already samples the host every 60 s into MetricsHistory for its own
charts. connect_metrics.MetricsPublisher turns the newest of those rows into
the summary Cloud expects, adds the counts Cloud alerts on (containers,
unhealthy apps, certificates near expiry, backup age) and hands it to the
relay socket on a `metrics` stream.
- services/connect_metrics.py build, buffer (five minutes, then drop rather
than send late), and apply the interval Cloud asks for
- services/connect_client.py publish on the interval from the WS loop and
read the stream close for Cloud's answer
Nothing new is collected: a minute the panel did not record is a minute Cloud
does not hear about. Every count source is optional — a panel without Docker,
domains or a backup policy sends null for those fields instead of failing the
whole summary.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
- app/services/connect_client.py: RelayClient background thread (WS hello
signed device_id|ts|nonce, 20s pings, jittered 1s->60s backoff with reset
after 2 stable minutes, >6 reconnects/10min flap log, close-code ->
state_reason mapping, terminal revoked, long-poll fallback with 15min WS
re-probe); connect-state.json runtime state merged into status();
disconnect() stops the client and removes the state file; run_doctor()
implements the connect check table
- app/services/connect_keys.py: strict load_keypair() for the transport
- app/__init__.py: start_client_if_paired() next to the linked-panel hook
- cli.py: connect status shows transport/last-seen/relay-instance; new
connect doctor command (exit 1 on hard-check failure)
- requirements.txt: websockets==15.0.1 (WS client; the SocketIO stack is
server-side only)
- tests/test_connect_relay.py: 21 unit tests for backoff, flap detection,
close-code mapping, hello signature, state file, loop transitions
- app/services/connect_keys.py: Ed25519 device keypair (0600, config dir),
fingerprint = sha256(raw pubkey)[:16] shown in groups of four
- app/services/connect_client.py: enroll/poll (signed PoP)/rotate against
the control plane, writes connect.json (0600) on claim;
status()/disconnect(); control-plane URL from --cloud / SERVERKIT_CLOUD_URL / https://app.serverkit.ai
- cli.py: 'connect' group (pair, status, disconnect)
- app/api/connect.py + core_blueprints.py: GET /api/v1/connect/status (JWT)
- tests/test_core_blueprint_registry.py: registry guard constants for the
new blueprint
The previous commit asked "does a compose file exist on disk?" and sent
everything else down the container path. That was too wide in both directions:
- test_start_app_authenticates_registry_before_compose_up seeds a
registry-bound app whose compose file has not been rendered yet, and
rightly expects `docker login` -> `compose up` -> `docker logout`. The
disk probe diverted it and the start 400'd.
- it also silently changed behaviour for any compose app whose file was
missing for an unrelated reason, turning a clear compose error into a
confusing "not been deployed yet".
Invert it. `_is_single_container_app()` is a positive test: a local app, no
compose file recorded, none on disk, and `buildpack_type` set -- i.e. one the
build pack deployed as a single container. Compose stays the default for
everything else, so no existing path changes.
Also seed the new test through `make_application` rather than constructing
Application(...) by hand, which is what test_seed_one_door_ratchet enforces
for files added after 2026-08-19.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Juan Denis