648 Commits
Author SHA1 Message Date
6e096fa286 Add real preview deployments and strengthen core boundaries (#184)
* fix(security): honour the failed-login time window (#180)

check_failed_logins computed a cutoff from since_hours and never used
it, so the Security -> Events "Failed login attempts (24h)" card and
its alert counted every failure still in /var/log/auth.log or
/var/log/secure: up to a week of SSH attempts under the usual weekly
log rotation.

Each matching line is now dated from its stamp, either RFC 3339
(rsyslog's high-precision format) or the traditional BSD syslog stamp
(no year, so a December line read in January is placed in the previous
year), and lines older than the cutoff are skipped. A line whose stamp
is not recognised is still counted rather than hidden. The log paths
move to SecurityService.AUTH_LOGS so the tests can point at a temp
file.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: separate backend startup phases and guard ordering

* refactor: route file operations through capability-checked adapters

* chore: bump version to 1.11.25 [skip ci]

* refactor: split File Manager state and views with regression guards

* test: enforce core boundaries and document extension rules

* test: cover File Manager tree races and listing failures

* fix: preserve File Manager deep links on initial mount

* refactor: keep File Manager download operations in the action hook

* fix: start agent maintenance only in the guarded runtime phase

* test: capture nginx rendering contract before extraction

* refactor: separate nginx configuration rendering from host operations

* test: enforce pure nginx configuration boundaries

* fix: require provisioned runtime before advertising previews

* feat: expose value-free effective environment provenance through app access boundary

* feat: add effective environment inspector with browser regression checks

* feat: provision isolated generic previews with retryable resource cleanup

* feat: queue preview automation and preserve cleanup ownership through app deletion

* feat: add pinned preview deployment controls and queued lifecycle feedback

* fix: preserve container port semantics for previews of slot deployments

* fix: preserve search focus and ignore stale list responses

* feat: run approved assistant deployments through persistent recipe operations

* test: run search and assistant deployment regressions in CI

* fix: refresh preview API and controller boundary inventories

* docs: refresh controller migration count after preview extraction

* fix: bind file editor previews to their storage target

* chore: bump version to 1.11.26 [skip ci]

---------

Co-authored-by: Max Freedom Pollard <272618364+MaxFreedomPollard@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-10-24 15:31:39 -04:00
Juan Denis 1a6335e7cf fix: remember agent IP alerts across panel restarts 2026-10-23 18:05:15 -04:00
Juan Denis b85cc18220 fix: require patched PHP runtimes in WordPress templates 2026-10-23 18:05:08 -04:00
Juan Denis fe42d19048 Feature templates the registry marks as featured
The Templates page only featured a hard-coded list, so `featured: true`
in a template (Prompture Hub, Torre) never reached the Featured-first
view. The page now honors the flag alongside the list, and bundled
templates carry it into the catalog and the published index.
2026-10-11 16:41:38 -04:00
Juan Denis b86c03dc6b Don't 500 on uptime stop when its data dir is unwritable
Stop now persists the tracking flag, and the save created the data
directory unguarded. Where that fails (an unprivileged host or CI
runner) the save is now logged and reported false, as its docstring
already promised, instead of raising.
2026-10-11 00:52:44 -04:00
Juan Denis 498901e70e Keep the panel booting when uptime can't resume
Resuming uptime tracking at boot read the history through a helper
that first created /var/lib/serverkit. The image runs unprivileged, so
that makedirs raised and the app never started. Reading now creates
nothing, and a failed resume is logged instead of aborting boot.
2026-10-11 00:50:14 -04:00
Juan Denis f3145289c8 Time out version and status probes after 60s
--version, which, postconf, pip list/freeze and mailq had opted out of
the run_checked default with timeout=None, so a wedged binary held a
request thread forever. Installs, migrations, dumps and restores keep
no timeout.
2026-10-10 22:22:40 -04:00
Juan Denis 6400b76a7d Drop imports that shadowed module-level names
Function-local re-imports of User, WorkspaceService and secrets, and a
local list named secrets, hid the module names they duplicated.
2026-10-10 22:20:03 -04:00
Juan Denis f840744341 Drop an unused install-dir constant
disk_reclaim_service read SERVERKIT_DIR, which app/paths.py treats as
the /var/serverkit data root, and never used the value.
2026-10-10 22:10:27 -04:00
Juan Denis bee7a4a1aa Fix stopping an app that deploys on compose slots
A function-local import of SlotDeployService further down
stop_application made the name local to the whole function, so the
compose-slot branch raised UnboundLocalError before reaching it: Stop
on such an app always failed. The module already imports the class.
2026-10-10 21:55:33 -04:00
Juan Denis 61994b248e Trim history from the deploy preflight docstring 2026-10-10 21:34:02 -04:00
Juan Denis 95e2e6e6da Move extension zip fetching into plugin_fetch
GitHub release resolution and the SSRF-guarded download (redirect
re-validation, byte caps, token scoping) were 250 lines of
plugin_service unrelated to installing. plugin_service re-exports the
names its own code calls, so existing monkeypatches on it still land;
tests that exercised the fetch internals directly now target
plugin_fetch.
2026-10-10 21:32:19 -04:00
Juan Denis 3451d5ba2c Split TemplateService into mixin modules
template_service.py held port allocation, magic-variable resolution and
the template catalog alongside install/update in one 2,800-line class.
Those three groups move verbatim into TemplatePortsMixin,
TemplateMagicMixin and TemplateCatalogMixin, each in its own module;
TemplateService inherits them, so its surface (106 attributes, same
classmethod/staticmethod kinds) and every caller are unchanged. Tests
that patched module globals now patch the module the code lives in.
2026-10-10 21:10:14 -04:00
Juan Denis ec8cd9462e Bound Docker metadata reads with a timeout
ps, inspect, info, version, tag and the network/volume inspects ran
with timeout=None, so a wedged dockerd held every request thread that
asked, forever; the panel runs a single worker process. They now give
up after 30s, and the Containers page reports Docker unreachable.
Pull, build, logs and compose stay unbounded on purpose.
2026-10-10 20:54:12 -04:00
Juan Denis ddaeb549ad Remove bare plan-section citations
Comments cited plan sections as "(§G3)" or "(§8 unification)", which
point into the same git-ignored plans the plan-number citations did.
References next to a real document (docs/REDESIGN_MAP.md §3) stay. Code
and runtime strings are unchanged (AST and token comparison).
2026-10-10 20:25:49 -04:00
Juan Denis e94537d0a4 Cut docstrings that narrate refactor history
The worst cases the audit named: docstrings and header comments that
told how three old copies drifted, or quoted subprocess counts that
were stale the day they were written, instead of stating the contract.
Each now says what the function does and keeps one line of why. Also
drops the always-zero n_cloud counter that existed only to keep a boot
log line's or-chain intact.
2026-10-10 20:21:00 -04:00
Juan Denis bf5405df9e Move the health check's database probe into a service
The previous commit ran SELECT 1 directly in the route, a new
persistence crossing in the API layer that the controller-boundary
ratchet rejects.
2026-10-10 18:48:53 -04:00
Juan Denis cc8c74e1ef Compute visible workspace ids in one place
The same "active workspace, else my workspaces, else all for an admin"
rule was written out in projects, environments and search. It now lives
in WorkspaceService, with a request-aware wrapper for routes.
2026-10-10 16:56:42 -04:00
Juan Denis daf6314791 Share one OAuth implementation across git providers
GitHub, GitLab and Bitbucket each carried their own copy of config
read/update, the authorize hop, the code exchange and the stored
connection. One implementation now reads the few real differences from
a per-provider table (Basic-auth token exchange and URL-less scopes for
Bitbucket, no grant_type for GitHub, profile field names). New tests
pin those differences; they passed on the old code before the change.
2026-10-10 16:43:14 -04:00
Juan Denis 6068b099dc Replace nine per-app access helpers with one
Nine route files each loaded the live app and checked access, with
four return shapes, docstrings promising error tuples the code never
returned, and 41 dead 'if err: return err' branches. require_app()
backed by ResourceGrantService.get_app_for does it once; conceal=True
keeps the 404-for-denied answer the previews, WAF and slot routes
already gave, so no route's status changes. Nine query sites leave the
API layer.
2026-10-10 16:36:01 -04:00
Juan Denis d7234b822d Delete nine backend functions with no callers
Each was referenced only by its own definition across the panel, its
tests, scripts, builtin extensions and every sibling extension repo
(~350 lines). Imports they alone used go with them.
2026-10-10 16:09:29 -04:00
Juan Denis 6296887bad Remove internal plan citations from app code
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
2026-10-10 15:50:45 -04:00
Juan Denis c1f909b966 Log best-effort failures instead of dropping them
A lost "backup failed" alert, a broken live deploy log and missing
deploy or build log lines were all except-pass, so none left a trace.
They still never raise; live-emit failures report once per kind so a
dead socket path is findable without flooding the log.
2026-10-10 14:26:59 -04:00
Juan Denis c44da15993 Hold allocated ports until the install claims them
The allocator checked app rows, containers and a bind test, none of
which exist while an install is still being planned, so concurrent
installs could be handed the same port. Port variables now claim the
port for 15 minutes and every lookup skips live claims. An exhausted
range raises a 409 instead of returning a random port that could
collide with anything.
2026-10-10 14:14:09 -04:00
Juan Denis 9b0323ebfb End tail -f when a log stream stops
The stop flag was checked between lines, but readline() blocks until
the log grows, so closing the viewer on a quiet log left a sudo tail -f
running for the life of the panel. A watcher now kills tail on stop.
The socket emitter also gave up after 30s without a line, silently
leaving the client subscribed to nothing; it now runs until the stream
is stopped or replaced.
2026-10-10 14:07:22 -04:00
Juan Denis db5f4df74a Show Docker list errors and ignore stale loads
The API now answers 503 when docker ps fails instead of an empty list,
so the Containers page no longer says "No containers" on a host
running many. The Containers and Images tabs show that error, and a
late answer from the previously selected server can no longer replace
the current server's list. Also fixes the local branch of the remote
Docker listing, which passed all= to a parameter named all_containers
and always failed.
2026-10-10 14:01:56 -04:00
Juan Denis 5fb35e8cc0 Fail scheduled backups whose offsite upload fails
The upload helpers report failure in their return value, which the
scheduler ignored: the schedule read success and no failure alert went
out while the only copy sat on the box being backed up. Auto-upload
failures are now logged instead of silently dropped.
2026-10-10 13:46:17 -04:00
Juan Denis e6658d70b3 Resume monitoring and uptime tracking at boot
Both loops only started from an API call, so every restart silently
stopped threshold alerting while the UI still showed it enabled. Uptime
tracking now persists its on/off flag, the monitor loop runs inside an
app context so its telemetry writes stop failing, and SMTP gets a
timeout so a dead mail host cannot wedge the loop.
2026-10-10 13:08:32 -04:00
Juan Denis 6901a4ff91 Run fleet threads inside the live app
Rollout and queued-delivery threads called create_app(), rerunning
migrations, plugin loading and the boot reconcile against the live
panel. The upgrade fan-out thread had no app context at all, so its
first query raised and online agents were never told to upgrade.
2026-10-10 13:06:09 -04:00
Juan Denis 4e45ecb5f5 Move the new auth lookups behind the model/service
The server access helper and the app-name clash checks queried the DB
from the API layer, which the controller-boundary scan rejects. They
now go through WorkspaceService.get_accessible_server and
Application.name_clash_error; the baseline drops by three.
2026-10-10 09:56:44 -04:00
Juan Denis 752fb83919 Compare legacy deploy webhook tokens safely
The token was compared with != (timing leak), and an app with no
webhook_token accepted a request that sent none, since None != None is
false. Require both tokens and use hmac.compare_digest.
2026-10-10 09:32:03 -04:00
Juan Denis acb4f4745c Keep panel secrets out of build and deploy scripts
Custom build commands, Python app commands, repo deploy scripts and
registry template scripts ran with os.environ.copy(), handing code from
a repository or registry the JWT signing key, the session key, the
database URL and every SERVERKIT_* credential. app_process_env() builds
their environment without those.
2026-10-10 09:30:51 -04:00
Juan Denis 7c6800a946 Verify webhook signatures before deduplicating
event_id is unique and the duplicate check ran before the signature
check, so an unsigned request claiming the next event's id was stored
under it and the real delivery was then dropped as a duplicate. Rejected
requests are now logged under a fresh id.

The id header was also never read over HTTP: Werkzeug presents it as
X-Webhook-Event-Id, which neither exact spelling matched, so every
request got a random id and deduplication never happened.
2026-10-10 09:25:22 -04:00
Juan Denis a8c5237d32 Refuse internal targets for user-set webhook URLs
Event subscriptions, the webhook gateway's forward_url and the
Discord/Slack/generic alert hooks posted to whatever URL a user typed,
from inside the network: the panel's own API on loopback, LAN hosts,
cloud metadata. Subscriptions followed redirects and stored the reply,
and the gateway forward can be triggered without logging in.

safe_http.post refuses non-public addresses, connects to the address it
checked so DNS can't swap it afterwards, never follows redirects and
caps the body it keeps. SERVERKIT_ALLOW_PRIVATE_WEBHOOKS=1 restores
internal targets for operators who need them.
2026-10-10 06:52:32 -04:00
Juan Denis 12f6dff430 Keep host paths and site names out of user hands
POST/PUT /apps let any developer set root_path, compose_file,
systemd_unit and managed_by, and store a raw name. Root-run code later
runs docker compose from that root, renders the name into an nginx
vhost and joins it onto sites-available for rm -f. The host-binding
fields are now admin-only (as /manual already was), create slugs the
name and checks for clashes, and a rename must already be a slug.

The nginx site methods all share write_vhost's plain-filename check, so
enable/disable/delete can no longer be pointed outside the site dirs.
2026-10-10 06:46:18 -04:00
Juan Denis fe8920e813 Restore app backups into the requested path
The archive's top-level dir is the app's original directory name, and
it was extracted into restore_path's parent: with a custom path the live
dir was moved aside, the files landed under the old name, and the call
still reported success with nothing at restore_path. Extract into a
staging dir, move it into place, and put the live copy back on failure.
2026-10-10 06:34:56 -04:00
Juan Denis 151d84f3b3 Keep cron jobs added in the same second apart
Job ids had one-second precision, so a second job added within the
same second overwrote the first's metadata and was silently lost.
Re-enables the test that has documented this since July.
2026-10-10 06:33:12 -04:00
Juan Denis 76432b1231 Fix gzip database backups and restores
A gzip.open() handle passed as a subprocess stream hands the child the
raw fd, so mysqldump/pg_dump wrote plain SQL inside a gzip header and
restores piped compressed bytes into mysql/psql. Backups, policies and
pre-deploy snapshots all produced files that could not be restored.

run_to_gzip/run_from_gzip pump the bytes through Python instead, with
stderr in a temp file so a noisy child can't deadlock the pipe. The two
hand-rolled db_sync dump pipes move onto them too.
2026-10-10 06:32:13 -04:00
Juan Denis fca18e9055 Gate server terminal, re-pair and move on workspace
Creating a terminal, regenerating the pairing token and moving a server
checked the role but not workspace membership, so a developer outside a
workspace could get a shell on its servers or re-pair them to their own
agent. REST now uses the rule the Socket.IO rooms already applied, in
one helper. The Default workspace counts as shared: servers created
without a context land there and nobody is a member of it.
2026-10-10 06:25:35 -04:00
Juan Denis d60eaf466b Protect the idle update slot in the file manager
PROTECTED_ROOTS covered only the running /opt/serverkit-<x>, so the
previous version's .env (JWT key) in the sibling slot was readable by
any role with files.read, including the default viewer.
2026-10-10 06:15:56 -04:00
Juan Denis 97a96d963b Give common API errors stable codes and service wording
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.

The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
2026-10-04 18:59:25 -04:00
Juan Denis 2f52c87a39 Add /ports/check and /ports/suggest
The deploy paths already know which host ports are taken (app rows,
slot rows, Docker mappings, a bind test) but only used that to pick a
port silently. Exposing it lets the shared port input say "8001 is
used by shop-api" and offer the next free port, instead of failing
at deploy time.
2026-10-03 22:46:54 -04:00
Juan Denis 4c70658f90 Harden the Cloud tunnel per review
Session keys must name their purpose, and both loopback legs skip
environment proxies so the grant and bearer token never leave the box.
In the tunnel the API always goes through the relay, which attaches the
grant, and the two raw root anchors become router Links so they stay
under the /t/<device> prefix.
2026-10-01 22:47:48 -04:00
Juan Denis 4eecee8bf0 Reject trailing newline in Postfix install hostname
re.match with $ accepted 'host\n'. Use fullmatch and pin the
GHSA-mc93-rc3x-fpgq fix with a regression test so the bash -c form
cannot return.
2026-10-01 22:22:00 -04:00
Juan Denis c7b6f145d1 Serve the panel through the Cloud relay tunnel
Until now the connect client refused every http and sio stream, so
Open panel in ServerKit Cloud showed a blank page. The client now
bridges them to the loopback listener, the panel serves its page
under the relay's /t/<device>/ prefix, and the SPA prefixes its
router, API and Socket.IO paths from it. The Cloud grant is checked
against Cloud's JWKS before signing in the panel user with the same
email; panel two-factor still applies.
2026-09-29 23:19:34 -04:00
Juan Denis 9eafe6709e Point compose app actions at the live slot project
Plan 87 phase E, compose part. For a compose app on slots, the live
release is a slot project, not the project its own compose file names.
Status maps to the live project only (a stopped standby never reads as
a degraded app), start/stop/restart and sleep/wake act on it, container
listings show the live and standby projects, and an image update runs
as a queued slot deploy instead of recreating the stack in place.
2026-09-28 22:27:13 -04:00
Juan Denis d0b11714e7 Deploy compose apps through A/B slots
Plan 87 phase C. A compose app on slots runs each slot as its own
project, <name>-a / <name>-b, rendered from the merged compose config:
container_name stripped, only the web service published, on the
slot's loopback port, and every named volume pinned to the name it has
today. Without the pin compose prefixes volumes with the project name
and slot B would start on empty ones; the real-Docker leg checks this.

Stateful services (a database, cache or queue with a named volume)
cannot run twice. They move once, on an explicit previewed split, into
a shared <name>-data project that both slots reach by the same names.

The webhook, template-update and rollback paths deploy through the
engine instead of stopping the stack. The --scale web=2 rolling
restart and the Webhook.zero_downtime flag are deleted: it assumed a
service named web, ignored its own health result, and no UI set it.
Compose slot images get immutable per-deployment tags and are pruned
with the rest.
2026-09-28 22:26:44 -04:00
Juan Denis 0749508599 Run release commands and DB snapshots before the switch
Plan 87 phase D. Both slots share the database, so what changes it
runs once, before the switch, while the old release still serves:
the app's managed databases are dumped (a warning, never a blocker),
then the release command runs in a one-off container of the new
image. A failed release aborts the deploy with the live slot
untouched. The command comes from the app's setting, the Procfile
release: line, or serverkit.yaml; a rollback never re-runs it.

Rolling code back never rolls the database back: after a switch
back the slot card offers the pre-deploy snapshot as an explicit
second step with a data-loss warning. stop_old_before_release is
the announced-downtime fallback for schema changes that cannot stay
backward-compatible for one deploy.
2026-09-28 21:41:52 -04:00
Juan Denis c697e7247b Show slot deploys in the console and app detail
Plan 87 phase F. A slot deploy job lists its real stages (Preflight,
Build, Boot slot X, Health gate, Switch, Watch) and advances through
them as the engine reaches each one, so PipelineStrip renders them
unchanged. The overview gets a slot card: which release is live,
which is the standby and how long it stays warm, with Switch back;
when slots are off it lists why the app does or does not qualify.
Settings gains the opt-in, watch window, standby minutes and the
shared-volume confirmation.
2026-09-28 21:33:45 -04:00
Juan Denis 8bd78d69cd Point container actions at the live slot
Plan 87 phase E, single-container part. Start, stop and restart act
on the live slot's container (stopping the app also stops its warm
standby), container listings show the live slot as web and the
standby separately, and status, stats, logs and the terminal follow
app.container_id, which the switch now updates.

A vhost restore point records which slot the captured file pointed
at, read from the file itself, and restoring it starts that slot
first, so a restore never points nginx at a stopped standby.
2026-09-28 21:32:41 -04:00