Commit Graph
597 Commits
Author SHA1 Message Date
Juan DenisandClaude Opus 5 a9d72ef7d3 fix(recycle-bin): restore a domain through write_app_vhost, not create_site
The teardown map for delete_domain landed after the hook shipped and found a
real hole in it.

When the deleted domain was the app's LAST one, delete_domain runs
disable_site() + delete_site() -- the sites-enabled SYMLINK is gone, not just
the server_name. The hook called NginxService.create_site and reloaded, which
rewrites the file and leaves it UNSERVED. Restoring the last domain of an app
looked like it worked and served nothing.

SiteDomainService.write_app_vhost is the correct seam and fixes three things at
once:

  1. it calls enable_site, so the symlink comes back;
  2. app_vhost_kwargs re-attaches a covering wildcard cert and the micro-cache
     flag -- a bare create_site drops both, so a restored site would silently
     fall back to plain HTTP;
  3. it handles every host-nginx app type. delete_domain only touches nginx for
     docker apps, so php/static/flask/wordpress vhosts still list the deleted
     domain; the rewrite is idempotent, so running it there repairs the
     asymmetry rather than preserving it.

New test pins the case that was broken -- restoring an app's only domain must
call BOTH create_site and enable_site -- and it fails against the previous
implementation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 08:19:35 -04:00
Juan DenisandClaude Opus 5 e54f467a5c feat(recycle-bin): on_restore for domains -- put the vhost back too
Deleting a domain did two things: tombstone the row and rewrite the app's
nginx vhost so traffic stopped. Restoring only undid the first, so a domain
could be "restored" and still not served. on_restore_domain rewrites the vhost
from the app's live domains -- the mirror of what delete_domain does -- and
reloads.

Adds a `pre_restore` hook to the registry, because "restorable" is a moving
target: deleting a domain FREES its name (the unique index covers live rows
only), so something else may hold it by the time you press Restore. Clearing
the tombstone then violates the index and raises IntegrityError at commit -- a
500 for what is really an explainable conflict. Now it refuses with the name of
the app that took it. Same guard for a parent Application that no longer
exists.

A restored domain that was primary does NOT become a second primary: whatever
stayed live keeps the role and this row returns as an alias.

The certificate is deliberately not re-issued. ssl_enabled may still be true
with cert files that expired while the domain sat in the bin, and quietly
talking to Let's Encrypt is not something a "put it back" button should do --
the domain returns on HTTP and SSL is re-enabled explicitly.

Five tests, including the two that would otherwise be 500s: name taken again,
and app gone. A failed vhost write warns and KEEPS the restore rather than
rolling back a row the user asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 08:15:41 -04:00
Juan DenisandClaude Opus 5 f49a3e6cfc feat(recycle-bin): soft delete for records, with restore and purge
Deleting is the only action in a control panel that repeating cannot undo, so
records a person destroys by hand now leave a tombstone and the Recycle Bin
hands them back.

SoftDeleteMixin adds deleted_at/deleted_by_id plus soft_delete/restore and
query_active()/query_deleted(). recycle_bin_service is a REGISTRY: a model
registers with a label function and an optional on_restore hook, so the API
lists and restores across types without knowing what a domain is. Adding the
next soft-deletable model is one register() call.

Applied to Domain (a real, destructive case -- deleting one used to lose the
record outright) and SavedView. DELETE /domains/<id> now tombstones and still
tears the vhost down; restoring a row does NOT re-do side effects unless the
type supplies on_restore, which is the honest default.

API: GET /recycle-bin (all kinds, newest first), POST
/recycle-bin/<kind>/<id>/restore, DELETE /recycle-bin/<kind>/<id> (purge,
admin-only -- the one irreversible action), POST /recycle-bin/purge-expired
with a 30-day default retention.

THE PART THAT NEEDED CARE: both tables had a plain UNIQUE covering what is now
a tombstone, so deleting a domain would have permanently burned its name and
deleting a view would have blocked re-creating one by the same name. Migration
083 replaces each with a PARTIAL unique index predicated on deleted_at IS NULL
-- supported by SQLite 3.8+ and Postgres. Tests pin exactly that: a deleted
name can be reused, and the live row stays unique.

Also gives SavedView a `slug` (backfilled), the handle the upcoming
?view=<slug> links resolve against. A tombstoned view does not reserve its
slug, so deleting a view frees the link name too.

Migration is idempotent on both columns and indexes: `deleted_at` is
index=True on the mixin, so a database built from the models already has that
index, and a bare create_index would abort the upgrade on exactly the installs
we most want to keep working.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 07:14:55 -04:00
Juan DenisandClaude Opus 5 3d9bf09bea fix(api): GET /servers never sent the metrics the servers list renders
The CPU/Memory/Disk gauges read `server.metrics`, the same key the
per-server status endpoint uses (servers.py:757). The list endpoint never
populated it -- to_dict() defaults to include_metrics=False, and even when
asked it emits `latest_metrics` -- so on a live panel every gauge cell fell
back to the "no data" dash.

It survived because the screenshot fixtures DO supply `metrics`
(sections.mjs:294): the UI looked correct in every capture and was empty in
production. The fixture had encoded the intended contract; the backend never
implemented it.

Batched deliberately: Server.metrics is lazy='dynamic', so the obvious
per-row lookup costs one extra SELECT per server. Newest row per server =
highest id (autoincrement on insert), which also avoids the tie a
max(timestamp) join hits for same-second rows -- two queries total,
regardless of server count.

Tests pin all three properties, and each fails without the fix: the key and
field names the table reads, absence (not {}) for a server that never
reported, and that the query count stays flat as servers are added.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 05:52:37 -04:00
Juan Denis d6bd0e6a52 fix(db): WAL + busy_timeout for SQLite — stop 'database is locked'
The queue-bus consumers, job system and metrics collector share one SQLite
file with request handling, and the default journal mode lets any writer
lock the whole file — the sporadic 'database is locked' errors in the
notification/job consumers. Connections now open with journal_mode=WAL
(readers and a writer coexist), busy_timeout=30s and synchronous=NORMAL.
Skipped under TESTING, where conftest owns connection pragmas.
2026-08-11 20:04:19 -04:00
Juan DenisandTyler Boykin 9fb0a6cab5 fix(security): close git argument injection in repo_url/app_path/branch (GHSA-8vx6-432p-h62q)
An authenticated user of any role could inject arguments into git
subprocesses running as the backend user (root under systemd):

- repo_url was passed unvalidated to git ls-remote/clone, enabling the
  ext:: transport (host command execution on git < 2.52 defaults) and
  file:// local-repository disclosure (all git versions).
- app_path on POST /deploy/clone was passed without a '--' terminator,
  enabling --upload-pack=<cmd> option injection independent of git
  version and protocol.*.allow policy.
- branch flowed into 'git fetch origin <branch>' as an unterminated
  refspec.

Fix (new app/utils/git_security.py applied at every affected sink):

- repo_url scheme allowlist (https/http/git/ssh + scp-like SSH);
  rejects ext::/file://, local paths, leading '-', whitespace.
- '-c protocol.ext.allow=never -c protocol.file.allow=never' pinned on
  every remote-touching git call, plus a restrictive GIT_ALLOW_PROTOCOL
  in the subprocess environment.
- '--' positional terminator before URL/path/refspec arguments —
  co-load-bearing with protocol pinning, which alone does not stop
  option injection such as --upload-pack=<cmd>.
- branch/ref validation (leading '-', ref-format metacharacters) for
  clone --branch, fetch refspecs and stored deploy configs.
- POST /deploy/clone now confines app_path to the managed apps root,
  mirroring how /apps/from-repository derives it server-side.

Adds 31 regression tests; bumps version to 1.7.86.

Reported-by: CaptBoykin
Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
2026-08-11 19:43:46 -04:00
Juan Denis ede0f0fdbd feat(activity): audit feed filters by target_id
AuditService.get_logs gains a target_id filter (paired with target_type) and
/admin/activity/feed passes both through — the per-resource activity
timeline can now ask 'what happened to THIS app'.
2026-08-09 13:53:59 -04:00
Juan Denis b6c257bfc7 feat(views): per-user saved table views API
SavedView model (user/page/name/state JSON/is_default, unique per
user+page+name), idempotent 082 migration, saved_view_service with
one-default-per-page enforcement, and a JWT-scoped /api/v1/views CRUD
blueprint. Covered by 9 endpoint tests.
2026-08-09 02:32:55 -04:00
Juan Denis 6e03c0a523 perf(templates): cache parsed local templates on file mtime
With the remote index memoized, re-parsing every bundled YAML on each call
was the entire remaining cost of a Templates page load. Entries are now
memoized per file and keyed on (st_mtime_ns, st_size), so a sync or an edit
invalidates exactly the file that changed: measured on the real catalog,
list_local_templates() goes from 304ms to 0.89ms (118 templates), and the
full catalog listing from 1707ms to 1.26ms.

The writers need no invalidation call - mtime keying covers create, overwrite
and delete, the last because the listing only returns what the directory scan
finds. Cache entries are pruned to what is still on disk so a long-running
panel that churns templates cannot grow it without bound.

Failed parses are cached too, keyed on the same stat tuple, so one malformed
YAML no longer costs a parse on every render; it is retried the moment the
file changes. A file that fails to parse still does not claim its id, so a
valid file of the same name in the fallback directory can supply it - pinned
by a test, since that is easy to lose in a refactor.

Entries are deep-copied out: a shallow copy would share the nested
categories/repo/engine containers, so a caller appending to categories would
corrupt every later listing.

Also swaps os.listdir for os.scandir (DirEntry.stat() reuses data the scan
already returned) and sorts directory entries, which makes a same-id
collision (foo.yaml vs foo.yml) resolve identically on every platform instead
of depending on filesystem order.
2026-08-06 09:32:40 -04:00
Juan Denis 9d64b4ab4d fix(templates): dedupe the catalog by id and cache remote indexes
Both bugs were latent only while DEFAULT_REPOS pointed at a 404. Now that
serverkit.ai proxies a live registry, they bite on every Templates page load.

list_all_templates() concatenated local + remote with no id merge, so every
bundled template was listed twice the moment a real registry answered.
Measured against the live registry: 224 entries for 116 unique ids. It now
merges by id with local winning, matching get_template(), which already
resolves local directories before any repo; among repos, the first enabled
one to claim an id keeps it.

fetch_remote_templates() ran inline on every listing with a 30s timeout and
no memoization, so one unreachable repo stalled the whole catalog on every
render. It is now memoized with a TTL (5m success, 60s failure), a 10s
timeout, and a last-good fallback so a transient blip cannot blank a catalog
the panel has already shown. The last-good payload is carried forward across
repeated failures rather than being overwritten by the first one. The cache
is dropped in save_config(), the single choke point for repo add/remove/
enable (sync_templates routes through it too), and force=True bypasses it.

11 proving tests, including the repeated-failure case and a regression test
that a cached entry cannot be corrupted by a caller mutating it.
2026-08-06 08:52:31 -04:00
Juan DenisandClaude Opus 5 32e3c8a839 test: record per-test durations so CI shards balance by time, not count
pytest-split falls back to splitting by test COUNT when no durations file
exists, and these tests range from 0.01s to 10s+. That produced visibly lopsided
shards on the first sharded CI run — 6m17s for the fastest against 10m46s for
the slowest, and the slowest shard is the whole job's wall clock:

    pytest (4/4)   6m 17s
    pytest (2/4)   8m 42s
    pytest (1/4)  10m 01s
    pytest (3/4)  10m 46s   <- Backend CI was 11m06s because of this one

With real durations, pytest-split packs the shards to equal TIME instead, which
should pull the slowest shard down toward the ~9m average.

Generated from a full local run: 3175 entries, 20 failed / 3108 passed /
47 skipped in 11m53s. All 20 failures are pre-existing and unrelated (a
serverkit-wordpress checkout absent locally, plus Linux-only fail2ban tests) —
verified by re-running those same files against the pre-plan-64 conftest.

Regenerating this file was only affordable because plan 64 Phase 1 landed
first; before it, a full run was slow enough that Phase 0 abandoned the attempt.
Regenerate it whenever suite timings shift materially:

    cd backend && python -m pytest tests --store-durations

Plan 64 Phase 3.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 03:25:21 -04:00
Juan DenisandClaude Opus 5 baff2452b0 test: guard the app fixture's scope so the suite can't quietly get slow again
Nothing catches a revert of `_flask_app` to scope='function'. The suite would
still pass — just ~3x slower — and CI would drift back to a 20-minute wait,
which is how it got that way in the first place. Same idea as the
collected-test-count ratchet: turn a silent regression into a red test.

Also asserts the inverse for `app`, which must stay function-scoped: it is what
resets the database between tests, and session-scoping it would leak each
test's rows into the next and fail while pointing at the wrong test.

Verified both directions — flipping _flask_app back to scope='function' fails
the guard with the explanation, and restoring it passes.

Ratchet 3173 -> 3175 for the two new tests.

Plan 64 Phase 4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 03:01:24 -04:00
Juan DenisandClaude Opus 5 9d570834aa perf(tests): build the Flask app once per session, not once per test
The `app` fixture called create_app('testing') for every one of 3173 tests,
registering 90+ blueprints each time. Measured on a fixed 199-test sample
(pytest --splits 16 --group 5 --durations=0):

    setup     144.9s   89.5%
    call        8.8s    5.4%
    teardown    8.2s    5.0%

95% of the suite's time was preparing to test; 5% was testing. The app is now
built once per process and shared.

Same 794-test slice, before and after — identical results, 2.66x faster:

    before   4 failed, 773 passed, 17 skipped   593.30s
    after    4 failed, 773 passed, 17 skipped   223.11s

(the 4 are pre-existing locally: serverkit-wordpress isn't checked out here.)

Sharing one app surfaces three things a per-test app hid, all handled in the
fixture rather than patched test by test:

1. create_app() is not a pure constructor — it SEEDS the bundled flagship
   extension rows as a boot side effect (test_cloudflare_extraction asserts
   they are "seeded on boot"). The per-test drop_all wipes them, and a shared
   app never re-boots, so every test after the first saw no flagship and the
   extension routes answered 503. seed_flagship_extensions() is now re-run
   after each create_all. The session fixture also boots twice on purpose:
   once to create the schema, once to seed into a schema that exists.

2. Config mutation now leaks where it used to die with the app (e.g.
   test_demo_deploys sets DEMO_DEPLOYS_ENABLED=False and never restores it),
   so config is snapshotted and restored around every test.

3. Structure mutation cannot be undone at all — Flask has no
   unregister_blueprint, and refuses setup methods once an app has served a
   request. Those tests opt out via `pytestmark = pytest.mark.fresh_app` and
   get a private app. Three groups needed it: 13 modules calling
   register_blueprint/add_url_rule directly, 11 that install plugins (
   plugin_service hot-loads their blueprints onto the live app), and
   test_audit_log which adds routes with @app.route.

Tests using the `wp_extension` fixture are detected from the fixture graph
instead of being marked, since that fixture mounts blueprints on whatever app
it is handed — miss one marker and the failure lands in an unrelated test much
later in the run.

Plan 64 Phase 1. Phase 2 (savepoint rollback instead of create_all/drop_all)
is NOT included: it currently takes the slice from 4 failures to 100 failures
and 201 errors, and needs its own round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 02:59:14 -04:00
Juan DenisandClaude Opus 5 4ccfa7e8cf test: ratchet 3138 -> 3173
10 from the template repo/sync suite added alongside the DEFAULT_REPOS fix;
the rest accumulated since the last ratchet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 00:39:21 -04:00
Juan DenisandClaude Opus 5 c3d70ed18b fix(templates): point the default repo at a host that exists, and verify what it sends
DEFAULT_REPOS pointed at raw.githubusercontent.com/serverkit/templates --
the org `serverkit` does not exist (the registry is jhd3197/serverkit-templates,
and its default branch is master, not main). So it has 404'd for its entire
existence: no panel has ever fetched a template remotely, and every /templates
page has been showing only the 118 bundled backend/templates/*.yaml.

Now points at https://serverkit.ai/templates, which proxies the registry and
was built for this consumer -- it serves <repo_url>/index.json and the
<repo_url>/templates/<id>.yaml path this class derives, behind a TTL cache
with last-good fallback. Using the product domain also means a branch rename
upstream cannot silently empty every panel's catalog, which is the exact
failure this is fixing.

Fixing the default alone would only help fresh installs, because get_config
returns whatever a saved templates.json holds. Known-dead URLs are therefore
healed on read. Nothing is lost by rewriting a URL that never once resolved.

The checksum half is here rather than in a follow-up because this commit
ACTIVATES a download path that has never run: sync_templates wrote whatever
came back straight to disk, ignoring the sha256 the index pins for every one
of the 106 official entries. Turning on an unverified fetch days after the
panel learned to verify ed25519 signatures on extensions would be shipping a
known downgrade. Mismatch is now a hard refusal that never reaches disk;
absent hash is allowed and counted, mirroring unsigned-vs-invalid for
extensions. Content is written as bytes so the file on disk is exactly what
was hashed.

Note get_template's in-memory remote read (it parses without saving) is not
covered -- it has no index entry on hand and would need an extra fetch.

DEPLOY ORDER: serverkit.ai must ship the master-branch fix before this helps.
Until then the proxy 502s and sync finds nothing, same as today.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 00:33:46 -04:00
Juan Denis e556baf238 test: ratchet 3117 -> 3138 (audit remediation proving tests) 2026-08-05 13:36:33 -04:00
Juan Denis 7c6b0f3958 security: hook seam lifecycle + registrant binding
Audit remediation (extension seams):
- disable/uninstall tear down the extension's backup kinds, event
  types, and template providers (unregister_capabilities by slug);
  enable re-registers (idempotent)
- wordpress_bridge fails closed unless the extension row is ACTIVE;
  orphaned-site db snapshot route returns clean 503
- provider registrations bound to their registrant: cross-slug
  replace/hijack rejected for backup kinds, event types, template
  providers
2026-08-05 13:36:33 -04:00
Juan Denis 670e9d8af7 security: signature lifecycle, update-path consent gates, download hardening
Audit remediation (install/signing surface):
- sig verdict reset on every install path (no stale verified badge)
- registry update route mirrors install consent gates (unreviewed 404,
  unsigned-first-party/untrusted 409 until acknowledged, bad sig 400);
  update offers carry installed_source_type; frontend risk dialog gains
  update branch
- first_party entries without verified pinned-key signature require
  acknowledge (community reviewed exempt, review-stamp-bound)
- downloads: https-only with per-hop redirect revalidation, private/
  link-local IP rejection, 100MB zip + 64KB sig caps, github token
  https-only
- extraction: declared+actual 250MB decompression cap with cleanup
- install/update audit logs record the signature verdict
2026-08-05 13:36:20 -04:00
Juan Denis d2d2d861cd test: ratchet 3213 -> 3117 (WP extension-owned suites moved out)
Collected 3142; floor keeps the usual ~25 headroom for
environment-dependent collection.
2026-08-05 02:38:54 -04:00
Juan Denis ba3bc685fe test: split WP suites — extension-owned move to serverkit-wordpress
10 extension-owned suites (surface, extraction, import-zip, php errors/
limits, pipelines alias, plugin library, promote, reports, ssh import)
move to the extension repo via the SERVERKIT_WORDPRESS_DIR loader.
Panel-core suites (bridge, hooks-absent, seeding, authz sweeps) updated
for WP-absent behavior; conftest seeds the manifest onto test plugin rows.
2026-08-05 02:38:41 -04:00
Juan Denis b3eca19f66 chore(registry): bundled index lists serverkit-wordpress
Offline-first installs see the WP entry; sha256 refreshes from the
published asset at panel release time.
2026-08-05 02:38:13 -04:00
Juan Denis 798467baba feat(sdk): bump to 1.2.0 + PluginSlot/ErrorBoundary exports
Runtime-ESM extensions (starting with serverkit-wordpress) need slot
mounting and error containment from the host; additive, versioned in
lockstep frontend/backend.
2026-08-05 02:38:13 -04:00
Juan Denis 72eaba34f2 refactor(wordpress): decouple core from the in-tree extension
Bridge, blueprint registration, app/attach flows and builtin job
handlers no longer assume the WP builtin is present; WP install paths
gitignored like other extracted extensions. WP absent = feature absent
gracefully (plan 52 phase 5, D4).
2026-08-05 02:37:48 -04:00
Juan Denis 0f8d179692 test: ratchet 3197 -> 3213 (wp hook inversion matrix) 2026-08-05 01:18:42 -04:00
Juan Denis d61985e73b test: WP hook inversion matrix (present x absent)
16 tests covering boot registration, idempotency, backup provider
resolve/validate/missing-state, event catalog present/absent, template
provider dispatch, monitor binding gate, health-sweep no-op, analytics
ext-to-ext guards. Existing suites minimally updated for the provider
interfaces (171 WP/monitor + 316 backup/events/templates green).
2026-08-05 01:18:32 -04:00
Juan Denis 060e600ffa feat(monitors): gate wordpress_site_id binding on WP presence
Create/update validation fails loudly naming the missing extension
instead of orphaning monitors; health sweep no-ops with zero WP sites
(plan 52 phase 3 task 17). StatusComponent.wordpress_site_id stays core
per D1.
2026-08-05 01:18:23 -04:00
Juan Denis e0888ecbba feat(templates): provider hook for extension template kinds
The wordpress template kind + wordpress-external-db preflight move behind
a template-provider hook the extension supplies; the backend filters the
catalog so the WP card simply does not appear when WP is absent — no
frontend change needed (plan 52 phase 3 task 16).
2026-08-05 01:18:23 -04:00
Juan Denis b153f68c12 feat(events): registered event-type catalog seam
Extensions register event types at load; wordpress.* types now come from
the WP extension instead of a hardcoded core catalog (plan 52 phase 3
task 15). emit_wp stays core as a dumb emitter helper.
2026-08-05 01:18:22 -04:00
Juan Denis 8d106aa5a0 feat(backups): dynamic target-provider registry
VALID_TARGET_TYPES is now provider-driven; the wordpress_site branch
moves behind a provider the WP extension registers. ProtectionPanel /
RestoreDrawer read available types from the API; policies whose provider
is absent surface a clear missing-provider state instead of crashing
(plan 52 phase 3 task 13).
2026-08-05 01:18:11 -04:00
Juan Denis 57d82c8c83 feat(extensions): boot-time hook registration seam
Extensions can register providers into core engines at load (backup
targets, event types, template kinds) via a core_hooks module; idempotent,
process-lifetime like blueprint registration. WP extension supplies its
entries through builtin-extensions/serverkit-wordpress/backend/core_hooks.py
(plan 52 phase 3, D4).
2026-08-05 01:18:02 -04:00
Juan Denis 73de373b80 test: move minecraft suites to the extension repo, adjust ratchet
The four minecraft test files now live in serverkit-minecraft/tests/
(panel-hosted via SERVERKIT_MINECRAFT_DIR loader). Ratchet floor
3280 -> 3197 (-83 minecraft tests; collected 3222, floor intact).
2026-08-05 00:09:19 -04:00
Juan Denis 31697cda9e feat(extensions): enforce signature policy in install flow
Invalid signature hard-blocks (400); untrusted key requires explicit
acknowledge (409 backstop); registry index carries signature/publisher
fields; install pinned to previewed sha256 with re-verified signature.
2026-08-04 23:34:15 -04:00
Juan Denis af64f3628d feat(extensions): ed25519 signing service + publisher key registry
Signature verification over extension bundles with pinned publisher
public keys; verdicts: verified/unsigned/unknown_signer/invalid.
24 tests: round-trip, tamper, malformed, untrusted key, forge attempts.
2026-08-04 23:34:02 -04:00
Juan Denis 00c66556b0 Bump cryptography to 50.0.0
Update the backend dependency pin from `cryptography==48.0.1` to `cryptography==50.0.0` in `requirements.txt` to keep encryption tooling current.
2026-08-04 09:10:56 -04:00
Juan Denis d961bd23f0 fix(api): close IDOR/authz gaps across deployment jobs, vaults, queues, docker, databases
Proactive hardening from an internal audit of the same broken-authorization
class as GHSA-6w78-q5vm-rfmh (authentication checked, app/workspace-level
authorization skipped):

- deployment_jobs: job detail/logs/retry resolve the job and gate on
  can_access_app / can_operate_app (app-less jobs: requester or admin);
  global job listing is admin-only
- secrets_webhooks: every vault/secret/webhook-endpoint/delivery by-id
  route resolves to its workspace and 404-seals non-members; secret
  reveal and signing-secret regeneration require workspace owner/admin;
  dead local admin_required placeholder removed
- shared_resources: variable-group CRUD and attach/detach gate on the
  resolved scope (workspace role / can_edit_app / can_operate_app),
  closing deploy-time env injection into foreign apps
- queue_bus: message receive/complete/fail/list on system-owned groups
  are admin-only; user-owned groups restricted to their owner
- docker: raw container inspect (exposes env vars) and container logs
  are admin-only, matching Decision 7; per-app logs stay app-gated
- databases: raw SQL query endpoints, host-level metadata reads, and
  process lists are admin-only; per-app managed database flows unchanged

Adds 4 persona-matrix regression suites (74 tests) and bumps the
test-count ratchet.
2026-08-04 08:53:27 -04:00
Juan DenisandTyler Boykin 783e195304 fix(git): enforce app-level authorization on deployment read endpoints
GHSA-6w78-q5vm-rfmh: four git deployment/webhook read endpoints checked
authentication but skipped app-level authorization, letting any active
user (even a foreign viewer) read cross-tenant deployment history and
logs that can contain secrets from pre/post-deploy scripts.

- git.deployments/<id>: resolve deployment -> app, 404/403 via
  app_access_tier before returning any log bytes
- git.deployments/webhook/<id>: gate on _webhook_visible_to (app-linked
  webhooks follow app visibility; unlinked stay panel-wide, plan 29 #11)
- deploy.history & deploy.webhook-logs: app_id queries require
  app_access_tier; global (no app_id) queries are panel-admin only

Adds backend/tests/test_git_deployment_authz.py (9 persona-matrix
regression tests) and bumps the test-count ratchet.

Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
2026-08-03 23:44:56 -04:00
Juan Denis 048a560f8c fix(security): correct PROTECTED_ROOTS path math; tighten root matching
Copilot review on the GHSA-rm3m-9mvw-68fh patch caught two issues:

- _BACKEND_DIR/_INSTALL_DIR were one level too shallow (backend/app and
  backend/ instead of backend/ and the install root), which left
  <install>/.env and frontend/dist reachable on the deployed layout —
  the exact PoC path from the advisory. _BACKEND_DIR stays in
  PROTECTED_ROOTS so flat layouts (code at /app) are covered too.
- The ALLOWED_ROOTS check used a bare startswith, so e.g. /optfoo
  matched the /opt root. Both checks now require an exact match or a
  path-boundary prefix.

New tests pin the level math (the existing cases computed the expected
layout independently and passed on Windows for the wrong reason) and
simulate the deployed layout with the install dir inside an allowed root.
2026-08-02 18:56:33 -04:00
Juan Denis 0b0bfcce70 feat(security): daily GHSA feed check with admin notifications
New builtin job security-feed (daily) pulls the normalized advisory feed
from serverkit.ai/api/security-feed.json (a cached proxy of the repo's
published GitHub Security Advisories) and:

- notifies admins (security.alert, critical) when the running panel
  version falls inside an advisory's affected range, once per advisory;
- fires a one-time post-fix reminder after an upgrade crosses a fix
  boundary when the advisory declares a "## Post-upgrade actions" section
  (e.g. rotate JWT_SECRET_KEY after GHSA-rm3m-9mvw-68fh).

Dedupe keys, the last-good feed copy, and the previously-seen version live
in SystemSettings, so restarts never re-notify and outbound failures serve
the cached feed. Opt out via the security_feed.enabled setting. Privacy
page on serverkit.ai updated to disclose the daily call.
2026-08-02 18:35:25 -04:00
Juan DenisandCaptBoykin 5bc3f26d77 fix(security): block panel-internal paths from the file manager (GHSA-rm3m-9mvw-68fh)
The 1.7.68 RBAC fix enforced files.read on read endpoints, but the default
viewer role legitimately has files.read=true, and on the documented
install.sh layout the panel install dir (/opt/serverkit) sits under the
allowed root /opt. A viewer could read the backend .env, steal
JWT_SECRET_KEY / SERVERKIT_ENCRYPTION_KEY, and forge admin sessions.

FileService now carries PROTECTED_ROOTS (the panel install dir and
SERVERKIT_CONFIG_DIR), checked inside is_path_allowed before the
ALLOWED_ROOTS test, so every file-manager operation (read, write, browse,
download, upload, ...) refuses panel-internal paths for every role.

Regression tests: protected paths rejected by is_path_allowed; viewer
read/download of the backend .env returns 403; allowed roots keep working.

Co-authored-by: CaptBoykin <CaptBoykin@users.noreply.github.com>
2026-08-02 17:21:02 -04:00
Juan Denis 937cbb2b81 feat(setup): capacity-aware install profiles and resource advisory
- install_profile_service: record what install.sh put on the box
  (minimal/standard/full), recommended from detected hardware and
  overridable; thresholds mirrored by recommend_profile in install.sh
- setup wizard: replace the hard ResourceGate/SetupStepTier with
  SetupStepCapacity + SetupStepSecurity
- ResourceAdvisory: inline capacity warning at the point of action
  (WordPress/Docker/Services/Deployments), backed by headroom.fits
- RequiresDocker component for docker-dependent surfaces
- install.sh/update.sh: profile detection + tests
  (test_install.sh, test_update.sh, test_server_capacity.py)
2026-07-31 14:56:37 -04:00
Juan DenisandClaude Opus 5 20f1659b51 test(monitoring): pin that monitors stay core
Walks the AST of monitor_service and the monitors blueprint to assert neither
imports the plugin loader nor reaches for StatusPageService, and that the routes
register without the extension. This is the property the promotion exists to
provide — watching a site must keep working on a lean panel with
serverkit-status uninstalled — so it should fail loudly if it regresses.

AST rather than a source grep: the modules' own comments discuss the extension,
which a text match would trip over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 00:28:19 -04:00
Juan DenisandClaude Opus 5 cceb444d27 fix(monitoring): give the TLS certificate read its own clock
_maybe_attach_certificate throttled on last_check_at, which is the probe clock.
An active monitor is checked every 30s, so that condition was never older than
the 6h refresh window: the certificate would be read once and then never
refreshed, leaving a stale expiry date on the detail page forever.

Adds cert_checked_at (migration 081) and throttles on that instead. The stamp
is written even when the read fails, so an unreachable TLS endpoint is retried
on the same cadence rather than on every single probe.

Also serves the Monitors list sparkline from one bulk query per page instead of
one per row, bounded by a time window so a busy monitor's history cannot crowd
out a quiet one's the way a single global LIMIT would.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 00:22:56 -04:00
Juan DenisandClaude Opus 5 9512975ebc feat(monitoring): promote status components to first-class monitors
A StatusComponent was already a full synthetic check (http/tcp/dns/smtp/ping,
target, interval, timeout, HealthCheck history, uptime windows, auto-incidents)
— it was just framed as "a component of a status page" and its engine lived
inside the serverkit-status extension, so watching a site required publishing a
status page. Promote it:

- migration 081 makes status_components.page_id and status_incidents.page_id
  nullable, and adds the columns backing the monitor UI (is_paused,
  check_method, expected_status, keyword, follow_redirects, verify_tls, retries,
  consecutive_failures, cert_issuer, cert_expires_at)
- new core app/services/monitor_service.py owns the check engine; the extension
  keeps page publishing and delegates
- new /api/v1/monitors (CRUD, check, pause, history, uptime buckets, incidents)
- new builtin.monitor_check sweep on a 30s tick — until now checks only ran when
  someone pressed "Check now", so a configured monitor never noticed an outage

Check-engine defects fixed while moving it:

- ping ignored the process return code and could only ever report 'up'
- ping used Linux-only flags with no guard, breaking the Windows dev server
- http ignored its own method/expected_status/follow_redirects/verify_tls
- smtp was documented on the column but unimplemented; keyword did not exist
- run_check never recomputed uptime and never opened or resolved an incident —
  only the WordPress health-sync path did, so network monitors never paged
- TLS issuer/expiry are now captured (throttled to one read per 6h)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:29:02 -04:00
Juan DenisandClaude Opus 5 b83cb5ca82 feat(cron): rebuild /cron on the Domains list pattern
The page kept a shape the rest of the app moved off: a four-card KPI
strip, its own in-body search input, a hand-rolled table, four icon
buttons per row, and an edit modal on row click.

It now matches /domains: the segmented filter carries the counts (no
KPI strip), SearchField lives in the top bar, the list is the shared
DataTable, and a row opens the DS Drawer. The top bar keeps its title
— /cron has no tab group, and the reference mock shows a titled bar
there; the body was what was stale.

Last run / Next run / Status columns need data list_jobs() never
returned: it skipped the schedule_human / next_run / run-tracking joins
that jobs_for_application() has done since plan 34, so the admin list
could only ever show enabled-vs-disabled. Apply the same enrichment,
with tests covering the schedule join, the last-run join, and the
no-history case.

The drawer's run history uses GET /cron/jobs/<id>/runs, which already
existed and only ScheduledTasksCard consumed. It's admin-only and
empty until a job carries the tracking shim, so both cases are stated
in the drawer (with an enable-tracking action) rather than rendering a
blank table.

Also fixes styles that never applied: .run-output-*, .spinner-inline
and the modal field overrides were nested under .cron-page, but Radix
Dialog and Sheet portal to <body>. Hoisted to top level.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 22:44:40 -04:00
Juan Denis e3c4fa778d fix(files): address PR review — literal grep match, tail clamp, extend RBAC coverage
- log_service: grep the service/unit filter with -F so names like
  'nginx.service' are matched literally instead of as a regex.
- log_service: clamp the in-Python tail like 'tail -n' — lines=0 sliced
  as [-0:] == all lines, negatives kept nearly everything.
- test_files_rbac: add files/upload and files/s3/upload to WRITE_CASES,
  and disk-usage/disk-mounts/analyze/type-breakdown to READ_CASES, so the
  permission gates on the full /api/v1/files surface are covered.
2026-07-26 16:25:12 -04:00
Juan Denis 88295ab000 fix(update): harden migration step against boot side-effects and slot venv mismatch
- create_app(): skip background daemons (queue consumers, metrics, job
  system, analytics flush, linked-panel client) when loaded by a Flask CLI
  one-shot such as 'flask db upgrade', or when SERVERKIT_SKIP_BACKGROUND=1.
  They query the DB before migrations run, so any error there (corrupt DB,
  pre-migration schema) aborted the migration and sank the whole update.
- update.sh: set SERVERKIT_SKIP_BACKGROUND=1 for both flask db upgrade
  invocations as an explicit contract.
- update.sh: fail fast with an actionable message when the SQLite slot copy
  fails PRAGMA integrity_check, instead of dying mid-boot with 'database
  disk image is malformed'.
- update.sh: rebuild the release tarball's prebuilt venv when its baked
  VIRTUAL_ENV path resolves to the old slot (it is baked as
  /opt/serverkit/venv), which previously ran new code on the old slot's
  dependencies during migration. Resolved-path comparison keeps the fast
  prebuilt path for fresh installs.
2026-07-26 16:18:37 -04:00
Juan Denis ddf2e948d1 fix(security): enforce files.read on reads; de-shell syslog unit filter
Follow-ups from the GHSA-4wqh-7f4f-5qmx / GHSA-mc93-rc3x-fpgq review:

- File Manager read endpoints (browse/info/read/search/disk-*/analyze/
  type-breakdown/download + s3/browse, s3/read, s3/download-url) only
  required a valid JWT, so a user whose files.read permission is revoked
  via custom permissions could still read and download files. They now
  require permission_required('files', 'read'), matching the write side.

- LogService._read_syslog() interpolated the unit name into a bash -c
  string using subprocess.list2cmdline(), which is cmd.exe-style quoting
  and does not stop $(...)/backtick expansion under bash. grep now runs
  as a plain argv list (with --) and the tail happens in Python.
  Reachable only on non-systemd hosts by admins, but the same bug class
  as the Postfix advisory.

Regression tests: revoked files.read 403s every read endpoint; the
syslog filter never invokes a shell and passes payloads verbatim.
2026-07-26 15:40:14 -04:00
Juan DenisandAruvasgaChithan 696226002b fix(security): enforce files.write permission on File Manager mutations (GHSA-4wqh-7f4f-5qmx)
Mutating File Manager endpoints (write/create/mkdir/delete/rename/copy/
move/chmod/upload and the S3 equivalents) only required a valid JWT, so a
viewer account (files.write=false) could write and delete files. They now
require the files.write permission via permission_required(). Read
endpoints are unchanged. Adds regression tests covering both advisories.

Co-authored-by: AruvasgaChithan <AruvasgaChithan@users.noreply.github.com>
2026-07-26 15:19:22 -04:00
Juan Denisandtonghuaroot 20e79ff363 fix(security): prevent command injection via hostname in Postfix install (GHSA-mc93-rc3x-fpgq)
The hostname parameter was concatenated into a bash -c string executed as
root via run_privileged(). Validate the hostname against a strict charset
and pipe debconf selections via stdin so no user input ever reaches a
shell command line.

Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
2026-07-26 15:19:08 -04:00
Juan DenisandClaude Opus 5 640e7e827e docs(screenshots): recapture, and fix two bugs the capture exposed
Six new sections cover the pages rebuilt this round — the dashboard as it ships,
its edit mode, the widget library, the widget editor, the databases explorer and
the engine catalog drawer. 31/31 sections render.

The capture is a render smoke test, not just a picture, and it earned that here
by surfacing two real bugs that no test could see:

`SearchField` merges its `className` onto its OWN root, so `.skw-lib__search`
IS the `.sk-searchfield` element rather than a wrapper around one. The rule
styling it as a parent matched nothing, leaving the shared `flex: 1 1 240px`
in force — and inside a column-direction filter bar that 240px basis is a
HEIGHT. The widget library's search box was rendering 240px tall.

`GET /auth/me` answers `{user: {...}}`, but Databases read `.role` straight off
the envelope, so `isAdmin` was always false. Every admin got the read-only
console — and with the DDL builder now shipped, would never have seen the button
that runs a CREATE TABLE.

Also carries `postgresql.yaml`, which I left out of the extensions commit.
Without `engine.extensions: true` stock PostgreSQL is not a host for pgvector at
all; the extension tests passed only because the working tree had the change the
commit did not.

Fixture note for whoever edits these next: `/metrics/history` points are NESTED
(`cpu.percent`, `load['1m']`) and carry no network data, `/servers/available` is
a bare array, and `DeployPill` queries `/deployment-jobs` twice and sums — a
single unfiltered payload makes it claim eight running deploys.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-26 09:59:06 -04:00