73 Commits
Author SHA1 Message Date
Juan DenisandClaude Fable 5 e6e605d1f4 refactor(remote): close the dispatch_agent_command seam
Migrate all 20 direct agent_registry.send_command call sites (api/servers,
terminal, fleet doctor/repair/fleet incl. the upgrade Thread target,
deployment_runner, survey, tunnel broker/publish, plugins SDK) onto the
shared dispatcher, and widen the ratchet from remote_*_service.py to a
tree-wide AST scan covering builtin-extensions/. The survey test's
transport stub gets the transport's real keyword signature.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 23:31:46 -04:00
Juan DenisandClaude Opus 5 fe02ce8eeb fix(auth): resolve the caller through one identity door, not the raw JWT
Plan 76 milestone A, drift/bug row 2 — scheduled as a fix, not a refactor,
because it is a behaviour difference shipping today.

rbac.get_current_user() is the only lookup that knows about both auth
mechanisms: it returns g.api_key_user for an X-API-Key request and falls back
to the JWT. 73 call sites answered "who is calling" with
`User.query.get(get_jwt_identity())` instead, behind 19 near-identical private
helpers (12 named get_current_user(), shadowing the real door).

The audit that scheduled this row predicted those return None for API-key
callers. Measured, it is worse: auth_required() deliberately skips
verify_jwt_in_request() once the key middleware has authenticated, so there is
no JWT context and get_jwt_identity() *raises*. Nine handlers reached through
an API-key-capable policy decorator therefore returned 500 to every API-key
caller:

  bandwidth.get_app_bandwidth          @viewer_required
  servers.set_server_workspace         @developer_required
  servers.create_server                @developer_required
  sso.get_sso_config / update_provider_config / update_general_settings
  source_connections.update_{github,gitlab,bitbucket}_admin_config
  source_connections.complete_github_app_manifest
  databases.execute_{mysql,pg,sqlite,docker}_query   (see below)

Three service helpers had the same lookup inside a blanket `except Exception`,
which converted the raise into a silent user_id=None — the write landed and
the audit trail lost the actor (shared_resource_service, cf_ops_change_service,
secrets_webhooks).

The four databases.execute_*_query sites were a second admin check *inside* an
already-@admin_required route: dead code that always passed for JWT callers and
crashed for API-key ones. They now call require_admin_user() rather than
re-deriving the answer.

Also folds in a sixth _require_admin() found in the cloudflare-ops extension
source (29 gates, the same inverted contract as jobs.py), and fixes
serverkit-gui's blueprint at its sibling-repo source rather than in this repo's
live copy.

Ratchet (second-wave rule 1 — a migration without one regrows):
tests/identity_door_census.py counts both bypass shapes by AST, direct and the
two-line `uid = get_jwt_identity(); User.query.get(uid)` variant. The ceiling is
87, all of them on @jwt_required()-only routes where reading the JWT is correct
today; the ratchet's third assertion holds the API-key-capable population at
exactly zero, so migrating one of those routes to a policy decorator without
fixing its lookup now fails. Verified non-vacuous: an injected bypass fails both
assertions and passes again on revert.

The first wave's controller-boundary baseline is regenerated — this removes 74
persistence crossings (593 -> 519), a debt reduction with no new entries.

Gate: tests/test_api_key_identity.py (7, each red before the fix),
test_identity_door_ratchet.py (10), test_api_controller_boundaries.py (4), plus
the 65 test files covering every touched module: 1024 passed, 10 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 19:33:57 -04:00
Juan DenisandClaude Opus 5 28d26a73a2 refactor: finish the §G1 subprocess migration (census 195 -> 28)
Plan 75 §G1, the remaining 33 files. Every raw subprocess call site in
backend/app that can go through a door now does. What is left is what
should be:

  8  app/utils/system.py        the doors themselves
  8  database_service           mysqldump|gzip, gunzip|mysql, pg_dump|gzip,
                                gunzip|psql — a pipe is two processes
  7  build/docker/db_sync/ftp   streaming output, gzip pipes
  1  log_service                tail -f
  2  serverkit-analytics        sibling repo; see below

Extensions were migrated in BOTH trees. builtin-extensions/ is the source
and backend/app/plugins/ the live copy, compared by normalised hash;
test_builtin_extension_drift stays green. serverkit-email has both sides
tracked, serverkit-ftp and serverkit-cloud-provision only the source.

serverkit-analytics (2 calls) is deliberately untouched: it has no source
in builtin-extensions/ at all, and backend/app/plugins/serverkit-analytics
is a gitignored materialised copy. Editing it would be a change that
cannot be committed and is overwritten on the next sync — that fix belongs
in the serverkit-analytics repo.

Exit-code semantics are preserved wherever they carry meaning, with a
comment at each site, because they are the thing a mechanical migration
destroys: clamscan 1 = INFECTED, grype 1 = vulnerabilities FOUND, yara
1 = matches, cron_runner 127 = never launched. Those read
result['returncode'], never result['success'].

run_checked gained two things real call sites needed: it yields when the
caller directs the streams itself (writing straight to a log file), rather
than adding capture_output on top and raising ValueError; and text/errors
pass through, so site_importers/ssh keeps decoding with errors='replace'
instead of gaining a UnicodeDecodeError on a bad byte.

A regression this migration introduced, caught by the suite and fixed:
install_profile_service._docker_usable() is deliberately TRI-STATE —
None means the probe could not run, because inside LXC the docker client
is usually present while the daemon never started, and callers "must not
treat a broken probe as proof of absence". Writing it as
run_checked(...)['success'] turned an exec failure into False, i.e. "docker
is not usable" — a fact nobody established. That is the §A violation this
plan exists to remove, reintroduced by the migration meant to prevent it.
test_probe_honesty_doctor caught it. Every other run_checked(...)['success']
caller was then audited; the rest are genuinely two-state, and their
originals also returned False on a missing binary.

Seven more slips of the same shape were caught before running by an AST
check for `x = run_checked(...)` followed by `x.stdout/.stderr/.returncode`
— a dict read as an object, which raises only on the failure path, so
happy-path tests stay green.

api_controller_boundary_baseline.json: pure deletion. Routing api/docker.py
through DockerService removed the last 6 subprocess crossings, so the
`subprocess` category is gone from the API layer entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:38:25 -04:00
Juan Denis aa3407750c refactor(extensions): own remote access styles 2026-08-18 03:14:15 -04:00
Juan DenisandClaude Opus 5 7e551da45c fix(email): a dovecot probe that cannot run reports unknown, not stopped
ServiceControl.is_active raises FileNotFoundError when systemctl is absent --
deliberately, so a host without systemd reads as "could not check" rather than
"not running". DovecotService.get_status called it unguarded, so
GET /api/v1/email/status returned a 500 on every such host. Email.jsx catches
that and renders "Mail server not installed" with an Install button, on a box
already running Postfix and Dovecot.

Guarding only the is_active line -- the literal review comment -- leaves the
second escape route open one statement later: both wrappers carry
timeout=PROBE_TIMEOUT, so a systemd that stops answering raises TimeoutExpired
straight through is_enabled, which swallows only the missing binary. Hence
(subprocess.SubprocessError, FileNotFoundError), covering both probes -- the
same pair dkim, spamassassin, ftp and postfix already catch.

running/enabled report None on an unreadable probe. Fabricating False here is
exactly the dishonesty this round removed from the firewall and SSL services.
email_service keeps its top-level `running` boolean with bool(), since
`True and None` is None and that field is documented as a boolean; the honest
tri-state stays in the per-component dicts.

Mutation-checked by line number: the narrow `except FileNotFoundError` fails 2
of the 4 new tests, a guard catching neither fails 3, and the healthy path
passes throughout.

Reported by Copilot on the plan-75 PR (is_active only).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 01:56:59 -04:00
Juan DenisandClaude Opus 5 b51d9c2018 fix(services): close the rest of the bare-name sbin exec class
The ufw fix repaired every run_privileged caller centrally, but three sites
reached subprocess by other routes and were broken the same way. All are
"$PATH omits sbin + the panel runs as root, so nothing prepends sudo to
resolve it".

- run_command() got the same argv[0] resolution as privileged_cmd. It does no
  privilege escalation, so nothing else would ever find these commands.
- nginx_advanced_service.test_config() ran `nginx -t` (/usr/sbin/nginx) via
  run_command, so it raised FileNotFoundError and reported every config
  invalid. It also returned valid=True unconditionally, ignoring the exit code
  entirely — so when it DID run, a broken config passed. Now run_privileged and
  the returncode is checked. reload_nginx() likewise reports a failed reload
  instead of swallowing it.
- dovecot_service.get_status() detected dovecot with `which`, which searches
  $PATH only and false-negatives on /usr/sbin/dovecot, then read the version by
  bare name under a shared except. Now is_command_available() + a separate try,
  so a version read that fails cannot unsay "installed". Patched in both the
  builtin-extensions source and the app/plugins live copy.
- SecurityService.get_lynis_status() inferred presence solely from exec'ing
  `lynis --version`; presence and version are now separate questions.

Adds test_sbin_command_guard.py so this cannot come back: an AST scan fails CI
on any raw subprocess call using a known sbin-resident command by bare name
(it caught the dovecot site), plus per-command proof that both helpers resolve
under the exact PATH the panel unit ships.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 09:37:59 -04:00
Juan Denis e6008afb38 refactor(wordpress): remove in-tree extension (extracted)
The full WordPress suite now lives at github.com/jhd3197/serverkit-wordpress
as a standalone runtime-ESM extension (backend blueprints/services, game
dashboard UI, tests, CI). Installed from the registry or offered during
onboarding from the bundled snapshot. WP data models stay core per D1
(documented core-data-seam).
2026-08-05 02:38:24 -04:00
Juan Denis 57d82c8c83 feat(extensions): boot-time hook registration seam
Extensions can register providers into core engines at load (backup
targets, event types, template kinds) via a core_hooks module; idempotent,
process-lifetime like blueprint registration. WP extension supplies its
entries through builtin-extensions/serverkit-wordpress/backend/core_hooks.py
(plan 52 phase 3, D4).
2026-08-05 01:18:02 -04:00
Juan Denis 5d8160dc73 refactor(wordpress): relocate frontend into the extension
5 WordPress* pages + components/wordpress (40 files) + 3 _wordpress*.scss
partials move from core into builtin-extensions/serverkit-wordpress/
frontend/ (synced to frontend/src/plugins via the baked-glob pattern).
200 imports rewritten to serverkit-sdk; remaining @/ imports only for
genuinely host-internal modules. main.scss drops the WP partials — the
extension bundle imports its own styles. npm build + lint green.
2026-08-05 00:48:12 -04:00
Juan Denis e280a446c1 refactor(extensions): extract serverkit-minecraft to its own repo
The Minecraft game-server extension now lives at
github.com/jhd3197/serverkit-minecraft as a standalone runtime-ESM
extension (analytics/gpu repo pattern). It is installed from the
registry like any first-party extension, no longer bundled in-tree.
2026-08-05 00:09:11 -04:00
Juan DenisandClaude Opus 5 9512975ebc feat(monitoring): promote status components to first-class monitors
A StatusComponent was already a full synthetic check (http/tcp/dns/smtp/ping,
target, interval, timeout, HealthCheck history, uptime windows, auto-incidents)
— it was just framed as "a component of a status page" and its engine lived
inside the serverkit-status extension, so watching a site required publishing a
status page. Promote it:

- migration 081 makes status_components.page_id and status_incidents.page_id
  nullable, and adds the columns backing the monitor UI (is_paused,
  check_method, expected_status, keyword, follow_redirects, verify_tls, retries,
  consecutive_failures, cert_issuer, cert_expires_at)
- new core app/services/monitor_service.py owns the check engine; the extension
  keeps page publishing and delegates
- new /api/v1/monitors (CRUD, check, pause, history, uptime buckets, incidents)
- new builtin.monitor_check sweep on a 30s tick — until now checks only ran when
  someone pressed "Check now", so a configured monitor never noticed an outage

Check-engine defects fixed while moving it:

- ping ignored the process return code and could only ever report 'up'
- ping used Linux-only flags with no guard, breaking the Windows dev server
- http ignored its own method/expected_status/follow_redirects/verify_tls
- smtp was documented on the column but unimplemented; keyword did not exist
- run_check never recomputed uptime and never opened or resolved an incident —
  only the WordPress health-sync path did, so network monitors never paged
- TLS issuer/expiry are now captured (throttled to one read per 6h)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:29:02 -04:00
Juan DenisandClaude Opus 5 3c84d152ea feat(wordpress): run environment promotions as watchable deployments
Promoting between WordPress environments ran in the request thread: no
console, no log stream, no retry, and a progress bar the pipeline page had
to keep alive itself. It now runs as a registered deployment kind, so it
inherits the Deploy Console, live logs, retry and a row in Deploy Activity.
POST /promote returns 202 with a job_id instead of blocking, and the UI
opens the console with it.

PromotionJob is kept, not replaced. It carries pre_promotion_snapshot_id,
the rollback pointer, which is WordPress domain a generic DeploymentJob has
no business holding — so the deployment records which promotion rows it
produced instead. They are collected by id watermark rather than from the
service's return value, because promote_* only reports the job on success
and a failed promotion is exactly when someone needs the snapshot.

A full promotion runs two five-step halves and restarts its own numbering
at 1 for the second, so the progress bridge carries an offset across the
reset — otherwise the console's step strip walks backwards halfway through.

Two gaps this exposed in the SDK, both fixed here:

- Plugin kinds never closed their log stream, so they got no per-step
  timings, no failure tail and no hint, and no terminal status push. The
  handler and the runner also built separate streams, so the timings one
  recorded were dropped by the other's close; they now share one per job.
- A handler that reported failure by return value logged nothing, leaving
  a failed run with no reason visible in the console.

The console now titles a run from plan.title, so a contributed kind names
itself without a branch per kind in the frontend.

A locked environment is refused by the endpoint rather than by the job:
queueing first would send the user to a console that fails a second later.
The pipeline service still re-checks, since a lock can be taken in between.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 13:29:12 -04:00
Juan DenisandClaude Opus 4.8 cf9cc5d662 fix(games): drop separate indexes on minecraft ext models
The index=True columns emitted separate CREATE INDEX statements that, once the
ext_serverkit_minecraft_* models register on the shared test metadata mid-session
(when the install test runs), collided during a later test's create_app schema
sync — aborting create_all before system_settings and cascading into errors
across the suite. A scaffold needs no secondary indexes; drop them (they can be
reintroduced with proper migration handling when the Docker create-flow lands).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:48:47 -04:00
Juan DenisandClaude Opus 4.8 fd6f1733c3 feat(games): serverkit-minecraft extension backbone + gamekit framework
First vertical of the game-server family (plan 53 Phase 0-1, backend core):

- gamekit/ — the reusable, dependency-free framework (D1): a Source RCON client
  (framing/auth/timeouts), a container-log event parser (join/leave/started/
  stopped/crashed, Java + Bedrock), a server.properties config-form builder over
  a JSON sidecar, and the save-aware hot-backup sequence (save-off -> flush ->
  zip -> save-on, always resumes) with retention.
- server_properties_meta.json sidecar (Phase 0 authoring task): grouped, labelled,
  restart-required metadata for the settings form.
- Extension scaffold: manifest (category 'games', models:register), ext_serverkit_
  minecraft_* models keyed by application (D9), and the /api/v1/minecraft blueprint
  (list/get, RCON proxy, backups).

Unit-tested (fake RCON socket, fixture logs, temp worlds) + install-wiring-tested
(install registers routes + creates ext tables + uninstall). The create/console/
UI/Docker flow is the dev-box phase and is intentionally not claimed here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:14:35 -04:00
Juan DenisandClaude Opus 4.8 59a64e4337 serverkit-localkit: advertise sync kinds + site kind (plan 26)
The LocalKit contract for per-kind sync (plan 26): /pair now returns a `kinds`
array (site kinds this extension can sync) and every /sites entry carries a
`kind`. Both keep old-server ↔ new-client safe — a LocalKit client defaults an
absent kind to "wordpress" and gates its php sync/import UI on the `kinds`
advertisement.

KINDS is ['wordpress'] today: serverkit-wordpress is this extension's only site
backend, so advertising php would be untrue. Add 'php' here in lockstep with a
php-stack backend that _resolve_site / _install_code / _import_db can dispatch
to; the client already speaks the php protocol (engine-native db, app/ code).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:57:33 -04:00
Juan DenisandClaude Opus 4.8 4411a3595f localkit ext: sync v2 chunked push + resumable ranged pulls
Sync v1 POSTs the whole payload in one request, so it is capped by the
panel's 100MB MAX_CONTENT_LENGTH and a drop at 99% means starting over.
v2 makes every request one 8 MiB chunk, which lifts the cap and makes a
retry cost only what was actually lost.

- POST /push/<kind>/init: describes the transfer, reaps stale ones, and
  returns the offsets already confirmed for an identical payload — that
  array IS resume.
- PUT /push/<kind>/chunk: per-chunk sha256, idempotent re-sends, marker
  written after the bytes so a crash costs one chunk, not a corrupt file.
- POST /push/<kind>/finish: verifies the whole-file sha256, THEN runs the
  processing. Missing chunks are a 409 with the received set (resumable);
  a hash mismatch discards. Because processing only ever runs on verified
  bytes, an abandoned transfer cannot leave half-applied state.
- push/code and push/db v1 now delegate to _install_code/_import_db, so
  v1 and v2 share exactly one processing path and the safe-extract policy
  stays mandatory in both.
- Pulls accept ?session= and serve with conditional=True, giving Range /
  If-Range. The session pins one materialized export: pull/db and
  pull/code build their payload per request, and ranges taken from two
  different mysqldump/tar runs would assemble into neither. No session =
  v1 export-and-delete, unchanged.

A transfer is a directory under the temp dir with per-chunk marker files
rather than a shared JSON list, so concurrent chunk writes need no lock.
Transfer ids are validated against uuid4().hex before touching a path.

FEATURES gains 'sync-v2'; older clients keep the v1 routes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 15:35:06 -04:00
Juan DenisandClaude Opus 4.8 8670ebae45 localkit ext: GET /pull/code, pair features, php_version on /sites
The download half of code sync — the missing piece for LocalKit's
"import this remote site as a new local site" flow (LocalKit plan 18).

- GET /pull/code streams a tar.gz of the site's wp-content, built inside
  the container with `docker exec tar czf -` so permissions and structure
  survive. Entries are prefixed `wp-content/`, the same format POST
  /push/code accepts, so both directions share one archive shape. GNU
  tar's exit 1 ("file changed as we read it") is tolerated — a live site
  writing uploads mid-archive is normal.
- GET /pair now reports a `features` array so clients gate their UI on
  capability instead of discovering a 404 mid-operation.
- GET /sites gains `php_version` (regexed off the compose image tag, not
  a per-site container shell) and an explicit `site_url`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 14:39:12 -04:00
Juan DenisandClaude Opus 4.8 0e7bf511f6 feat(extensions): add serverkit-localkit builtin
Backend-only builtin that pairs the panel with LocalKit, the local WordPress
companion app. Ships as a marketplace one-click builtin (no frontend surface),
so it appears in the Marketplace but adds nothing to the sidebar until a user
installs it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 01:16:25 -04:00
Juan DenisandClaude Opus 4.8 9639288494 refactor(extensions): extract seven builtins to standalone repos
Analytics, CrowdSec, DNS Server, GPU Monitor, Kubernetes, Mail Server and
Automations (tramo) now live in their own repositories and install from the
registry as versioned, sha256-pinned zips. Their sources leave
builtin-extensions/, their pre-bundled frontends leave frontend/src/plugins/,
and registry_index.json gains a `bundled: false` entry for each so the
Marketplace offers them as normal registry installs.

The panel keeps only the wiring: comments at the former mount points explain
where each surface now comes from, and the gpu auto-install gate in
extension_migration.py is dropped since a host-probe can no longer decide to
re-vendor an extension that is not in this tree.

Also untracks backend/app/plugins/serverkit-{gpu,k8s,tramo}/, which were the
panel's own install artifacts rather than source. Committing those shipped a
second copy of code that now lives in three other repos, guaranteed to drift.
.gitignore now excludes the install path for every extracted slug on both the
backend and frontend side, so a local install can never be committed back.

Four tests from test_tramo_extension.py exercise panel core rather than the
extension (the retired /api/v1/workflows blueprint, the plan-45 event emitters
ported to EventService, and the docker_service double-emit fix); they move to
test_tramo_panel_core.py and stay here. The rest of the extension suites go
with their repos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 01:16:16 -04:00
Juan DenisandClaude Opus 4.8 7bc323b41b feat(analytics): log ingestion + optional geo (Phase 6)
- log_ingest_service.py: generalized apache/nginx combined-log parser (skips
  assets, non-GET, and bots so log hits approximate real pageviews). Incremental
  per-site cursors — file sources use (inode, byte_offset) with rotation/
  truncation reset; docker sources use `docker logs --since <last_ts>` with a
  ts-based boundary dedup. Events tagged source='log' so JS vs log stay
  comparable. run_log_tail (scheduled) + ingest_site.
- geo.py: optional country-only GeoLite2 lookup, off by default, cached reader,
  returns None on any miss (no package / no DB / private IP). Wired into JS
  build_event and log ingestion; no IP is ever stored.
- analytics.py: admin POST /sites/<id>/ingest-logs (ops/testing trigger).
- 12 new tests (parser, asset/bot/non-GET filter, log events, since-dedup, file
  cursor advance, disabled skip, run_log_tail + global toggle, endpoint, geo
  off/no-db, country null default). 83 total.

The realtime Socket.IO namespace is intentionally deferred (a plan fast-follow);
the Realtime tab already polls.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:44:58 -04:00
Juan DenisandClaude Opus 4.8 d6e8c737dc feat(analytics): WordPress + nginx tracker wiring (Phase 5)
- wp_integration.py: writes a ServerKit Analytics mu-plugin into a managed WP
  site. Docker sites (the default template uses a named volume, not a bind mount)
  get the file via `docker exec tee` into /var/www/html/wp-content/mu-plugins;
  bare-metal sites via a host write + www-data chown. Idempotent, remove-on-
  disable, feature-detects serverkit-wordpress, remove_all_injections on
  uninstall.
- nginx_integration.py: injects an idempotent, marker-guarded sub_filter block
  (rewrites </body>) into a managed app's vhost, following the WAF include-
  injection precedent. Validates with nginx -t and REVERTS the file if the test
  fails (never leaves nginx broken); reloads (never restarts).
- analytics.py: admin POST/DELETE /sites/<id>/inject/{wordpress,nginx}.
- AnalyticsTab.jsx: feature-detects the extension and shows a banner linking to
  the persistent dashboard, keeping the on-demand log view as a complement.
- 18 new tests (mu-plugin PHP, WP inject/idempotent/docker-exec/remove/
  remove-all, wordpress_available, nginx block insert/strip/idempotent/revert-
  on-invalid/remove, admin-gated endpoints). 71 total. lint + build green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:38:39 -04:00
Juan DenisandClaude Opus 4.8 0def464703 feat(analytics): dashboard frontend, six tabs (Phase 4)
- AnalyticsPage: route-driven tab shell (Overview/Pages/Referrers/Devices/
  Realtime/Sites), loads the site list, topbar site picker for report tabs,
  zero-sites empty state.
- OverviewTab: KPI tiles + recharts area chart (visitors+pageviews) + top
  pages/referrers DataTables + live 30-min counter.
- PagesTab / ReferrersTab: range picker + sorted DataTable.
- DevicesTab: device/browser/OS/country tables in a responsive grid.
- RealtimeTab: active-visitor + pageview counters, minutes selector, recent-hit
  table, 15s polling.
- SitesTab: site CRUD (Modal add form, ConfirmDialog rotate/delete, enable
  toggle) + tracking-snippet modal (copy + outbound-link opt-in).
- RangePicker + format utils; isolated analytics.scss (CSS custom props +
  neutral rgba, namespaced under .analytics-page).
- Synced into frontend/src/plugins/serverkit-analytics. npm run lint (0 errors)
  and npm run build both green; sync --check clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:25:41 -04:00
Juan DenisandClaude Opus 4.8 b61e1d5c18 feat(analytics): tracker JS + snippet generator (Phase 3)
- backend/tracker/sk.js: real cookieless tracker (~3.1 KB min) — pageview,
  external referrer, Navigation-Timing load ms, screen-class bucket, language;
  SPA nav via history patch + popstate; opt-in outbound/download click tracking;
  honors DNT; navigator.sendBeacon (text/plain, no preflight) with fetch
  keepalive fallback; fails silently.
- scripts/build-analytics-tracker.mjs: dependency-free conservative minifier
  (strip comments, trim, keep newlines) → backend/tracker/sk.min.js, with a
  --check drift gate. Built artifact checked in (3147 bytes, under the 4 KB cap).
- analytics.py: GET /sites/<id>/snippet returns the ready-to-paste <script>
  snippet (site_key + panel tracker URL, optional data-outlinks).
- dev/test-tracker.html: manual smoke page (load tracker, fire SPA nav/outlinks).
- 6 new tests (artifact bounded + comment-free, real build served, snippet +
  outlinks flag + 404). 59 total.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:05:29 -04:00
Juan DenisandClaude Opus 4.8 e0cde1d93e feat(analytics): rollups + report API (Phase 2)
- rollup_service.run_rollup: DB-side aggregation of raw pageview events into the
  daily table (overall + page/referrer/device/browser/os/country dims), with
  single-pageview bounce detection. Idempotent + backfill-safe (recomputes every
  (site,date) with raw events).
- report_service: rollup-backed overview/pages/referrers/devices/timeseries with
  a range parser (range= or start/end), plus realtime read from raw events.
  Documents the sum-of-daily-uniques range caveat.
- site_service: tracked-site CRUD (create/update/rotate-key/delete with data
  cascade) + hostname normalization.
- analytics.py: JWT'd endpoints — /sites CRUD + /sites/<id>/{overview,pages,
  referrers,devices,realtime,timeseries} + /rollup. Viewer reads, admin
  mutations (tramo/k8s/mail RBAC precedent).
- 15 new proving tests (rollup overall/dims/idempotent, overview/pages/realtime,
  range parse, full CRUD, RBAC viewer-read/admin-mutation, 401 unauth). 54 total.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 02:00:45 -04:00
Juan DenisandClaude Opus 4.8 713e0e811c feat(analytics): data model + public collector (Phase 1)
- models.py: 4 tables (site/event/daily/log_cursor), all
  ext_serverkit_analytics_* prefixed; site_key generator; JSON settings blob.
- ingest_service.py: cookieless visitor hashing (daily-rotating in-memory HMAC
  salt, never persisted, raw IP/UA discarded), UA bot filter + lightweight
  browser/os/device parse, in-process token-bucket rate limiter (per site_key +
  per IP), buffered batch inserts with a daemon flush thread (5s / buffer_max,
  skipped under TESTING), payload validation + event build.
- analytics.py: public POST /collect (site_key auth, 8 KB body cap, per-site
  origin allowlist + reflected CORS, DNT + bot drop as silent 204, 429 on rate
  limit) and GET /tracker.js (placeholder; real build Phase 3). Client IP via
  the plan-48 trusted-proxy helper.
- rollup_service.py: retention prune (raw events + old rollups); rollup stub.
- 39 proving tests (models/purge, bot/UA/hash units, collector happy path +
  404/403/400/413/429, DNT + per-site override, origin allowlist, CORS reflect,
  preflight, query-string gate, retention prune).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 01:55:02 -04:00
Juan DenisandClaude Opus 4.8 cc3626267a feat(analytics): scaffold serverkit-analytics extension (Phase 0)
Native, privacy-first web-analytics builtin extension. Phase 0 lands the
manifest, backend skeleton, and pre-bundled frontend placeholder:

- plugin.json: entry_point analytics:analytics_bp, /api/v1/analytics prefix,
  models/jobs/schedules/config_schema/lifecycle, operations-nav contribution,
  permissions db/network/filesystem.
- backend: analytics_bp (public /ping), config accessor (DEFAULTS + get_cfg),
  lifecycle (seed config defaults, upgrade backfill, injection cleanup on
  uninstall), job handler stubs (rollup/retention_prune/log_tail) fleshed out
  in later phases.
- frontend: AnalyticsPage placeholder, synced into frontend/src/plugins.
- tests: 14 proving tests (manifest validates, refs resolve, config accessor,
  lifecycle seeding, ping route).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 01:46:15 -04:00
Juan DenisandClaude Opus 4.8 95681f8b80 refactor(status): extract status-pages backend into serverkit-status extension
Plan 47 Phase 2. The status-pages blueprint (public /status/<slug> + /badge +
authenticated management API) and its service move into
builtin-extensions/serverkit-status/backend/ (entry_point status_pages:
status_pages_bp, url_prefix unchanged). Core no longer registers /api/v1/status
(management) or imports the service; the unused /api/v1/observability/status-pages
alias is dropped. The StatusPage/StatusComponent/etc models stay core (G2). The
public status/badge pages work whenever the extension is installed.

The core WordPress health-check job (run_health_checks) now reaches
StatusPageService.sync_component_from_health via get_installed_extension_attr and
no-ops cleanly on a lean panel.

Tests: test_status_extraction.py (8) — extraction shape, install=routes,
health-job no-op, relocated status-page mutation + component-check authz.
Adjusted test_observability_namespace + test_api_authz_hardening accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:34:24 -04:00
Juan DenisandClaude Opus 4.8 e6bf4915b4 refactor(remote-access): extract WireGuard-tunnels backend into extension
Plan 47 Phase 2. The tunnels blueprint + broker/publish/netutil services move
into builtin-extensions/serverkit-remote-access/backend/ (entry_point tunnels:
tunnels_bp, url_prefix unchanged). Core no longer registers /api/v1/tunnels or
imports the services; the Tunnel/ExposedService models stay core (G2).

New reusable core->extension seam get_installed_extension_attr(slug, module,
attr): the agent gateway reaches TunnelBrokerService.schedule_reconcile through
it, so tunnel self-heal still fires when the extension is installed and no-ops
cleanly otherwise. Backend re-acquisition on upgrade is covered by the Phase-2
migration.

Tests: test_tunnel_netutil.py + test_tunnel_reconcile.py re-homed into
test_remote_access_extraction.py (17), which installs the extension and runs the
helpers/reconcile against its modules.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:27:02 -04:00
Juan DenisandClaude Opus 4.8 44a73d7796 refactor(cloud): extract cloud-provision backend + add upgrade re-acquisition
Plan 47 Phase 2. The cloud-provider blueprint + service move out of core into
builtin-extensions/serverkit-cloud-provision/backend/ (entry_point, url_prefix
unchanged). Core no longer registers /api/v1/cloud or imports the service; the
CloudProvider/CloudServer models stay core (G2). The one-time legacy cloud-secret
encryption is dropped from core boot (already run on any panel reaching this
version; the extension encrypts on write).

Also adds run_backend_acquisition() — a one-shot migration that re-acquires the
now-extracted backend on upgraded panels that installed a converted builtin
frontend-only (whose API used to come from core). Without it, ftp/cloud/etc would
lose their API on upgrade. Wired into create_app after run_auto_install.

Tests: test_cloud_provision_extraction.py (7, incl. relocated secret-at-rest +
backend-acquisition migration); secret-encryption cloud block moved out of core.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:22:18 -04:00
Juan DenisandClaude Opus 4.8 2a060c518d refactor(ftp): extract FTP backend into serverkit-ftp builtin extension
Plan 47 Phase 2. The FTP blueprint + service move out of core into
builtin-extensions/serverkit-ftp/backend/ (entry_point ftp:ftp_bp, url_prefix
unchanged). Core no longer registers /api/v1/ftp or imports ftp_service — a
fresh panel that never touches FTP loads none of it. Already in
CONVERTED_BUILTIN_SLUGS, so upgrades re-acquire the backend automatically.

Acceptance (tests/test_ftp_extraction.py): uninstalled = no /api/v1/ftp routes +
app.services.ftp_service gone; installed = blueprint hot-loads and /ftp/status
responds; uninstall removes it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 22:15:07 -04:00
Juan DenisandClaude Opus 4.8 f909103656 Add per-row busy state to Marketplace plugin actions
Track the in-flight plugin id/action so uninstall/update/enable/disable can't be double-fired and each row shows live progress; await loadExtensions() before clearing busy. Includes matching tramo editor and workflow_store tweaks (source + live copies) and tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 06:14:45 -04:00
Juan Denis 6bae6a7a1c Align K8s page with tabgroup layout
Updated both K8sPage copies to use the full-bleed `sk-tabgroup` page shell and content wrappers, including the loading state. This moves warning/empty/main tab content into `sk-tabgroup__content`/`__inner` for consistent spacing and tabbed-page structure, and removes the extra topbar meta text to match the new header pattern.
2026-07-12 04:16:15 -04:00
Juan DenisandClaude Opus 4.8 4fa318fa8b feat(marketplace): curated featured top-5 + k8s/wordpress covers
- Featured order: Faro (100), Automations (90), WordPress (80),
  Cloudflare Zone Ops (70), GPU Monitor (60). Scores set in each bundled
  extension's manifest (both copies); Faro's lives in the registry index.
- Add illustrated covers for Kubernetes and WordPress (extension-icons +
  resolver rules) so they stop falling back to generic category glyphs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 03:01:33 -04:00
Juan DenisandClaude Opus 4.8 30fbdb3bf8 feat(marketplace): feature Faro + Cloudflare Zone Ops, add Faro cover
- Add Faro illustrated cover (extension-icons/faro.png) + resolver rule so
  serverkit-faro shows a 3D lighthouse instead of a flat glyph.
- Mark Cloudflare Zone Ops featured (score 95) in both plugin.json copies.
- Add a wordpress.detail.panel PluginSlot so the Faro (and future WP)
  extensions can render on WordPress site pages.
- Sync the registry offline fallback with the Faro v1.0.1 entry (featured).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 02:09:31 -04:00
Juan DenisandClaude Opus 4.8 475c492d19 feat(marketplace): featured Automations with illustrated icon
- Add an illustrated 3D cover icon for the Automations extension
  (assets/extension-icons/automations.png, transparent bg, 256px) and map
  tramo/automation to it in the extension icon resolver.
- Add manifest 'featured' + 'feature_score' (Automations = featured, score 100).
- Sort featured extensions to the top of the Marketplace by score (score is
  hidden; used only for ordering) and show a gold 'Featured' ribbon on the
  cover + a subtle featured border.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:21:42 -04:00
Juan DenisandClaude Opus 4.8 483f1fa4c0 feat(automations): side-nav Settings layout (like core/WordPress settings)
Replace the single long scroll of stacked cards with the core settings-layout:
a left sub-nav (Engine / Events & port / Pack secrets) and a content pane that
shows only the selected section. Reuses the global .settings-layout/.settings-nav
classes so it matches core Settings and WordPress. Drops the redundant per-card
Refresh (the topbar Refresh covers it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:10:19 -04:00
Juan DenisandClaude Opus 4.8 5fce53f779 feat(automations): Templates marketplace tab (search, favorites, import)
Promote 'New from template' from a popup to its own Templates tab, built to grow
like a marketplace:
- Responsive card grid (icon, name, description, integration tags) with a
  'Use template' action per card.
- Search box to filter.
- Per-card favorite star; favorites persist in localStorage and float to the
  top (server-side store can replace this later without UI changes).
- 'Import template' action: paste a tramo workflow JSON or load a .json file to
  create it as a new automation.
Removes the old modal picker and its topbar button.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:01:03 -04:00
Juan DenisandClaude Opus 4.8 049235b12d feat(automations): two-column New automation dialog with trigger picker
Replace the name-only New workflow modal with a two-column dialog: an explainer
panel (badge, mini flow diagram, use-case examples) on the left and, on the
right, the name field plus a 'How should it start?' trigger picker. Choices seed
a valid builtin trigger node (manual / cron / panel-event via webhook on
/sk/events / incoming webhook) so the editor opens with the trigger already
placed. Own copy/wording; collapses to a single column on narrow screens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:51:19 -04:00
Juan DenisandClaude Opus 4.8 f4b8630e3b feat(automations): richer template picker
Replace the three plain text rows with proper cards: a brand-tinted icon tile
(Telegram/GitHub/Discord/generic), bold name, muted description, integration
tags, and a hover chevron. Wider modal (size lg) with an intro line, and the
Close button moves to the modal footer slot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:42:56 -04:00
Juan DenisandClaude Opus 4.8 6f4246bf0e fix(automations): un-nest template-picker styles from .tramo-page
The 'New from template' picker renders inside a <Modal>, which portals to
<body> -- outside the .tramo-page subtree -- so the .tramo-template rules
nested under .tramo-page never matched it. The name/description spans fell back
to inline defaults and ran together. Move .tramo-template-list/.tramo-template
to top-level scope so the portaled modal picks them up (name bold on its own
line, muted description below).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:33:25 -04:00
Juan DenisandClaude Opus 4.8 52ea07426f fix(automations): center content in sk-tabgroup__inner (match WordPress)
Add the .sk-tabgroup__inner wrapper WordPress/Domains use inside
.sk-tabgroup__content so the content is padded + max-width centered under the
full-bleed header, instead of sitting edge-to-edge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:23:09 -04:00
Juan DenisandClaude Opus 4.8 801fe3bbf6 fix(automations): full-bleed tab-group header to match WordPress/Domains
The topbar was rendered inside a plain padded page-container, so it floated
inset with no header band. Switch the wrapper to the same classes TabGroupLayout
uses for WordPress/Domains -- 'page-container--full-bleed sk-tabgroup' with the
content in 'sk-tabgroup__content' -- so Automations gets the identical
full-width header strip (bordered band, icon+title+tabs+actions) over centered
content.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:20:08 -04:00
Juan DenisandClaude Opus 4.8 18a8248fc1 fix(automations): match the clean Domains/WordPress header layout
Drop the '<icon> tramo . workflows . triggers . integrations' topbar subtext
and the redundant inner card-header on the Workflows/Runs tabs. Actions
(Deploy, New from template, New workflow, Refresh) now live in the PageTopbar
and are contextual to the active tab, so each tab shows a single clean header
over its content -- the same pattern as Domains and WordPress.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:13:23 -04:00
Juan DenisandClaude Opus 4.8 e62d969a8b feat(automations): retire React-Flow Workflow Builder UI (Phase 4 frontend)
The Automations extension (tramo) replaces the drag-and-drop builder.

- Delete builtin-extensions/serverkit-workflows + its mirror, pages/
  WorkflowBuilder.jsx, and all of components/workflow/ (its @xyflow nodes,
  edges, and config panels).
- KEEP @xyflow/react (vendor-flow chunk): pages/AppMap.jsx still uses it.
- Redirect /workflow -> /automations in App.jsx (retired-route pattern).
- Drop the pages/_workflow SCSS partial + its main.scss import.
- Refresh the stale serverkit-workflows comments in DashboardLayout +
  sidebarItems to point at the Automations extension.

Build + lint clean; sync-builtin-frontends --check green (14 managed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 22:27:20 -04:00
Juan DenisandClaude Opus 4.8 79be6e082d test(automations): tramo extension suite + host/model refinements
- test_tramo_extension.py: 40 tests (manifest, models+purge, workflow store
  CRUD/materialize/deploy, host argv/control/status/nt-guard, run_sync,
  jobs harvest+health, events bridge + call-back key, blueprint auth/run
  proxy/settings mask/auth-exempt hooks passthrough, Phase 4 ported emitters +
  legacy export + old routes gone).
- host_service: add _is_windows() seam (test platform without touching global
  os.name, which would break pathlib).
- models: dirty detection via deployed_version counter (race-free vs the
  deploy commit's own updated_at onupdate bump).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 22:23:53 -04:00
Juan DenisandClaude Opus 4.8 675543541b feat(automations): starter workflow templates (Phase 3)
Three seed WorkflowDocs (backup-failed->Telegram, nightly health->GitHub
issue, panel event->Discord) served at GET /tramo/templates, plus
POST /tramo/workflows/from-template/<id> to instantiate one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 22:09:10 -04:00
Juan DenisandClaude Opus 4.8 38c1e33097 feat(automations): serverkit-tramo extension backend (Phase 1)
Embed tramo as the panel's automation surface. Managed @tramo/server
container is the executor; the panel owns workflow docs.

- plugin.json: opt-in builtin, /api/v1/tramo, Automations nav -> /automations,
  harvest+health schedules, ext_serverkit_tramo_* models.
- models.py: TramoWorkflow (doc/slug/enabled/dirty) + TramoRun (harvested).
- host_service.py: TramoHostService, stalwart_service mirror -- _docker/_api
  choke-points, loopback API, encrypted Bearer key + pack secrets in config.
- workflow_store.py: CRUD + materialize enabled docs + deploy (restart).
- run_sync.py: defensive run parser + upsert-by-run_id.
- events_bridge.py: scoped call-back ApiKey + managed EventSubscription.
- lifecycle.py / jobs.py: install/uninstall hooks, harvest + health jobs.
- tramo.py: JWT blueprint (workflows/runs/approvals/host/settings) + one
  auth-exempt /hooks/<path> inbound-webhook passthrough + legacy export.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 22:04:12 -04:00
Juan DenisandClaude Opus 4.8 bde4647fdc feat(k8s): serverkit-k8s extension — manage remote clusters via kubectl
Opt-in builtin extension. Add a cluster by pasting its kubeconfig (stored
encrypted, one row per cluster); the engine shells out to kubectl against a
per-call temp kubeconfig to show cluster health, nodes, namespaces, workloads,
pods with live logs, services, and ingresses, plus scale/restart/delete and
apply-from-paste. Reads need viewer, mutations need admin. Not platform-gated
(kubectl runs anywhere); no cluster needed on the panel host.

Backend: models (ext_serverkit_k8s_clusters), kubectl_service (argv build +
temp-file materialize/cleanup), cluster_service (CRUD + normalizers), k8s
blueprint (/api/v1/k8s), lifecycle. Frontend: K8sPage (7 tabs) mirrored to the
baked plugin tree via sync-builtin-frontends. 30 extension tests green;
manifest/platform suites green; frontend builds; sync-drift clean.

Plan: docs/plans/44_KUBERNETES_EXTENSION_PLAN.md

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 03:41:34 -04:00
Juan DenisandClaude Opus 4.8 b9ee243ba2 feat(cloudflare): restore Round 2 engine — DNSSEC, Origin CA, rules, activity (plan 42 ph2)
Restore the plan-36 Cloudflare Round 2 backend lost in the rebuild (migrations
073/074 survived; the engine + test_cloudflare_round2.py did not):

- CfOpsChange model (verbatim from clean pyc dump, matches mig 074) +
  CfOpsChangeService.record (never-raises) / list.
- Core client (dns/cloudflare.py): get/set_dnssec; create/list/revoke Origin
  certificates; X-Auth-User-Service-Key header mode (origin_ca_key, mig 073).
- Extension service: DNSSEC get/set; Origin CA issue (panel-side CSR, key never
  leaves the box, proxied-hostname warning, scoped-cred 400 not 502) /list/revoke
  + Linux-guarded install via upload_custom_cert; Redirect+Transform rules
  (RULE_PHASES/ACTIONS/PRESETS reusing _find_custom_ruleset + injection guard);
  best-effort _record wired into every mutating method; scope_check probe;
  list_activity.
- Extension routes: /zones/<id>/dnssec, origin-ca, /rules/<slug>, /activity,
  /scope-check (admin-gated, CloudflareError->400, provider->502).
- ssl_service.list_certificates walks /etc/ssl/serverkit + "Origin CA
  (proxy-only)" badge; fixed certbot-branch early-return hiding custom certs.

Also registers EmailBounceState (plan 42 ph3) in models/__init__ (shared file).
Deviation: origin_ca_key implemented as the Fernet Origin CA *service key* (plan
Decision 3), not mig-073's docstring "generated private key" — the generated key
is used inline for install and never persisted. R2 preset backend-untouched (it
lives in the storage-provider + frontend layers). Verified: 83 passed
(round2 30 + zone_settings 42 + extraction 8 + ssl 3), no regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 05:02:54 -04:00
Juan DenisandClaude Opus 4.8 dcde6ab64a fix(extensions): gate GPU auto-install on a present GPU; purge em dashes from manifests
Move serverkit-gpu from CONVERTED_BUILTIN_SLUGS into GATED_BUILTIN_SLUGS behind
a _gpu_present() predicate (shutil.which('nvidia-smi')), mirroring the email
gate: fresh installs stay marketplace-only, upgraded panels without a GPU no
longer get it, upgraded panels with a GPU keep it (D3 continuity). Rewrite the
12 builtin + baked extension descriptions that used em dashes into plain
sentences, keeping every pair in sync. New test_extension_manifests.py walks
both manifest trees to assert no em/en dashes and per-slug description parity.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 23:12:48 -04:00