1423 Commits
Author SHA1 Message Date
Juan Denis d7b3b8526e Merge pull request #78 from jhd3197/dev
The Deploy Console and runtime extension frontends
v1.7.53
2026-07-22 20:36:58 -04:00
github-actions[bot] d320cacb73 chore: bump version to 1.7.53 [skip ci] 2026-07-23 00:07:30 +00:00
Juan DenisandClaude Opus 4.8 3f3bff3132 test: bump BASELINE_COUNT ratchet 2507 -> 2534 for the batch's new suites
Locks in the proving tests added across plans 52/53/55 (runtime-frontend recovery
suites, db-snapshot relocation, generic fail2ban jail, gamekit framework, minecraft
install wiring, html-sink sweep) so they can't silently disappear.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:22:43 -04:00
Juan DenisandClaude Opus 4.8 28cece380f feat(security): HTML-sink sweep guard (D4) — keep XSS sinks swept
Plan 55 Phase 1. The sweep found zero vulnerabilities: every raw-HTML sink is
safe by construction (escape-first SQL/file highlighters, sanitizeSvgInner for
extension icons, the escape-then-allowlist markdown renderer) and the backend
has no |safe/Markup(/render_template_string.

To keep it that way: scripts/check-html-sinks.mjs (wired into npm run lint) and a
Python mirror test (test_html_sink_sweep.py, runs in the backend suite) fail on
any raw-HTML sink that neither references an allowlisted sanitizer nor carries a
'sink-safe:' annotation. Standardized annotations on the three variable-fed sinks
(markdown, SQL editor, marketplace icon) and documented the rule in SECURITY.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 19:22:43 -04:00
Juan DenisandClaude Opus 4.8 cf9cc5d662 fix(games): drop separate indexes on minecraft ext models
The index=True columns emitted separate CREATE INDEX statements that, once the
ext_serverkit_minecraft_* models register on the shared test metadata mid-session
(when the install test runs), collided during a later test's create_app schema
sync — aborting create_all before system_settings and cascading into errors
across the suite. A scaffold needs no secondary indexes; drop them (they can be
reintroduced with proper migration handling when the Docker create-flow lands).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:48:47 -04:00
Juan DenisandClaude Opus 4.8 fd6f1733c3 feat(games): serverkit-minecraft extension backbone + gamekit framework
First vertical of the game-server family (plan 53 Phase 0-1, backend core):

- gamekit/ — the reusable, dependency-free framework (D1): a Source RCON client
  (framing/auth/timeouts), a container-log event parser (join/leave/started/
  stopped/crashed, Java + Bedrock), a server.properties config-form builder over
  a JSON sidecar, and the save-aware hot-backup sequence (save-off -> flush ->
  zip -> save-on, always resumes) with retention.
- server_properties_meta.json sidecar (Phase 0 authoring task): grouped, labelled,
  restart-required metadata for the settings form.
- Extension scaffold: manifest (category 'games', models:register), ext_serverkit_
  minecraft_* models keyed by application (D9), and the /api/v1/minecraft blueprint
  (list/get, RCON proxy, backups).

Unit-tested (fake RCON socket, fixture logs, temp worlds) + install-wiring-tested
(install registers routes + creates ext tables + uninstall). The create/console/
UI/Docker flow is the dev-box phase and is intentionally not claimed here.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 18:14:35 -04:00
Juan Denis ee105d0dc6 Ignore graphify output directory
Adds `/graphify-out` to `.gitignore` so generated Graphify output files are not tracked in the repository.
2026-07-22 17:48:10 -04:00
Juan DenisandClaude Opus 4.8 b6692b798f refactor(security): generalize the Fail2ban jail engine beyond WordPress
Split the WP-specific jail service into a generic engine + thin WP wrappers:
enable_jail(key, filter_name, filter_content, logpath, port, ...), a generic
ensure_filter(name, content), remove_jail(key), and jail_name_for_key(). The
jail template's port is now parametrized. enable_wp_jail/disable_jail/get_status
keep byte-identical behavior (existing WP suites unchanged and green), so the WP
extension's callers are untouched. This gives other verticals (game-server
login/RCON jails, plan 53) a reusable per-service brute-force jail seam.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:37:20 -04:00
Juan DenisandClaude Opus 4.8 fb19778e7f docs(wordpress): correct extraction audit — frontend not yet extracted
The audit overstated the WP frontend as contributed by the extension; in the
tree the extension index.jsx is still a re-export shell over core pages/
components/SCSS. Corrected to match reality before plan 52 Phase 4 moves it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:31:29 -04:00
Juan DenisandClaude Opus 4.8 2ffab5bf86 refactor(models): relocate DatabaseSnapshot/SyncJob; untrack stale gui/email copies
Phase 1 housekeeping for the full-isolation round:
- Move DatabaseSnapshot + SyncJob out of models/wordpress_site.py into a neutral
  models/db_snapshot.py (pure module move, tables unchanged); wordpress_site.py
  re-exports both for import compatibility.
- Untrack the stale committed live copies of serverkit-gui (sibling repo, no
  in-tree source) and serverkit-email (source of truth: builtin-extensions/
  serverkit-email/), matching how every other extracted builtin is already
  gitignored. Files stay on disk; only removed from the index.
- .gitignore: add the email + gui frontend/backend artifact paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:31:18 -04:00
Juan DenisandClaude Opus 4.8 ffc1fd7728 feat(extensions): deliver runtime ESM frontends to production panels (#39)
Wire the server half of runtime-frontend delivery: get_active_contributions()
now emits panel sdk_version and a 'frontends' descriptor map {slug: {entry,
hashes, sdk_version}} for every active extension whose frontend_entry is a .mjs
ESM bundle. The bundle sha256 is pinned at install time into the plugin config
under the reserved _frontend_hashes key (no migration — reuses config_json);
update_plugin_config() and the config API preserve/hide reserved keys. Add the
extensions.runtime_frontend kill switch (default on). The client loader already
consumed this shape, so an installed extension's UI now renders on a pre-built
panel with no Vite rebuild. Un-skips the restored plan-25 recovery suites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:24:36 -04:00
Juan DenisandClaude Opus 4.8 54204d69c5 test(deploy): bump CI test-count ratchet to 2507 for the Deploy Console suites
The Deploy Console adds test_run_log_stream.py and test_deploy_console.py
(+77 collected tests). Full backend suite green: 2454 passed, 53 skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 17:01:22 -04:00
Juan DenisandClaude Opus 4.8 6bf8c87b9d docs(deploy): document the Deploy Console; update CHANGELOG and ROADMAP
Add docs/DEPLOY_CONSOLE.md (the feature + architecture), an architecture section
in docs/ARCHITECTURE.md, a CHANGELOG (Unreleased) entry, and move the Deploy
Console into the ROADMAP's Recently Completed list with its Active Development
checkboxes ticked.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:43:50 -04:00
Juan DenisandClaude Opus 4.8 0c094aea8d feat(deploy): deployments index deep-links to the console + global "deploy running" pill
The /deployments index is now a single table whose rows open the full-page Deploy
Console (the old inline detail pane is removed). A small global pill appears while
any deploy is pending/running and links straight to its console — so you can
navigate away mid-deploy and always find your way back (plan 51 §1.3, task 26/28).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:43:43 -04:00
Juan DenisandClaude Opus 4.8 16518b55bd feat(deploy): route every deploy flow to the Deploy Console
Template install (InstallModal), New Service repo create, Service ▸ Deploy Latest
Commit, and the Build tab's Deploy Now all now navigate to /deployments/<jobId>
on job start instead of embedding progress or dropping the user on a page with no
watcher (plan 51 Phase 2). The template modal's embedded progress block is
removed; the deploy API client gains retryDeploymentJob and a plan option.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:37:35 -04:00
Juan DenisandClaude Opus 4.8 b0148d03f9 feat(deploy): the Deploy Console — full-page live install/deploy view
New /deployments/:jobId page (plan 51 §1.1, Phase 3): humanized title, status
pill, live elapsed timer and step counter; a step rail (✓ with per-step duration,
spinner, ✗, ○) that scrolls the log pane to a step; a dark monospace log pane
with follow/wrap/timestamps/level-filter/search/copy/download and a "Jump to live"
chip; a pinned error card (failed step, real failure tail, plain-language hint,
Retry / Copy / Ask-AI); a success banner (duration, step timings, Open app / View
service / View runtime logs); plus queued, degraded (poll-fallback) and history
states. Design-system SCSS only, bespoke .deploy-console__* BEM (D12).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:37:28 -04:00
Juan DenisandClaude Opus 4.8 a5247573e5 feat(deploy): live deploy transport — socket client + useDeployJobStream hook
Add the deploy_log/deploy_status handlers and subscribeDeploy/unsubscribeDeploy
to the socket singleton, and useDeployJobStream — one hook for the console, the
compact widget, the LogsTab banner and the topbar pill. It boots from a full
snapshot, prefers the live socket channel, de-dupes log rows by DB id, re-syncs
with after_id on every (re)connect, and falls back to 2s polling when sockets
are unavailable (plan 51 D2/§5.5). DeploymentJobProgress now consumes the hook
and links to the full console.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:37:16 -04:00
Juan DenisandClaude Opus 4.8 ebdb7cc16f feat(deploy): every deploy is a job — async deploy_app, retry endpoint, plan-in-GET
POST /builds/apps/<id>/deploy now enqueues an app_deploy job and returns 202 +
{deploy_job_id} so the UI opens the live Deploy Console; ?wait=true preserves the
old synchronous contract for CLI/tests (plan 51 D7). Deploy options (no_cache,
version_tag) ride the job plan through to the async runner.

Add POST /deployment-jobs/<id>/retry: clones a FAILED job (both kinds) and
enqueues a fresh run on a new console URL (D8). Extend GET /deployment-jobs/<id>
with ?plan=true so the console can render the step rail.

Tests cover the socket emit helpers (room + payload), the subscribe_deploy auth
guard, the 202/wait contract, retry guards + clone fields, plan-in-GET, and the
streaming-compose variant incl. a failure persisting a real stdout-bearing tail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:37:08 -04:00
Juan DenisandClaude Opus 4.8 a1c5d17d78 feat(deploy): batched RunLogStream seam + streaming compose + deploy socket channel
Introduce app/services/run_log_service.py — the single crash-proof write seam
for all deploy-path logging (plan 51 D4/D5/D6/D10). It batches writes (one DB
commit + one socket emit per flush instead of commit-per-line), keeps a truthful
80-line in-memory failure tail, records per-step timings, sanitizes ANSI/\r,
caps persisted rows at 5000, and matches a plain-language failure hint. The
DeploymentPlanRunner now funnels log/step/close through it.

Add DockerService.compose_up_streaming (Popen + --ansi never --progress plain,
line callback) so template installs stream the real pull/build transcript live
and a failure persists a meaningful tail — the biggest reason install errors
were unreadable. The runner's local compose.up step adopts it.

Replace the orphaned build_* socket surface (zero subscribers, grep-verified)
with the Deploy Console channel: emit_deploy_log / emit_deploy_status to room
deploy_{job_id}, plus subscribe_deploy/unsubscribe_deploy (authenticated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 16:36:58 -04:00
Juan DenisandClaude Fable 5 ab37bcca23 docs(roadmap): refresh to v1.7.x reality, add H2 2026 active-development section
The roadmap still said v1.6.0 / last-updated March 2026 while dev sits at
v1.7.52. Reframe the status note (actively developed, three tracks), refresh
Recently Completed with the actual 1.7.x highlights, add an Active Development
section (deploy console, extension platform, game server extensions, agent
streaming, release engineering), fix the milestones table incoherence, and
retitle the long-completed 'Upcoming Development' block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 14:21:03 -04:00
github-actions[bot] 566c2b80d9 chore: bump version to 1.7.52 [skip ci] 2026-07-22 13:31:39 +00:00
Juan Denis 3628da8b79 Merge branch 'dev' into main: link panel tab in Add Server drawer v1.7.51 2026-07-22 09:31:29 -04:00
Juan Denis b50a008984 feat(ui): move linked-panel flow into the Add Server drawer as a third tab
The standalone card below the servers list was UI drift — the link flow
('Link panel') now lives inside the existing Add Server modal next to
'Connection string' and 'Pair code', using the drawer's own form styles.
2026-07-22 09:31:25 -04:00
github-actions[bot] cf045400bf chore: bump version to 1.7.51 [skip ci] 2026-07-22 13:08:39 +00:00
Juan Denis d80518966b Add Trendshift badge to all README variants
Adds the Trendshift repository badge to the main README and the Spanish, Portuguese, and Simplified Chinese README files so project visibility/trend metrics are shown consistently across all language entry points.
2026-07-22 09:08:26 -04:00
github-actions[bot] 4fc849da90 chore: bump version to 1.7.50 [skip ci] 2026-07-22 04:48:22 +00:00
Juan Denis 472dfab443 Merge branch 'dev' into main: linked-panel compose path fix v1.7.49 2026-07-22 00:48:15 -04:00
Juan Denis b0ccd2f5b5 fix(linked-panel): accept compose file path in project_path from remote runner
The deployment runner forwards the plan's compose_file in the project_path
field; the embedded agent now splits file paths into (dir, compose_file)
before calling DockerService, fixing '[Errno 20] Not a directory' on
remote template installs.
2026-07-22 00:48:13 -04:00
github-actions[bot] cef8a3697b chore: bump version to 1.7.49 [skip ci] 2026-07-22 04:35:45 +00:00
Juan Denis 5255d09658 Merge branch 'dev' into main: ServerKit-to-ServerKit peering v1.7.48 2026-07-22 00:35:36 -04:00
Juan Denis db3c29179d feat(servers): ServerKit-to-ServerKit peering — embedded agent mode
Lets a panel link itself to a master ServerKit panel as a worker WITHOUT
installing the standalone Go agent: the panel registers via the master's
existing token flow, then runs an embedded agent client (HMAC auth + HTTP
long-poll transport) that dispatches agent commands (file:write,
docker:compose:up/ps/logs, system:info/metrics) onto its own local
services. The master needs zero changes — a linked panel is just another
agent-backed Server row with scoped, revocable credentials.

- linked_panel_config table (migration 077) + Fernet-encrypted secret
- embedded agent client: connect/poll/result loop, heartbeat metrics,
  per-command threads, allowed-roots guard on file:write
- API: GET/POST/DELETE /api/v1/linked-panel (+ startup resume hook)
- Servers page: LinkedPanelCard (link form, live status, unlink)
2026-07-22 00:35:33 -04:00
github-actions[bot] 40a86928ab chore: bump version to 1.7.48 [skip ci] 2026-07-22 03:26:22 +00:00
Juan Denis be20b59820 Merge branch 'dev' into main: template bind-mount fix v1.7.47 2026-07-21 23:26:16 -04:00
Juan Denis 465a4ee9a1 fix(templates): only bind-mount absolute file paths; relative files are build-context only
A template file with a relative path (prompture-hub's Dockerfile) was
bind-mounted as './Dockerfile:Dockerfile', failing compose up with
'invalid mount config: mount path must be absolute'.
2026-07-21 23:26:13 -04:00
github-actions[bot] 343073c244 chore: bump version to 1.7.47 [skip ci] 2026-07-22 03:13:17 +00:00
Juan Denis 0e3bddf30b fix(templates): pin prompture-hub to 0.0.1 (0.0.2 was never published to PyPI) 2026-07-21 23:13:08 -04:00
Juan Denis c0c1d36456 Merge branch 'dev' into main: prompture-hub template pin fix 2026-07-21 23:12:42 -04:00
Juan Denis 79420e58ad fix(templates): pin prompture-hub to 0.0.1 (0.0.2 was never published to PyPI) 2026-07-21 23:12:39 -04:00
Juan Denis 9baa8f43ea Merge branch 'dev' into main: fix stuck deployments, repo-app auto-deploy, unified install UX 2026-07-21 22:57:59 -04:00
github-actions[bot] f601782fde chore: bump version to 1.7.46 [skip ci] 2026-07-22 02:57:45 +00:00
Juan Denis 98ad0481a5 fix(deployments): stop stuck 0% installs, deploy repo apps on create, unify install UX
- deployment_job_logs.id: BigInteger -> with_variant(Integer, 'sqlite');
  BIGINT PK never autoincrements on SQLite, so the first log insert crashed
  every fresh install (jobs stuck 'running' at step 0 with no logs)
- migration 076 rebuilds the table on existing SQLite databases
- runner: first status/log/telemetry inside try, non-raising log(),
  rollback-safe failure marking — any failure now fails the job visibly
- service: guard enqueue, crash-proof run_job, fail jobs left 'running'
  across restarts at boot
- repo-based services (POST /apps/from-repository) now enqueue a real
  'app_deploy' DeploymentJob (build + start via DeploymentService.deploy)
  and return deploy_job_id
- compose override injects restart: unless-stopped for services without a
  policy, so apps come back after reboot
- GET /deployment-jobs gains an app_id filter
- frontend: shared DeploymentJobProgress component (timeout + visible poll
  errors) used by the Templates InstallModal and the service Logs tab;
  repo wizard navigates to the Logs tab to watch the deploy live; install
  success lands on the Logs tab; failure toasts surface real server errors
2026-07-21 22:57:33 -04:00
Juan DenisandClaude Opus 4.8 59a64e4337 serverkit-localkit: advertise sync kinds + site kind (plan 26)
The LocalKit contract for per-kind sync (plan 26): /pair now returns a `kinds`
array (site kinds this extension can sync) and every /sites entry carries a
`kind`. Both keep old-server ↔ new-client safe — a LocalKit client defaults an
absent kind to "wordpress" and gates its php sync/import UI on the `kinds`
advertisement.

KINDS is ['wordpress'] today: serverkit-wordpress is this extension's only site
backend, so advertising php would be untrue. Add 'php' here in lockstep with a
php-stack backend that _resolve_site / _install_code / _import_db can dispatch
to; the client already speaks the php protocol (engine-native db, app/ code).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:57:33 -04:00
Juan Denis f1c5ff1688 Merge pull request #77 from jhd3197/dev
Test Sandbox, resumable LocalKit sync, and installer hardening
v1.7.45
2026-07-21 00:24:07 -04:00
github-actions[bot] db8d3126a1 chore: bump version to 1.7.45 [skip ci] 2026-07-21 03:52:15 +00:00
Juan Denis baebb92d3a feat(registry): default to serverkit.ai index and ship logo fields
Flip DEFAULT_REGISTRY_URL to https://serverkit.ai/ext/index.json (the
long-planned go-live default: cached proxy of the raw index with logo URL
rewriting). Add logo fields to the bundled offline index so Marketplace
cards get real art even before the first successful fetch, and update the
registry tests + docs for the new default.
2026-07-20 23:46:02 -04:00
Juan Denis 99ba978731 fix(deployments): add target_server_name property to DeploymentJob
DeploymentRunner.run() logged self.job.target_server_name, but the model
only exposed that value as a to_dict() key, so every deploy job crashed
with AttributeError at its first step and was marked failed before any
work ran. Add a target_server_name property and reuse it in to_dict().
2026-07-20 22:38:30 -04:00
github-actions[bot] e2887342a8 chore: bump version to 1.7.44 [skip ci] 2026-07-21 02:29:07 +00:00
Juan Denis 5e8012de61 Add test sandbox and harden install scripts
Introduces a full-stack Test Sandbox feature to run distro matrix tests in Docker from the UI: new backend API/service, sandbox_runs model + migration, run/log endpoints, and a new frontend page with mode selection, run history, live polling, cancel, and per-distro logs.

Also adds ARM64 scripts CI coverage (ubuntu-24.04-arm) and strengthens install/test automation for minimal images and overlay rebuilds: early curl/bootstrap index refresh, stricter venv validation, release-venv glibc fallback, Node reprovisioning on source fallback, rsync --delete overlay behavior, npm bootstrap in VM install, Windows tar pinning in PowerShell harness, plus updated test docs and distro coverage notes.
2026-07-20 22:26:19 -04:00
Juan Denis e79fb1cf11 Merge branch 'dev' of https://github.com/jhd3197/ServerKit into dev 2026-07-20 22:25:56 -04:00
Juan DenisandClaude Opus 4.8 4411a3595f localkit ext: sync v2 chunked push + resumable ranged pulls
Sync v1 POSTs the whole payload in one request, so it is capped by the
panel's 100MB MAX_CONTENT_LENGTH and a drop at 99% means starting over.
v2 makes every request one 8 MiB chunk, which lifts the cap and makes a
retry cost only what was actually lost.

- POST /push/<kind>/init: describes the transfer, reaps stale ones, and
  returns the offsets already confirmed for an identical payload — that
  array IS resume.
- PUT /push/<kind>/chunk: per-chunk sha256, idempotent re-sends, marker
  written after the bytes so a crash costs one chunk, not a corrupt file.
- POST /push/<kind>/finish: verifies the whole-file sha256, THEN runs the
  processing. Missing chunks are a 409 with the received set (resumable);
  a hash mismatch discards. Because processing only ever runs on verified
  bytes, an abandoned transfer cannot leave half-applied state.
- push/code and push/db v1 now delegate to _install_code/_import_db, so
  v1 and v2 share exactly one processing path and the safe-extract policy
  stays mandatory in both.
- Pulls accept ?session= and serve with conditional=True, giving Range /
  If-Range. The session pins one materialized export: pull/db and
  pull/code build their payload per request, and ranges taken from two
  different mysqldump/tar runs would assemble into neither. No session =
  v1 export-and-delete, unchanged.

A transfer is a directory under the temp dir with per-chunk marker files
rather than a shared JSON list, so concurrent chunk writes need no lock.
Transfer ids are validated against uuid4().hex before touching a path.

FEATURES gains 'sync-v2'; older clients keep the v1 routes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 15:35:06 -04:00