28 Commits
Author SHA1 Message Date
Juan Denis 9d750a4f6f Remove the /system process and service routes
Both used psutil without importing it, so every call has been a 500
since they were written. Nothing calls them: the process list and
service status come from /processes and /processes/services.
2026-10-10 21:51:15 -04:00
Juan Denis 2f52c87a39 Add /ports/check and /ports/suggest
The deploy paths already know which host ports are taken (app rows,
slot rows, Docker mappings, a bind test) but only used that to pick a
port silently. Exposing it lets the shared port input say "8001 is
used by shop-api" and offer the next free port, instead of failing
at deploy time.
2026-10-03 22:46:54 -04:00
Juan Denis c7b6f145d1 Serve the panel through the Cloud relay tunnel
Until now the connect client refused every http and sio stream, so
Open panel in ServerKit Cloud showed a blank page. The client now
bridges them to the loopback listener, the panel serves its page
under the relay's /t/<device>/ prefix, and the SPA prefixes its
router, API and Socket.IO paths from it. The Cloud grant is checked
against Cloud's JWKS before signing in the panel user with the same
email; panel two-factor still applies.
2026-09-29 23:19:34 -04:00
Juan Denis d0b11714e7 Deploy compose apps through A/B slots
Plan 87 phase C. A compose app on slots runs each slot as its own
project, <name>-a / <name>-b, rendered from the merged compose config:
container_name stripped, only the web service published, on the
slot's loopback port, and every named volume pinned to the name it has
today. Without the pin compose prefixes volumes with the project name
and slot B would start on empty ones; the real-Docker leg checks this.

Stateful services (a database, cache or queue with a named volume)
cannot run twice. They move once, on an explicit previewed split, into
a shared <name>-data project that both slots reach by the same names.

The webhook, template-update and rollback paths deploy through the
engine instead of stopping the stack. The --scale web=2 rolling
restart and the Webhook.zero_downtime flag are deleted: it assumed a
service named web, ignored its own health result, and no UI set it.
Compose slot images get immutable per-deployment tags and are pruned
with the rest.
2026-09-28 22:26:44 -04:00
Juan Denis 0749508599 Run release commands and DB snapshots before the switch
Plan 87 phase D. Both slots share the database, so what changes it
runs once, before the switch, while the old release still serves:
the app's managed databases are dumped (a warning, never a blocker),
then the release command runs in a one-off container of the new
image. A failed release aborts the deploy with the live slot
untouched. The command comes from the app's setting, the Procfile
release: line, or serverkit.yaml; a rollback never re-runs it.

Rolling code back never rolls the database back: after a switch
back the slot card offers the pre-deploy snapshot as an explicit
second step with a data-loss warning. stop_old_before_release is
the announced-downtime fallback for schema changes that cannot stay
backward-compatible for one deploy.
2026-09-28 21:41:52 -04:00
Juan Denis f77aaa1f4d Deploy single-container apps through A/B slots
Plan 87 phase B. An app that opts in keeps two slots, like the
panel updater's two installs. A deploy boots the new release in the
idle slot on its own loopback port while the live one serves, holds
it to the health gate, then switches nginx by moving app.port to the
new slot and rewriting the vhost, so every app.port reader follows
without edits. A watch window switches back on its own if the
release falls over; the old slot stays warm for an instant switch
back, then is stopped (not removed) by a standby sweep.

Opting in adopts the running container as slot a with no restart.
Eligibility is shown with reasons and re-checked on every deploy;
opting out folds the slots back on the next in-place deploy. Slot
images are never pruned and slot ports stay reserved.
2026-09-28 21:29:33 -04:00
Juan Denis acdf6b878d Remove auto-scaling and the load-balancer builder
Replica counts could be changed, but nginx only proxied to one port
and the auto-scale sweep was never scheduled, so neither the
Auto-scale section nor its five routes could work; the nginx
upstream builder had no UI at all. With one live copy per app they
are gone, and the scale-policy table is dropped. Running containers
are not touched. The scaling spec now says replicas are not
pursued.
2026-09-28 09:03:34 -04:00
Juan Denis e688b7b5c3 Let proxied sites long-cache fingerprinted assets
The Docker vhost set no caching at all. An opt-in rule serves files
whose name carries a content hash with a one-year immutable
Cache-Control, proxying exactly like location /. A name only counts
as hashed with 8+ characters and a digit, so app-settings.js is never
pinned. Off by default: the panel cannot know an app's asset layout.
2026-09-27 22:45:53 -04:00
Juan Denis 3feba262cb Name an app's likely bottleneck on its Overview
A rule-based check reads the app's request metrics, its container
CPU and the CPU and top queries of the databases it references, and
names what fits: page cache for a busy app over an idle database,
a cache or index when one query dominates a busy database, slot
deploys for errors during deploys, and TTL for a low hit ratio. Each
hint links to the fix and states its failure mode; nothing is added
automatically.
2026-09-27 22:36:14 -04:00
Juan Denis 45f7100d93 Offer an opt-in PgBouncer beside PostgreSQL
An installed PostgreSQL can switch on a transaction-mode PgBouncer
sidecar through its managed override, with no host port, on the
shared network. fromService gains pooledUrl; apps keep the direct
connectionString unless they switch, because transaction pooling
breaks prepared statements in some drivers, LISTEN and advisory
locks, and the card says so.
2026-09-27 22:27:32 -04:00
Juan Denis dc08f6975c Show top queries and hit ratios per database
The explorer showed live sessions but nothing about which queries
cost the most or whether the cache was working. A container's new
Insights tab lists top queries by total time (pg_stat_statements,
performance_schema digests) with connections in use and the buffer
cache hit ratio, and says why a list is empty. PostgreSQL's
pg_stat_statements is switched on through the config tuner, keeping
existing preloads, so its rollback undoes it.
2026-09-27 21:30:07 -04:00
Juan Denis 6986f497f3 Attach a cache, storage or queue from the app page
Installed Redis and a running app were unrelated until someone wrote
the env by hand. The Overview's Attached services card reuses an
installed service or installs one, writes the fromService references
(REDIS_URL, AMQP_URL/BROKER_URL, the S3_* set), shows that kind's
failure mode, and offers a redeploy. Detach removes exactly what the
attach wrote.
2026-09-27 21:16:58 -04:00
Juan Denis f23c7a5009 Give each app its own bucket and a scoped key
MinIO's images no longer pull, so object storage is now the Garage
template. Attach storage creates a bucket named for the app and a key
that can read and write only that bucket, keeps the secret in an
internal vault, and wires S3_* and AWS_* env through fromService and
fromSecret. Detach revokes the key and removes the env but keeps the
bucket and its objects. A docker leg proves the scope against a
real Garage.
2026-09-27 20:36:38 -04:00
Juan Denis d2fd3b38b0 Show per-app request rate, errors and latency
A one-minute builtin tick tails each app's timed access log from a
stored byte offset and rolls it into minute and hour rows: status
classes, cache HIT/MISS/BYPASS/STALE and a latency histogram, so
p50/p95/p99 merge exactly across ticks and into hours. Minute rows
go after 48 h and hour rows after 30 days on the same tick. The
Overview tab gets a Traffic card.
2026-09-27 18:18:30 -04:00
Juan Denis d3baa27b72 Regenerate inventories for the Storage work
Adds GET /system/storage to the API surface, records the Storage
tab's form block in the migration inventory, and seeds the history
retention test through make_application.
2026-09-27 17:04:36 -04:00
Juan Denis 04eecd5eb3 docs(api): regenerate API_SURFACE.md for the managed-profile route
The connect managed presentation profile (d866de54) added
GET /connect/managed-profile without regenerating the committed
inventory, failing test_committed_surface_matches_served_surface.
2026-09-19 05:09:53 -04:00
Juan Denis 172e410ce3 chore: prepare 1.11.0 and refresh release inventories [skip version] 2026-09-08 09:31:21 -04:00
Juan Denis fde0acee6d fix(ai): satisfy release gates and prepare 1.10.0 [skip version]
Keep connection persistence behind the service boundary, use typed errors and shared JSON parsing, and refresh reviewed inventories. The new GET connection catalog accepts API keys so API clients managing connections can discover IDs without exposing credentials; existing JWT-only routes stay unchanged. Add regression coverage and release notes.
2026-09-08 01:30:03 -04:00
Juan Denis d67e5d07d0 fix: complete audit follow-ups and add regression coverage
Centralize backup schedule calculations and shared request/form flows, encode query values, and move repository finalization into its service. Add browser CI coverage, repair stale test fixtures, and provide opt-in request and build measurement tools.
2026-09-05 14:41:15 -04:00
Rebertt RodriguesandJuan Denis a092ba1afe fix(dashboard): avoid content blockers on metrics history requests (#133)
Co-authored-by: Juan Denis <jhd3197@gmail.com>
2026-09-04 22:20:22 -04:00
Juan DenisandClaude Opus 5 4dfb468f77 chore(connect): regenerate the surface, inventory and locale files
Generated files the connect work moved and nobody re-ran:

- docs/API_SURFACE.md gains GET /connect/status
- docs/MIGRATION_INVENTORY.md: the hand-shaped error bodies count is 1145
  (ceiling 1150) for the 409 the backups PUT now answers when Cloud owns
  the destination
- en.json gains app.backups.storageManagedByCloud, so the sentence the
  read-only backup form shows is translatable like every other string

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018q56AfqpgKjio4n9KYubvZ
2026-09-03 00:19:35 -04:00
Juan DenisandClaude Opus 5 c8ae7f35f2 chore(docs): regenerate the API surface and migration inventory
Both committed inventories went stale behind the walkthrough work, so the
two ratchets that pin them failed on dev:

  test_api_surface_inventory::test_committed_surface_matches_served_surface
  test_migration_inventory::test_the_committed_inventory_matches_the_measured_state

API_SURFACE picks up GET and PUT /walkthroughs/definitions, the endpoints the
Walkthrough Studio contribution contract added. MIGRATION_INVENTORY moves the
hand-shaped {'error': ...} census 1141 -> 1142 against an unchanged ceiling of
1150; the routes above are the new sites and the ceiling still holds.

Regenerated, not hand-edited:
  SERVERKIT_UPDATE_API_SURFACE=1 pytest tests/test_api_surface_inventory.py
  python scripts/generate-migration-inventory.py

Version Bump failed only as a dependent of Backend CI ("Sibling CI failed;
VERSION will not advance") and needs no change of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 00:21:38 -04:00
Juan DenisandClaude Fable 5 160bf6dab7 feat(panel): one-click self-update from the About tab
The About tab could only detect a new release and link to it; applying the
update meant SSH and scripts/update.sh. Wire that script to the UI:

- panel_update_service launches update.sh via systemd-run in its own
  transient unit — a plain subprocess child would be killed when the
  updater restarts the panel's own systemd unit. Status is stateless
  (transient-unit state + the updater's own update-*.log) so it survives
  the mid-update backend restart.
- Capability gates keep the button honest: root systemd installs only.
  Docker installs get the docker compose pull instructions; dev servers
  and hosts without systemd-run get a reason instead of a button.
- GET/POST /api/v1/system/update (admin_required, confirm-gated,
  audited). API surface doc regenerated.
- About tab gains Update Now with inline confirm, live log tail while
  the update runs, a restarting phase that tolerates the panel being
  briefly unreachable, and success/rollback/timeout outcomes. New
  aboutTab locale keys in en + es.

The heavy lifting (blue/green switch, migrations, health check,
automatic rollback) stays in update.sh, which is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-31 08:09:52 -04:00
Juan DenisandClaude Fable 5 aee50fea0c cutover: contracts follow the flagship departure
API surface regenerated (the flagship-seeded cloudflare zone routes no
longer boot with a lean panel); the frontend route contract gains an
explicit EXTRACTED_EXTENSION_PREFIXES excuse list for the /git, /email and
/cloudflare prefixes core clients probe fail-soft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 14:57:57 -04:00
Juan DenisandClaude Fable 5 2bdddd4d77 chore: regenerate ratchet inventories after the git split
Controller-boundary baseline drops the three api/security.py crossings that
moved to serverkit-image-scan; API surface sheds the 18 Gitea operations;
migration inventory regenerated by preflight; test-count floor raised
4735 -> 4799 (SERVERKIT_CLEAN_COLLECT).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 14:35:38 -04:00
Juan DenisandClaude Fable 5 9f92850cbd feat(security)!: extract the install-gated security suite to extensions (plan 47 Ph3b-5 / plan 55 Ph3)
The 1791-line security omnibus splits along tool lines and the install-gated
surfaces leave core for five standalone extensions - serverkit-clamav
(ClamAV+YARA scanning, quarantine, scan jobs), serverkit-fail2ban
(management UI; probes stay core on Fail2banJailService), serverkit-lynis,
serverkit-auto-updates, and serverkit-image-scan (headless CVE/SBOM
scanning; models stay core per the plan 52 D1 data seam). Each mounts its
routes back at the same /api/v1/security prefix when installed, so route
shapes never change; absent extensions mean plain 404s, no dead UI.

Core keeps the zero-host-package baseline: status/config, legacy integrity,
suspicious activity/events, SSH keys, IP lists, the audit (fail2ban check
now probes via the jail service; updates check was already self-contained),
scoped FIM, and the scan/alert logs + notification helpers extensions write
through. Job handlers for security.malware_scan/lynis_scan/image_scan now
come exclusively from the extensions' manifest jobs key.

Security page becomes core-tabs-only; extension tabs contribute into the
security tab group (3a groupId). Overview posture drops the hardcoded
ClamAV rows and offers the Marketplace when no security extension is
installed. Upgrade parity: run_registry_auto_install() installs each
extension from the registry once, gated on the host tool actually being
present, fail-soft until the registry entry is published.

Wizard round 2 (Ph5): a security-posture chooser (minimal/recommended/
hardened) on the Summary step, catalog-resolved so unpublished slugs never
render dead installs; persisted as onboarding_security_posture.

Moved test suites re-home to the extension repos; panel keeps
test_security_suite_extraction.py (18 proving tests: absent-shape 404s,
lean summary, jail-service probes, migration gates, posture wiring).
docs/API_SURFACE.md regenerated (34 operations moved out).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-29 13:53:04 -04:00
Juan DenisandClaude Fable 5 69b5fbe9eb docs(api): regenerate API_SURFACE.md for the connection provider routes
The seven /connections/providers operations shipped in 939c77b6 without
the surface pin being regenerated; the diff is exactly those routes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-28 22:20:34 -04:00
Juan DenisandClaude Fable 5 0fe6d090fd test(api): pin the served API surface to a committed inventory
OpenAPIService.generate_spec() existed but nothing snapshotted it, so a
route rename/removal/method change shipped invisibly to API-key
consumers, the CLI, and agents. docs/API_SURFACE.md now pins the sorted
METHOD /path inventory (1418 operations) with the same
regenerate-and-review contract as docs/MIGRATION_INVENTORY.md: an
intentional surface change regenerates the doc in the same commit
(SERVERKIT_UPDATE_API_SURFACE=1 pytest
tests/test_api_surface_inventory.py) so the diff is the review.
Method+path only on purpose — summaries and schemas churn without
breaking anyone. Named *_inventory so preflight's globs run it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-27 05:03:40 -04:00