cli.py runs as a plain python script, so the factory's flask-argv sniff
never matched it: every serverkit verb booted a second job consumer,
scheduler and relay beside the live panel, whose boot reconcile marks
the panel's in-flight deploys failed.
A fresh panel no longer lets whoever reaches it first become admin:
registering the first account needs a one-time code the installer
prints (`serverkit setup-code` repeats it). SERVERKIT_ADMIN_EMAIL and
SERVERKIT_ADMIN_PASSWORD create the admin unattended instead.
Git apps now deploy what users expect: a repository with a compose
file deploys every service it declares, "Deploy latest" builds the
branch's newest commit, and push webhooks rebuild the containers.
Also fixes branch switching on single-branch clones, uploads of a
lone file, compose v1 port preflight, and CLI commands that ignored
their flags.
- app/services/connect_client.py: RelayClient background thread (WS hello
signed device_id|ts|nonce, 20s pings, jittered 1s->60s backoff with reset
after 2 stable minutes, >6 reconnects/10min flap log, close-code ->
state_reason mapping, terminal revoked, long-poll fallback with 15min WS
re-probe); connect-state.json runtime state merged into status();
disconnect() stops the client and removes the state file; run_doctor()
implements the connect check table
- app/services/connect_keys.py: strict load_keypair() for the transport
- app/__init__.py: start_client_if_paired() next to the linked-panel hook
- cli.py: connect status shows transport/last-seen/relay-instance; new
connect doctor command (exit 1 on hard-check failure)
- requirements.txt: websockets==15.0.1 (WS client; the SocketIO stack is
server-side only)
- tests/test_connect_relay.py: 21 unit tests for backoff, flap detection,
close-code mapping, hello signature, state file, loop transitions
- app/services/connect_keys.py: Ed25519 device keypair (0600, config dir),
fingerprint = sha256(raw pubkey)[:16] shown in groups of four
- app/services/connect_client.py: enroll/poll (signed PoP)/rotate against
the control plane, writes connect.json (0600) on claim;
status()/disconnect(); control-plane URL from --cloud / SERVERKIT_CLOUD_URL / https://app.serverkit.ai
- cli.py: 'connect' group (pair, status, disconnect)
- app/api/connect.py + core_blueprints.py: GET /api/v1/connect/status (JWT)
- tests/test_core_blueprint_registry.py: registry guard constants for the
new blueprint
A panel host filled to 100% (0 bytes free) purely from routine
`serverkit update` runs, and recovering it meant hand-running du and
sqlite. `serverkit disk` measures every reclaimable item and offers them
for selection: numbers, "safe", or "all".
Upgrade snapshots are grouped by timestamp and offered individually with
age and size, since one update writes both a tree backup and a database
copy under one stamp -- half a snapshot is not a restore point. They can
be picked by number, by age (--older-than N, or "older-than N" in the
prompt), or left to the "all-but-newest" default.
Two ordering rules are enforced regardless of selection order:
* filesystem reclaimers run BEFORE the database VACUUM. VACUUM rewrites
the database into a temp file beside it, so it needs free space equal
to the database -- on a full disk it is the one step that cannot go
first. It refuses with a clear message rather than failing halfway.
* `jobs` rows are deleted before `queue_messages`, and any message still
referenced by a surviving job is kept, so jobs.queue_message_id never
dangles.
in_flight messages and pending/failed jobs are never touched. /tmp is
only swept for entries matching mktemp naming, owned by the invoking
user, over 24h old AND holding an unpacked ServerKit tree. Docker prunes
dangling images and build cache only -- never tagged images or volumes.
Sizes are measured rather than estimated wherever measurement is
possible: docker reports its own reclaimed tally (shared layers make
dangling-image sizes overlap), and the journal and package caches report
before/after deltas -- apt-get clean empties the cache before a naive
count could see it, which would otherwise report 0 freed for real work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 10, disposition DROPPED -- not a defect. `serverkit apps
stop/start/logs/delete` were tried on a recorded review and failed because
that surface was never built: the apps group has only `list`. Click already
reports the unknown subcommand clearly, so nothing is broken; what was missing
is any hint about where those verbs do live.
The group help now says it outright, at the exact point of contact. It also
names the trap that most likely caused the confusion: `serverkit services`
*does* start and restart things, but those are system services like nginx and
docker, not applications.
Building CLI app lifecycle would be a feature rather than a fix, so it is not
in this plan's scope.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
status, doctor, list-users, list-apps, list-servers, db-status,
deployment-status, services list, apps list, and manifest plan gain a
--json flag emitting machine-readable output, so the CLI can feed
scripts, cron checks, and external monitors. Table-only extras refuse
the combination loudly (--all, --repair).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Route manifest-created apps to a fleet server via top-level/per-service server:
(sets Application.server_id; unknown server = plan issue). Clone the
manifest-resolved config (env + per-preview overrides + domain) into non-WP PR
previews via ManifestPreviewService. Add 'serverkit manifest plan|apply|diff'
CLI verbs against the manifests API.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
serverkit status / services / apps / doctor [--repair] / repair /
update / login-url / support-bundle. Break-glass auth mints a 10-minute
admin JWT in-process (audit-logged) — root on the box already implies
full control. The support bundle exports a secret-scrubbed zip of
versions, service states, setting shapes and recent job failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Introduce a deployment jobs system to support cross-server template installations and remote execution. Adds DeploymentJob and DeploymentJobLog models, DeploymentJobService for job lifecycle, and DeploymentPlanRunner to execute plans locally or via agents. TemplateService gains build_install_plan and related helpers to produce executable plans; Template installs can run async (202) or synchronously. Backend API/CLI updated: new deployment-jobs endpoints, apps API uses RemoteDockerService for remote compose actions, and CLI commands (deploy-template, deployment-status, list-servers) were added. Agent and config changes: file access validation with allowed_paths, file write create_dirs option, default allowed paths, and install script/systemd read-write paths updated to permit /var/lib/serverkit and /var/serverkit. Migration file for deployment_jobs schema included.
When creating an admin via the backend CLI, mark setup as complete by calling SettingsService.complete_setup so the web UI won't show the setup wizard. In the frontend, add a sign-in flow to SetupStepAccount when registration is disabled (e.g. an admin was created via CLI): pull login and registrationEnabled from Auth, implement handleLogin, show an informational banner, and keep the original register flow as handleRegister. In AuthContext, make checkSetupStatus retry a few times (with a short delay) if the backend isn't ready, and on exhausted retries assume a fresh install (needsSetup and registrationEnabled = true) to avoid locking out users while the backend initializes.
Add full SSO (Google, GitHub, generic OIDC, SAML) support and integrate Alembic-based DB migrations. Introduces OAuthIdentity model, sso_service, SSO API blueprint (authorize/callback/link/unlink/admin/config/test), and settings to enable/configure providers and enforce SSO-only logins. Integrates Flask-Migrate and a MigrationService with API endpoints for status/backup/apply/history and includes Alembic scaffolding (migrations/versions/001_baseline.py). Update app init to register SSO and migrations blueprints and initialize Migrate; replace fragile auto-column hack with MigrationService.prepare/check. Update User model (nullable password_hash, auth_provider, has_password/to_dict fields), add AuditLog SSO actions, add frontend SSO components/pages and a migration UI, and update ROADMAP and requirements. Notes: token encryption uses Fernet derived from SECRET_KEY; migrations may add the new auth_provider column at startup.
Introduces an _auto_migrate_columns function in app initialization to automatically add missing columns to existing tables. Refactors the CLI migrate_db command to use a unified expected columns list and checks both 'applications' and 'wordpress_sites' tables for missing columns, improving consistency and reducing manual migration steps.
Expanded the cleanup and factory reset commands to remove orphaned Docker containers, networks, and volumes while preserving ServerKit infrastructure. Improved the list-apps command to optionally display all Docker containers with a new --all flag, and updated documentation and CLI help to reflect these changes.
Port variables in templates are now hidden and always auto-generated, never using user-supplied values. The TemplateService's port assignment checks both database and Docker for used ports, and removes obsolete Docker Compose version fields. Adds new CLI commands for cleaning up all applications, performing a factory reset, and listing applications. Minor update to AuthContext to mark setup as complete after registration.
Introduces a new 'migrate_db' command to the CLI that checks for and adds missing columns to the 'applications' table. This helps keep the database schema up to date without requiring manual intervention or full migrations.
CLI commands failed with "unable to open database file" because
environment variables weren't loaded from .env before create_app().
Now checks multiple .env locations:
- Same directory as cli.py
- Parent directory
- /opt/serverkit/.env (production)
- /opt/serverkit/backend/.env
Add --debug flag to show which env file was loaded.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces a comprehensive Bash CLI tool (`serverkit`) for managing ServerKit services, users, and database operations. Adds a quick-install script (`install.sh`) for Ubuntu, a sample environment file (`.env.example`), and a Python CLI backend (`backend/cli.py`) for administrative tasks. Updates Docker Compose for production readiness, enhances deployment documentation, and adds Click to backend requirements. Also updates references to the correct GitHub repository.
Juan Denis