Every emit re-validated the socket's saved JWT including exp, and
access tokens last 15 minutes, so metrics, terminals, logs and
notifications were cut a quarter-hour after connecting; clients do
not reconnect after a server-side disconnect. Re-checks now skip exp
only; revocation, auth_version and deactivation still end the socket,
and connecting still needs an unexpired token.
Each was referenced only by its own definition across the panel, its
tests, scripts, builtin extensions and every sibling extension repo
(~350 lines). Imports they alone used go with them.
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
Until now the connect client refused every http and sio stream, so
Open panel in ServerKit Cloud showed a blank page. The client now
bridges them to the loopback listener, the panel serves its page
under the relay's /t/<device>/ prefix, and the SPA prefixes its
router, API and Socket.IO paths from it. The Cloud grant is checked
against Cloud's JWKS before signing in the panel user with the same
email; panel two-factor still applies.
Centralize backup schedule calculations and shared request/form flows, encode query values, and move repository finalization into its service. Add browser CI coverage, repair stale test fixtures, and provide opt-in request and build measurement tools.
GHSA-r4q7-x795-vr4j was one forgotten check, but the audit behind it found
the same shape across the surface: `@jwt_required()` authenticates, it does
not authorize, and ServerKit has no default-deny to catch the difference.
This closes the routes that audit left open — fleet-monitor alert
acknowledge/resolve, recycle-bin restore, nginx test/diff, registrar
sync/test, server-template drift checks, uptime tracking on/off, server ping
and remote compose ps/logs, refresh-capabilities, template
test-db-connection, deploy branches, buildpack detect — plus three inline
gaps: a viewer could widen their own API key's scopes, the demo-deploy flag
was doing double duty as an authorization one, and tagging a shared resource
had no write check at all while attaching one did.
`_ensure_group_mutable` only ever blocked *system* queue groups, so every
queue_bus mutation now also calls `_ensure_group_accessible` (owner-or-admin)
— renaming, draining or deleting another user's group was open to any
authenticated caller.
The rbac policy decorators authenticate through `auth_required()`, so they
replace `@jwt_required()` rather than stack under it; stacking rejects
API-key callers before the policy can authorize them
(tests/test_auth_decorator_boundaries.py enforces this). Where a route then
needed the caller it reads `get_current_user()` — `get_jwt_identity()` raises
for an API-key request. The decorators also carry a `_sk_authz` marker so a
static test can see the decision from the source.
`migrations.get_migration_history` moves to `require_admin_user()` for the
same reason its siblings did: the inline `User.ROLE_ADMIN` check it kept was
already reading two names this module no longer imports, so the route raised
NameError for every caller, admin included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
config.py's parsers move to app/utils/env.py and are imported back; the
three incompatible truthiness variants collapse to one TRUTHY tuple.
Fixes the measured bug: SERVERKIT_ALLOW_PRIVATE_DOWNLOADS=on silently
read false in plugin_service (its tuple lacked 'on'); same for the pip
gate, SERVERKIT_STAGING, and SERVERKIT_DEMO_MODE. Ratchet: no new
os.environ truthiness tuples outside utils/env.py.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
utils/sensitive_data_filter.SENSITIVE_KEY_PARTS is now the union of the
four diverging lists (response filter, audit_service, middleware/audit —
previously the only sink covering totp/otp/csrf/session — telemetry);
all sinks delegate to the shared is_sensitive_key(), and mask_payload()
gives log sinks a scope-free recursive masker. Safe-key exceptions keep
identifier-shaped fields (private_url, certificates, totp_enabled, …)
readable. Ratchet: no SENSITIVE list literals outside the one door.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 76 milestone A, drift/bug row 3. Five API modules had grown a private
_require_admin() with *three mutually incompatible* return contracts:
jobs.py -> returns the user; None means DENIED
dns_cutover.py -> returns None on success; a response tuple means denied
waf.py -> same as dns_cutover
manifests.py -> (user, None) on success, (None, response) on denial
themes.py -> same as manifests
jobs.py's is the exact inverse of dns_cutover.py's, so a guard copied between
the two reads identically at the call site and fails open. Nothing in the code
told you which contract you were looking at.
Replace all five with rbac.require_admin_user(), which raises a typed
PermissionDeniedError the global ApplicationError handler already renders. A
gate that raises has no return contract to get backwards.
Two convergences ride along, both toward the stricter shared behaviour:
- manifests.py gated on `user.role != 'admin'` where the others used
`is_admin`. These are the same predicate today (is_admin is defined as
role == ROLE_ADMIN), so no route changes status, but there is now one
truth source rather than two.
- none of the five checked is_active, so a deactivated admin still passed
them; @admin_required has always checked it. The shared door checks it, so
these 25 call sites now reject deactivated admins like every other
admin route.
The unknown-user case keeps its 403 'Admin access required' rather than
becoming a 401/404: a non-admin caller should not learn from the status code
whether the account behind the token still exists.
Gate: tests/test_jobs.py, test_themes.py, test_waf.py, test_dns_cutover_api.py
(82 passed) and -k manifest (152 passed).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rate-limit buckets, login lockout and audit-log source IPs hand-parsed the
leftmost X-Forwarded-For token — a client-controlled value — letting an
attacker rotate it to dodge per-IP limits or forge audit trails.
- config: add TRUST_PROXY_HEADERS / TRUSTED_PROXY_HOPS (+ _env_bool/_env_int)
- create_app: wrap in gated Werkzeug ProxyFix (rightmost trusted hops) before
the limiter/handlers so every remote_addr consumer sees the real client
- new app/utils/client_ip.py::get_client_ip() — the single trusted seam
- replace all six leftmost-XFF parsers (rate_limit, audit middleware,
api_key_auth, api_analytics, audit_service, ddns) with get_client_ip()
- deploy: root .env.example ships TRUST_PROXY_HEADERS=true (behind nginx);
backend/.env.example documents it off for dev
- tests: test_trusted_client_ip.py (7) incl. real flask-limiter bucket isolation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 1 of the July-5-7 recovery rebuild (plan 38). Backend now boots.
- models: reconstruct DnsCutoverSnapshot (from bytecode) + FleetDoctorResult
(from migration 063 DDL); restore lost columns applications.ports/
bootstrap_done, app_volumes.declared_size, dns_provider_configs.origin_ca_key
- migrations: recreate the 15 missing revisions 056/057/060/062/063/064/065/
067/068/069/070-074, reconciling the chain to a single linear head (074).
add_column revisions guard on live schema so a fresh-DB upgrade reaches head
- rbac: restore _APP_ROLE_RANK + require_workspace_access/require_workspace_role/
app_access_tier/require_app_member (plan 19/29 4-tier authz), recovered from
bytecode; unblocks api/cron.py import
- services: reconstruct cron_run_service.CronRunService (record_run/recent_runs/
stats, edge-triggered failure/recovery transitions) from bytecode
Gate: create_app() imports clean, db.create_all() OK, flask db upgrade on a
fresh SQLite DB reaches head 074 (linear chain).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Single-use, short-TTL, optionally IP-bound login URLs (hash-at-rest,
migration 049, hourly reap job) minted from Settings > Users; plus a
config-gated demo mode that 403s mutating API calls and seeds read-only
credentials on the login page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four foundational enhancements toward the deploy-code developer experience:
- Container status aggregator (C7): new ContainerStatusService applies a
deterministic priority hierarchy (degraded > restarting > running:unhealthy >
starting > running:healthy > exited > unknown) over an app's containers,
cached with a short TTL. Exposed at /api/v1/status and pushed over a new
container_status Socket.IO channel; Applications page now renders an
aggregated ContainerStatusPill with sub-status tooltip.
- API token scopes + sensitive data masking (C10): require_scope() decorator
enforces fine-grained scopes for API-key requests (pass-through for JWT),
a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable
sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains
a grouped ApiKeyScopesModal picker and per-key scope pills.
- Server onboarding state machine (C8): new onboarding lifecycle
(pending -> validating -> installing_prerequisites -> installing_docker ->
pairing_agent -> ready/failed) with ServerOnboardingService, a
server_onboarding_logs table, servers.onboarding_* columns, REST endpoints,
and an OnboardingWizard surfaced on the server detail overview.
- Declarative template catalog (C5): documented catalog schema and
magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*)
layered onto the existing YAML template loader without changing behavior for
templates that don't use them; GET /api/v1/templates/catalog/schema.
Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend
builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Flask middleware emitted HSTS with includeSubDomains+preload whenever not in
debug mode -- even on HTTP-only or Cloudflare-Flexible installs, where preload is
a hard-to-reverse commitment. HTTPS is intentionally optional, so HSTS is now
gated on the operator's recorded SSL choice.
- config._resolve_ssl_mode(): SERVERKIT_SSL_MODE env -> /etc/serverkit/ssl-mode
file -> safe 'insecure' default. Exposed as SSL_MODE / HSTS_ENABLED.
- security middleware sends HSTS only when HSTS_ENABLED and not debug. Behind a
proxy Flask can't distinguish real TLS from a Flexible edge via
X-Forwarded-Proto, so we trust the installer's choice, not the request scheme.
- install.sh records SERVERKIT_SSL_MODE in .env; existing installs are covered by
the /etc/serverkit/ssl-mode file. The nginx edge still emits HSTS in its own
secure server block, so secure installs are unaffected.
Proving tests in tests/test_security_headers.py. Full backend suite: 342 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add secure nginx config with TLS 1.2+, HSTS preload, CSP, and security headers
- Add insecure HTTP fallback config for users who cannot or do not want HTTPS
- Make installer best-effort SSL: try Let's Encrypt, fall back to HTTP without failing
- Preserve secure/insecure SSL choice across updates via /etc/serverkit/ssl-mode
- Support SERVERKIT_SKIP_SSL=1 to disable HTTPS entirely
- Add CAA record to default DNS zone template
- Harden generated per-app nginx configs with HSTS preload, CSP, and security headers
- Align Flask security middleware headers with nginx edge config
Introduce fallback audit logging and enrich audit tooling; add builtin extension discovery/install; and update default launcher ports and related docs/config.
Key changes:
- New middleware app.middleware.audit registers an after-request fallback that records authenticated mutating API requests when routes don't emit explicit audit logs.
- Audit model/action constants extended and AuditService enhanced (request context handling, redaction for sensitive settings, optional commit flag, sets g.audit_logged, emits events).
- Routes/services updated to create explicit audit entries for marketplace and plugin operations (create/update/publish/install/uninstall/enable/disable) and SSO/settings flows switched to use AuditService.
- Added tests for audit logging behavior and redaction (backend/tests/test_audit_log.py).
- Plugin service: added BUILTIN_EXTENSIONS_DIR, list_builtin_extensions and install_builtin_extension; plugin API endpoints to list/install builtin extensions.
- Contribution system extended to support layouts and route layout metadata; frontend wiring updated to surface standalone/custom layouts and extension routes.
- Launcher port defaults changed (backend -> 47927, frontend -> 41921). Updated backend .env.example, config CORS defaults, Dockerfiles and docker-compose to use SERVERKIT_BACKEND_PORT, and many docs/README files and frontend envs to reflect new ports.
- Minor frontend cleanups and new builtin extension example (serverkit-git) plus associated manifest and frontend re-export.
These changes improve observability/security for mutating APIs, enable opt-in builtin plugins, and align local dev/run ports with the launcher defaults.
Adds cross-server Fleet Monitor endpoints and models, plus broad security and robustness improvements across agent, backend, and tooling.
Key changes:
- New fleet monitor API (endpoints for heatmap, comparison, alerts, thresholds, anomalies, forecast, search, exports, and Prometheus) and service skeleton.
- New DB models for ServerAlertThreshold and MetricAlert to support metric alerts.
- Agent discovery/registration hardening: HMAC-signed discovery requests, signature/timestamp validation, minimal discovery responses, HMAC-based unregister headers, and enforced command timeout cap in agent command handling.
- WebSocket and agent client TLS handling gated by SERVERKIT_INSECURE_TLS env var; added auth rate limiting on agent gateway.
- Docker-compose: parameterize agent image via AGENT_VERSION and avoid running as root by adding docker group access.
- Backend security & policy: tightened CSP script/style sources, request size limit, 2FA pending-token rejection on non-2FA endpoints, registration flow/guards, admin-only SSO admin routes via middleware, rate limits on sensitive endpoints, and logging for auth/2FA events.
- Database service hardening: identifier validation, avoid passing passwords on CLI (use env vars), safer subprocess usage, and docker exec handling improvements.
- CronService: command validation to prevent shell injection and run commands without shell=True; build_service and other subprocess usages updated similarly.
- Misc: .env.example notes clarified for encryption key generation, invitation token generation improved, agent registry timestamp window reduced, various logging and input validations added, frontend FleetMonitor page and styles stubbed in, and Vite/config/style updates.
Overall this commit implements fleet monitoring scaffolding and a set of security, validation, and operational improvements to make agent discovery, registration, command execution, and backend services safer and more robust.
Introduce team invitation support and a revamped email subsystem plus admin features. Adds Invitation model and API endpoints (create, list, revoke, resend, validate) and integrates invitation validation into user registration. Replaces old email account model with richer email models (EmailDomain, EmailAccount, EmailAlias, EmailForwardingRule, DNSProviderConfig) and updates models __init__. Refactors email API to manage domains, accounts, aliases, forwarding, DNS providers, spamassassin, Roundcube/Postfix services and queue/log endpoints. Adds several backend services (invitation, DNS provider, DKIM, Dovecot, Postfix, Roundcube, SpamAssassin, permission service) and a DB migration for permissions/invitations. Enhances admin API with permission management and activity dashboard endpoints, extends audit log actions, and adds a permission_required RBAC decorator. Includes frontend additions for settings/invitations UI and miscellaneous config/service updates and tests.
Introduce programmatic API features and observability: add API Key management, API usage analytics, webhook event subscriptions, and OpenAPI docs. New models (ApiKey, ApiUsageLog, ApiUsageSummary, EventSubscription, EventDelivery), services (api_analytics, api_key, event, openapi), and blueprints (/api-keys, /api-analytics, /event-subscriptions, /docs) were added and registered in create_app. Middleware for API key authentication, buffered analytics logging (with a flush thread), and enhanced rate limiting (custom key function and response headers) were implemented; RBAC was updated to accept API key auth via an auth_required decorator. Background threads for hourly aggregation and event retry were added; security CSP updated to allow resources for Swagger UI. Frontend settings UI and styles were added/updated for API keys and webhooks, and requirements were updated.
Introduces backend endpoints and models for admin features including user management, system settings, and audit logging. Adds RBAC middleware, audit log and system settings models, and services for audit and settings. Updates authentication to support setup flow and registration control. On the frontend, adds settings tabs for users and audit logs, a setup page, and updates routing and context to support setup and registration state. Styles and API service are updated to support new features.
Introduces new API endpoints and services for backups, monitoring, and Git deployment. Adds security middleware, account lockout on failed logins, and improves domain/SSL management. Updates Dockerfiles, deployment configs, and frontend to support new features and UI components.
Juan Denis