Block stopping/restarting/removing ServerKit's own Docker containers and surface that in the UI. Backend: add DockerService helpers (is_protected_name / is_protected_container), route guard (_reject_if_protected) to reject destructive actions, annotate list_containers with a protected flag, and update cleanup logic. Add tests (backend/tests/test_docker_protection.py). Frontend: hide/replace lifecycle controls for protected containers (lock badge + styles), add Organization tab group (ORG_TABS) and migrate Projects/Shared Variables/Workspaces into a TabGroupLayout with topbar action hooks; update related styles and sidebar item matching.
A variable can now land on one compose service instead of all of them. Both
EnvironmentVariable and SharedVariable gain an optional target_service
(NULL = all services), threaded through the shared resolver as an inclusion
filter (a var applies to a service when it targets all OR that service).
EnvService.get_effective_env_for_services(app, services) computes each service's
merged env (all-services vars + that-service vars, local overriding shared), and
ComposeEnvService builds the override's per-service environment from it — so a
var targeted at "db" only appears under the db service, untargeted vars appear
everywhere. The single-container docker-run path is unchanged (one container
gets the full effective env regardless of target).
API: env-var create/update and shared-variable create/update accept
target_service; GET /apps/<id>/compose-services lists a compose app's services.
UI: the Environment tab shows an "Applies to" selector (gated to compose apps via
the service list) with a per-row target chip; the shared variable editor gains a
target-service field.
Migration 036 adds the two columns; single Alembic head; chain applies from
empty. Proving tests cover per-service distribution + the resolver helper. Full
backend suite: 949 passed, 14 skipped; frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Single-container apps already get the effective env via docker run -e. Compose
apps get container env from the compose file's environment:, so this wires them
the same way through a managed override compose file.
ComposeEnvService.refresh_for_project writes docker-compose.serverkit.yml next to
a managed app's base compose, adding an environment: overlay to every service
built from EnvService.get_effective_env (shared variable groups under the app's
own local env vars). Compose merges the override on top of the base, so the
effective env wins on collisions — consistent with the docker-run path. Dollar
signs are escaped ($ -> $$) so values survive compose interpolation intact.
DockerService.compose_up now includes the override as a second -f (via
_compose_cmd_with_overlay); `up -d` recreates changed containers so updated env
takes effect on the next deploy. It's non-destructive (ServerKit only owns the
override; the base compose is never edited), best-effort/guarded (a build failure
falls back to the plain base command and never blocks a deploy), regenerated each
up, and removed when the app has no effective env. Non-app dirs (e.g. proxy
stacks) and env-less apps are left untouched.
Proving test test_compose_env_overlay.py (6 cases): overlay injects shared+local
into every service with local winning collisions, $ escaping, stale-override
cleanup, non-app dirs untouched, and compose_up running with -f base -f override.
Full backend suite: 947 passed, 14 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Shared variable groups were resolved/displayed but never reached the container.
EnvService.get_effective_env(app) now merges them in: shared groups
(workspace < project < environment < direct) form the base layer and the app's
own local env vars override them, so a key set in both yields the local value —
matching the "Set locally — local value applies" UI hint. Secrets are decrypted
for injection; shared resolution is best-effort and never blocks a deploy.
The docker deploy path (_deploy_docker) now injects this merged env via
run_container, which also fixes a latent bug there (it called .get('success')
on get_env_vars()'s list result, so env was effectively never applied). Updated
the shared-variables panel note to say shared vars now apply at deploy with
local taking precedence.
Proving test test_effective_env.py (6 cases) covers the merge precedence, the
best-effort fallback, and that _deploy_docker hands run_container the merged env.
Full backend suite: 941 passed, 14 skipped; frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tightens the seams the recent features introduced:
- Secret precedence/conflict: the embedded shared-variables panel now resolves
via the hierarchical endpoint, shows a per-variable source-scope badge +
precedence legend, and flags keys that are also set locally with a
"Set locally — local value applies" badge (shared groups are a facade and are
not injected into the container; the app's own env var is authoritative).
- Split inventory: apps now expose project_name/environment_name; the Services
list shows a project/environment badge (or "Unassigned") and a "Move to
project" bulk action (POST /apps/move-to-project, workspace-validated,
null-unassigns).
- Fleet proxy actionability: fleet_overview rows carry a {level,text}
recommendation (mismatch warning / idle stack / empty host / aligned),
rendered as a per-row pill.
- Onboarding wizard: "Step X of 5" + progress bar + rough ETA, and a
collapsible step-history trail so returning users get a clean summary.
- Config diff: a plain-language summary ("3 environment variables and the image
tag (app:1.2.1 -> app:1.1.9) changed") above the technical diff and echoed in
the restore confirmation.
- Container status: an always-visible "N issues" / "healthy/total" badge so a
degraded sub-container isn't hidden behind hover.
- Sidebar: Projects, Shared Variables and Workspaces collapse under a single
"Organization" group.
No schema change (project/environment names are derived). Backend suite: 935
passed, 14 skipped; frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both reverse proxies bind 80/443, so pointing the same domain at host Nginx and
a Dockerized proxy stack is an operational footgun. This tags each app with the
plane it expects and surfaces disagreements instead of leaving them implicit.
- Each app carries an `ingress_plane` ('nginx' | 'proxy_stack'); NULL reads as
the host-Nginx default. New pure helpers in app/utils/ingress.py: only
container-based services are proxy-stack eligible, and a requested proxy_stack
is forced back to nginx for PHP/WordPress/static/Python apps. to_dict() also
exposes `ingress_proxy_eligible`.
- ProxyStackService.ingress_audit(server) reports which of a server's apps
disagree with its active proxy mode; fleet_overview now includes per-server
app_count + mismatch_count. New GET /servers/<id>/proxy/ingress-audit.
- Create wizard offers the proxy stack only for Docker/Compose services and
keeps the default on Nginx; app detail shows an Ingress badge; the fleet proxy
dashboard flags mismatch counts; the server proxy panel warns and lists the
specific mismatched apps with reasons.
Migration 035 adds applications.ingress_plane. Backend suite green (incl. 8 new
ingress tests); frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Server onboarding completion (C8): a pure pre-flight compatibility check
(arch/distro/kernel/cpu/memory) gating validate; real-but-guarded
distro-aware install_prerequisites/install_docker/pair_agent steps that
no-op safely on dev/Windows and never raise; audit + best-effort
notifications on each transition; and a live auto-scrolling log trail +
elapsed timer in the onboarding wizard.
- Fleet-wide proxy dashboard (C6): ProxyStackService.fleet_overview aggregates
every server's proxy posture (configured stack vs host Nginx default) behind
GET /api/v1/servers/proxy/overview, surfaced as a new Fleet Proxy page in the
Servers tab group with summary tiles + click-through to each server's proxy.
- Polymorphic shared resources depth (C9): hierarchical variable resolution
(workspace < project < environment < direct attachment < resource) with
source provenance and {{group.KEY}} interpolation, an audit trail on every
tag/group/variable/attachment mutation, GET /api/v1/shared/resolved/
hierarchical, and the reusable Tags/Environment-Variables panels embedded on
the application and server detail pages.
- Docs & coverage: docs/ENHANCEMENTS.md (brand-neutral guide to all ten
capabilities + verified endpoints), README/CHANGELOG updates, and a
cross-cutting integration test asserting the new endpoints are wired.
Full backend suite: 924 passed, 14 skipped; frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Project / Environment hierarchy (C2): Workspace -> Project -> Environment
org model. New Project/Environment models + CRUD APIs, nullable
applications.project_id/environment_id (opt-in, no destructive backfill),
ProjectService.ensure_default, a Projects list + ProjectDetail (env tabs),
a project/environment selector in the create wizard, and sidebar nav.
- Polymorphic shared resources facade (C9): ResourceTag + SharedVariableGroup
/SharedVariable/attachment models keyed by (resource_type, resource_id),
reusing the env-var Fernet encryption; SharedResourceService resolves merged
variables (most-recent attachment wins) with secrets masked. /api/v1/shared
endpoints + reusable TagsPanel/EnvironmentVariablesPanel + a Shared Variables
management page. Existing per-resource tables untouched (facade first).
- PR preview environments (C3): ApplicationPreview + settings models, a
PreviewService with pure domain-templating + reconcile (open PRs vs active
previews) cores and best-effort provisioning (incl. guarded WordPress
file+DB clone), a pull_request webhook endpoint, preview.* jobs, and a
Previews tab on the app detail page.
- Per-server managed proxy stack (C6): opt-in Dockerized Traefik/Caddy as a
Compose stack (host Nginx stays default and superior for PHP/WordPress).
ProxyStack model + ProxyStackService with pure compose/config generators and
versioned config backups, /api/v1/servers/<id>/proxy endpoints, and a Proxy
tab on the server detail page.
Migrations 031-034 add the new tables/columns; the full chain applies cleanly
from an empty DB. Full backend suite: 871 passed, 14 skipped; frontend builds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Build packs / zero-Dockerfile deploys (C1): new BuildpackService detects the
stack from a cloned repo (Node/Python/Go/PHP/Ruby/Rust/static), produces a
transparent build plan and a generated Dockerfile, and persists the plan +
overrides on the Application. POST /api/v1/buildpacks/{detect,generate}; a
BuildpackPreview shows the detected plan + editable overrides + Dockerfile in
the create wizard and the app Build tab. Builds on the existing build_method
plumbing instead of duplicating it.
- Deployment config snapshots + diff engine (C4): ConfigurationService captures
an immutable, hashed snapshot of resolved config (env keys + masked values,
domains, image/tag, build method, volumes) at every deploy and rollback; new
DeploymentSnapshot model + /api/v1/apps/<id>/snapshots[/diff|/restore]. A
DeploymentTimeline + ConfigDiffModal surface "what changed" and one-click
restore in the app Deploy tab. Secret values are never serialized.
- Template catalog certification (C5 cont.): a whole-catalog validation test
proves all 105 declarative templates conform to the documented schema; fixed
the stale index.json (was 63/105 entries) to list every bundled template.
Migration 030 adds the buildpack columns + deployment_snapshots table. Full
backend suite: 820 passed, 14 skipped; frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four foundational enhancements toward the deploy-code developer experience:
- Container status aggregator (C7): new ContainerStatusService applies a
deterministic priority hierarchy (degraded > restarting > running:unhealthy >
starting > running:healthy > exited > unknown) over an app's containers,
cached with a short TTL. Exposed at /api/v1/status and pushed over a new
container_status Socket.IO channel; Applications page now renders an
aggregated ContainerStatusPill with sub-status tooltip.
- API token scopes + sensitive data masking (C10): require_scope() decorator
enforces fine-grained scopes for API-key requests (pass-through for JWT),
a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable
sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains
a grouped ApiKeyScopesModal picker and per-key scope pills.
- Server onboarding state machine (C8): new onboarding lifecycle
(pending -> validating -> installing_prerequisites -> installing_docker ->
pairing_agent -> ready/failed) with ServerOnboardingService, a
server_onboarding_logs table, servers.onboarding_* columns, REST endpoints,
and an OnboardingWizard surfaced on the server detail overview.
- Declarative template catalog (C5): documented catalog schema and
magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*)
layered onto the existing YAML template loader without changing behavior for
templates that don't use them; GET /api/v1/templates/catalog/schema.
Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend
builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce a ServerKit-managed per-site Fail2ban layer to protect WordPress login endpoints. Adds Fail2banJailService (write-side) to generate filter and per-site jail configs, enable/disable jails, reload fail2ban, and unban IPs (Linux-only, best-effort, ownership-guarded). Wire it into WordPress service lifecycle (enable on site create, remove on delete) and WpSecurityService wrapper methods. Expose API endpoints for status, toggle and unban in backend API. Add unit tests for Fail2banJailService (config rendering, naming/safety, lifecycle, graceful degradation). Frontend: Security tab loads brute-force status, can toggle protection, shows ban counts and unban action; API client and styles updated. NginxService gains canonical per-site log/error path helpers. Docs updated to reflect shipped brute-force jail feature.
Validates all 105 bundled templates parse + validate via the real TemplateService, ids match filenames, and every template has an icon; the 42 roadmap templates additionally use inline base64 icons, list-shaped vars, and are brand-neutral; and build_repo_index() covers the whole catalog. 149 cases, all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- /domains is one table: app-linked domains + live zones from connected DNS
providers (Cloudflare, ...) with provider badges. New GET /dns/portfolio
(aggregates provider zones, marks which are adopted) and idempotent
POST /dns/adopt (lazy-materialize a managed zone on demand).
- Every row opens one drawer = the unified DNS records surface
(components/domains/DomainDnsPanel): live Cloudflare records via
GET /dns/provider-records (no adopt needed), type badges, proxied=>HTTPS hint,
inline Add record, deep-links to Cloudflare ops / full DNS page. Fixed-layout
table (no horizontal scroll); actions moved into the drawer header.
- Registration expiry/registrar from Cloudflare Registrar + lazy RDAP fallback
(GET /dns/registration), persisted/cached in the new domain_registrations
table and merged into the portfolio so it survives a refresh. New
frontend/src/utils/expiry.js formatExpiry() date util.
- Connections moved to admin-only (Settings Admin group + isAdmin gate;
@admin_required on /api/v1/connections); CF connection card links to Domains.
- Cloudflare client: expose each zone's account_id; add list_registrar_domains.
- Tests: backend/tests/test_dns_portfolio.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Storage tab listing and managing Cloudflare's account-scoped storage
primitives — R2 buckets, KV namespaces, and D1 databases. Management only
(list/create/delete); the object/key/row data planes are out of scope.
- CloudflareClient: r2/buckets, storage/kv/namespaces, d1/database list+create
+delete.
- CloudflareService: combined list_storage (each product fetched independently so
a missing token scope degrades to a per-product error, not a dead tab),
validated create/delete for each. R2 bucket names validated S3-style.
- /api/v1/cloudflare/zones/:id/storage* routes (read any user; writes admin).
- StoragePanel (three generic create/list/delete sections; R2 notes its
S3-compatibility for future backup targeting); api methods; SCSS.
No new tables — wiring an R2 bucket as an S3 backup backend (minting scoped R2
keys) is a deliberate follow-up, noted in the UI.
Tests: aggregate listing + per-product error degradation, R2 name validation,
create/delete, KV/D1 required-field guards (6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Tunnels tab for creating and managing Cloudflare Tunnels (cloudflared) —
exposing a local/private service through Cloudflare's edge with no public IP.
Clearly distinct from ServerKit's WireGuard remote-access tunnels (always
labelled "Cloudflare Tunnel" in the UI). Account-scoped; account resolved from
the zone.
- CloudflareTunnel model: records the connector token (encrypted at rest) and
owning connection per (account, tunnel_id).
- CloudflareClient: cfd_tunnel create (config_src=cloudflare) / list
(is_deleted=false) / delete / token / connections / configurations get+put.
- CloudflareService: create (returns + stores the one-time token, yields the
cloudflared install command), list (managed-flagged), delete, fetch install,
and public-hostname routing — sets the ingress config (always re-appends the
required http_status:404 catch-all) and best-effort upserts the proxied CNAME
hostname→<id>.cfargotunnel.com via the shared ownership-guarded DNS path.
- /api/v1/cloudflare/zones/:id/tunnels* routes (read any user; writes admin).
- TunnelsPanel (create form + one-time install box w/ copy, tunnel list with
per-tunnel install reveal and public-hostname add/remove); api methods; SCSS.
Tests: list is_deleted param, create config_src, name required, token stored
encrypted + install string, managed-flagging, delete prunes local row, ingress
built with catch-all last (7).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Workers tab to the Cloudflare Zone Settings page for deploying edge
scripts and managing Worker routes. Workers are account-scoped, so the owning
account is resolved from the zone — the whole feature reuses the existing
Cloudflare DNS connection (no new credential).
- CloudflareWorker model: records the source ServerKit uploaded per (account,
name) so a worker can be tracked and re-deployed (created via create_all).
- CloudflareClient: zone→account lookup, list/delete scripts, and a stable
multipart PUT /scripts/{name} module upload (the still-beta resource-oriented
Workers API is deliberately avoided per the roadmap's fallback note), plus
zone Worker routes list/add/delete.
- CloudflareService: list (live scripts flagged when ServerKit-managed + routes),
deploy (validated lowercase name, records source, optional route attach),
delete, route add/delete.
- /api/v1/cloudflare/zones/:id/workers* routes (read any user; writes admin).
- WorkersPanel component (deploy form with a starter script, worker list, routes
list); api methods; SCSS.
Tests: multipart upload shape, name validation, deploy records source, list
managed-flagging, delete prunes the local row (5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a WAF tab to the Cloudflare Zone Settings page for managing custom firewall
rules (the http_request_firewall_custom phase entry-point ruleset).
- CloudflareClient: ruleset list/get + phase-entrypoint create (PUT) + per-rule
POST/PATCH/DELETE.
- CloudflareService: list rules (empty when no custom ruleset yet, distinguished
from an auth error by listing rulesets first), add rule (creates the ruleset on
first use), update/delete, and three one-click presets — lock WordPress admin
to an IP, block common exploit paths, challenge suspicious bots. The IP in the
lock preset is validated with ipaddress so it can't inject expression syntax;
rule actions are constrained to a safe terminal subset.
- /api/v1/cloudflare/zones/:id/waf/* routes (list any user; add/preset/update/
delete admin).
- WafPanel component (preset picker + custom-rule form + active-rules list with
enable toggle and delete); api methods; SCSS.
Tests: ruleset discovery, empty vs auth-error, create-on-first-rule vs append,
action validation, preset IP validation + safe expression, update/delete (10).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Cloudflare cache management to the Zone Settings page. Purge everything (with
a confirm) or purge specific URLs (one per line, capped at 30 to match Free/Pro
limits); hosts/prefixes/tags are accepted by the API for Enterprise plans and
their plan errors are surfaced verbatim.
- CloudflareClient.purge_cache(zone_id, payload).
- CloudflareService.purge_cache: validates intent (raises on empty), caps files
at 30, normalizes the payload.
- POST /api/v1/cloudflare/zones/:id/purge-cache (admin).
- Actions tab gains a "Purge cache" card (purge-everything button + URL list);
api method; SCSS.
Tests: client payload, purge-everything, 30-file cap, empty-raises, provider
error surfaced (5 tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Cloudflare Zone Settings surface on top of the existing Cloudflare DNS
connection — SSL/TLS, Speed, Caching and Security toggles plus a one-click
"recommended hardening" preset. Reuses the connected DNSProviderConfig
credential and shared CloudflareClient, so auth/encryption-at-rest are not
re-implemented.
Backend:
- CloudflareClient: generic v4 request() helper + zone-settings get/update
(envelope normalized to {success, error?, result?}).
- CloudflareService: resolve zone+credential by ServerKit DNS zone id, curated
setting groups (UI metadata), get/update setting, apply recommended preset
with per-setting reporting.
- /api/v1/cloudflare blueprint: settings GET / GET-one / PATCH + apply-preset
(reads any user, writes admin; mutations captured by the audit fallback).
Frontend:
- CloudflareZoneSettings page (tabbed SSL/TLS · Speed · Caching · Security ·
Actions) with declarative toggle/select/HSTS controls and optimistic save.
- Route /cloudflare/zones/:zoneId, "Cloudflare" link from Cloudflare-managed
DNS zones, api module, SCSS.
Tests: client envelope/zone-settings + service resolution guards, indexing,
and preset reporting (11 tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#12 SiteDomainService.give_subdomain(app, label): one-click publish an app at
<slug>.<base> — primary Domain row + nginx vhost + (per-site mode) an auto-created,
guarded/logged A record. Endpoints: POST /domains/give-subdomain and
GET /domains/suggest-subdomain.
#14 GET /api/v1/dns/managed: every record ServerKit owns across all provider zones,
enriched with the app that triggered each (from the ownership ledger).
#13 inline propagation reuses the existing /dns/propagation/<domain> check.
Tests: test_dns_give_subdomain.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#9 sites_dns_mode setting (wildcard | per-site) via SiteDomainService.dns_mode();
surfaced in SitesHttpsService.status().
#10 One-click wildcard already exists (SitesHttpsService.setup creates *.<base> +
issues the wildcard cert); it now also pins the mode to wildcard, and the wildcard
A record flows through the Phase 1 guarded/logged path.
#11 SiteDomainService.ensure_site_dns(): in per-site mode a new managed site gets
its own A record auto-created via a connected provider (ownership-guarded + logged),
wired into WordPressService._provision_routing. Wildcard mode stays a no-op.
Tests: test_dns_subdomain_modes.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#6 dns_changes audit log (migration 027) — every record write ServerKit sends to a
connected provider is recorded at the write choke point (guarded_upsert /
guarded_delete): create/update/delete, source, result (ok|error|conflict|skipped),
and the value set. Powers the "Changes to your Cloudflare" activity feed.
GET /api/v1/dns/changes lists it (filter by config_id / zone / result).
#8 A real write failure now surfaces an admin notice via the Notification Bus
(dns.sync_failed) instead of being swallowed to the log. Best-effort — surfacing
never breaks the write it describes.
Tests: test_dns_change_log.py. The frontend activity/mirror UI lands separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#3 managed_dns_records ledger (migration 026) — the single source of truth for
records ServerKit created in a connected provider zone, written by every write
path (zones page, DDNS, WordPress auto-DNS, email).
#4 DnsOwnershipService.guarded_upsert/guarded_delete: automatic paths refuse to
overwrite or delete a record ServerKit didn't create (the user's own records),
while the explicit Zones page adopts a matching record and takes ownership.
Wired into both CF paths (DNSZoneService._cloudflare_sync and
DNSProviderService._cloudflare_set_record/_delete_record). ensure_a_record now
surfaces a 'foreign_record' reason instead of clobbering.
#5 Live zone mirror: CloudflareClient.list_records +
DNSZoneService.list_provider_records classify every live record as serverkit vs
external; GET /api/v1/dns/<id>/mirror.
Tests: test_dns_ownership.py. Existing set_record mocks updated for the new
source kwarg.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 0 of the DNS roadmap.
#2 Shared Cloudflare client (app/services/dns/): DnsCredential + DnsRecordSpec +
CloudflareClient. Both DNSProviderService and DNSZoneService delegate to it, so
auth (scoped token / global key), the CAA structured-data wire format, MX/SRV
priority parsing, and idempotent upsert live in one place. Fixes dup-on-recreate,
silent update/delete no-op, and missing proxied/priority on the provider path.
#1 Single credential source: new FK dns_zones.dns_provider_config_id (migration
025) + DNSZoneService._resolve_credential (linked -> auto-discover-by-domain ->
legacy inline-token). create_zone accepts a connection id; link_legacy_zones()
converges inline tokens onto encrypted connections at boot. The /dns create-zone
modal now picks a connection instead of pasting a token. DDNS rides it for free.
Tests: test_dns_cloudflare_client.py, test_dns_credential_resolution.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Telemetry: ensure sanitized payloads bigger than the 32KiB threshold are replaced with a {_truncated: True} marker before storage, and add a unit test covering that behavior.
Backend API: adjust imports in secrets_webhooks to use WorkspaceService and include User model.
Frontend: make Sidebar resilient to malformed active_workspace JSON in localStorage (try/catch), and ensure socketService.disconnect() is called when user becomes unauthenticated in NotificationsProvider to clear notifications and unread count.
Files: backend/services/telemetry_service.py, backend/tests/test_telemetry.py, backend/app/api/secrets_webhooks.py, frontend/src/components/Sidebar.jsx, frontend/src/contexts/NotificationsContext.jsx
WorkflowEventBus.emit no longer spawns a per-event daemon thread to match +
trigger workflows. It now enqueues a single 'workflow.dispatch' job; the new
WorkflowEventBus.dispatch_event handler does the matching (active event-trigger
workflows whose eventType matches, honoring the 60s cooldown) and enqueues a
workflow.execute job per match. emit is a non-blocking enqueue, best-effort
(never raises into the caller), and the whole event -> dispatch -> execute path
now flows through the bus (observable, persistent across restart). Removes the
last ad-hoc thread on the workflow path.
Registered alongside workflow.execute in WorkflowEngine.register_jobs.
tests: dispatch handler registered; emit enqueues a dispatch job (not a thread);
dispatch fans out only to the matching event workflow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire scheduled backups onto the unified job system and retire the orphaned
backup scheduler thread.
- BackupService.start_scheduler/_scheduler_loop were defined but NEVER called,
so scheduled backups never actually ran. Replace with check_backup_schedules()
(enqueues a backup.run job per due schedule) driven by a new
builtin.backup_scheduler tick, plus a run_backup_job handler that executes one
schedule via the existing _run_scheduled_backup. max_attempts=1 (backups
aren't idempotent); a failed run marks the job failed. This activates
scheduled backups, but stays gated by the existing backup config.enabled
(default False), so installs without configured backups are unaffected.
- Fix a latent bug in _send_backup_notification: it called
NotificationService.send_all(title=, message=, severity=) but send_all takes
a list of alert dicts — corrected (faithful: still routes to system channels
via the queue-backed bus).
- Monitoring intentionally left as-is: a user-toggled continuous stream whose
alert delivery already goes through the queued send_all. Phase 8 orchestration
is already met — send_all is a non-blocking shim over broadcast_system.
- tests: backup.run register/handler success+failure; scheduler tick enqueues
only due+enabled schedules and is inert when disabled.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate workflow executions onto the unified job system (single-job wrap).
WorkflowEngine is split into _create_execution (validate graph + persist the
running WorkflowExecution row) + run_execution (the synchronous DAG core) +
enqueue_execution (create row, enqueue a 'workflow.execute' job, return the
execution id immediately). execute_workflow stays as the synchronous primitive.
- All four triggers route through enqueue_execution: manual API, webhook,
cron tick (builtin.workflow_schedules), and the WorkflowEventBus. The manual
and webhook endpoints no longer block the HTTP request on the whole DAG; the
cron tick enqueues one job per due workflow instead of running them serially
inside the tick; the event bus thread now only matches + enqueues.
- max_attempts=1 (workflows deploy/script/notify — not idempotent). A failed
run raises so the unified job is marked failed; the WorkflowExecution row
keeps the detailed per-node status/logs. Graph validation still happens
synchronously at enqueue, so an invalid workflow surfaces immediately.
- tests: register, enqueue-creates-row+job, empty workflow runs to success,
failure path marks the job failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate deployment installs off their ad-hoc daemon thread onto the unified
job system. DeploymentJobService.install_template now enqueues a
'deploy.install' Job instead of spawning a thread via _start_background_job
(retired); the JobConsumer runs it by calling the existing run_job.
- max_attempts=1: installs create containers/files/an app row and are not
idempotent, so no auto-retry (preserves prior single-shot behavior).
- The unified Job shares the DeploymentJob's correlation_id and links back via
owner_type='deployment_job'/owner_id; the DeploymentJob row stays the rich
record (plan/steps/logs/progress). A failed run raises so both rows show failed.
- Bonus: CLI `deploy-template` (no --wait) now persists to the queue and runs
when a consumer is available, instead of dying with the CLI process.
- tests: handler success/failure mapping + install_template enqueues-not-threads.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generalize the Notification Bus pattern (thin {id} message -> fat DB row
-> single consumer) into a Job/WorkUnit layer on top of the Queue Bus, so
background work flows through one pipe instead of many ad-hoc threads.
- app/jobs: Job + ScheduledJob models, handler registry, JobService +
ScheduledJobService, one generic JobConsumer, one JobScheduler ticker,
and a plugin SDK surface (app.plugins_sdk.jobs).
- Migrate the 8 per-domain daemon scheduler threads (auto-sync,
snapshot-retention, workflow, health-check, wp-update, api-background,
pairing-prune, registrar-expiry) to ScheduledJob rows + built-in
handlers that reuse the exact prior check functions: cadence and
per-task de-dup unchanged, only the trigger moved.
- Retire the orphaned in-memory BackgroundJobService; repoint
/api/v1/performance/jobs at JobService.
- Add /api/v1/jobs (list/get/cancel/retry/stats; scheduled list/run/
toggle) as a single pane over all background work.
- tests/test_jobs.py: enqueue->run->succeed, dead-letter->failed,
unknown-kind, cancel, retry, scheduler tick/advance, built-ins.
No Alembic migration (tables created via create_all), avoiding a
multi-head clash with the in-flight telemetry work. Plan lives in
docs/ORCHESTRATION_ROADMAP.md (local/git-ignored).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add SystemEvent model, TelemetryService, /api/v1 telemetry endpoints, and
a /telemetry UI page. Instrument audit, notification, webhook, backup,
deployment, and monitoring paths. Add correlation_id to Notification,
EventDelivery, and DeploymentJob so related events can be grouped.
- 18 backend tests added and passing
- Frontend production build clean
Reusable, non-blocking notification layer on top of the Queue Bus:
notify.send(event, to, data) records a Notification + per-channel
NotificationDelivery, queues them, and a daemon consumer delivers via
pluggable channel adapters (in-app, email, chat) inheriting retry,
backoff and dead-letter.
- Jinja2 email system: branded, dark-mode, Outlook-safe base layout +
per-event templates (security/backup/monitoring/welcome/invitation).
The legacy inline f-string email was removed and now renders here.
- In-app center API (inbox/unread-count/mark-read) + per-user Socket.IO push.
- Admin delivery log (stats/filter/retry).
- Email providers (SMTP/SendGrid/Postmark/SES/Mailgun) with encrypted
credentials + test; the email adapter prefers the default provider and
falls back to the legacy SMTP config; surfaced in ConnectionRegistry.
- NotificationService.send_all() is now a non-blocking shim that routes
system alerts through the bus. Migrations 023 + 024.
Tests: backend/tests/test_notification_bus.py (20). Full suite: 391 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a POST /security/clamav/start endpoint and SecurityService.start_clamav() to enable+start the ClamAV daemon (tries clamav-daemon and clamd). Add unit tests that mock ServiceControl to verify success/failure and auth. Wire a startClamAV API call into the frontend and extend the Security Overview: richer posture UI, one-click fixes (install, start service, init integrity, update defs), KPIs, icons, and accessibility improvements. Update related styles and file-manager minor styles. This improves operator workflows by allowing a one-click "Start service" posture fix and provides immediate UI feedback when fixes run.
- Add pluggable queue bus with SQLAlchemy backend (Redis-ready interface)
- Models: QueueGroup, Queue, QueueMessage with retries, visibility timeout,
delayed delivery, priority ordering, and dead-letter handling
- REST API at /api/v1/queue for groups, queues, messages, and stats
- Plugin SDK: from app.plugins_sdk import queue
- Operations UI page with group/queue/message management and send-message form
- Migrate webhook deliveries from raw threading.Thread to the queue bus
- Add webhook consumer with retry/dead-letter support
- Migration 022_queue_bus and backend tests
The Flask middleware emitted HSTS with includeSubDomains+preload whenever not in
debug mode -- even on HTTP-only or Cloudflare-Flexible installs, where preload is
a hard-to-reverse commitment. HTTPS is intentionally optional, so HSTS is now
gated on the operator's recorded SSL choice.
- config._resolve_ssl_mode(): SERVERKIT_SSL_MODE env -> /etc/serverkit/ssl-mode
file -> safe 'insecure' default. Exposed as SSL_MODE / HSTS_ENABLED.
- security middleware sends HSTS only when HSTS_ENABLED and not debug. Behind a
proxy Flask can't distinguish real TLS from a Flexible edge via
X-Forwarded-Proto, so we trust the installer's choice, not the request scheme.
- install.sh records SERVERKIT_SSL_MODE in .env; existing installs are covered by
the /etc/serverkit/ssl-mode file. The nginx edge still emits HSTS in its own
secure server block, so secure installs are unaffected.
Proving tests in tests/test_security_headers.py. Full backend suite: 342 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a cert is issued for a domain managed by a connected DNS provider,
ServerKit now auto-creates a `CAA 0 issue "letsencrypt.org"` record at the
zone apex -- pinning issuance to our CA and clearing the "CAA not enabled"
scanner finding. Degrades to manual instructions when no provider manages the
domain; never fails issuance on a CAA hiccup.
- DNSProviderService.ensure_caa_record(): high-level helper mirroring
ensure_a_record (find managing zone -> upsert apex CAA -> degrade to manual).
- parse_caa_value(): BIND 'flags tag "value"' -> Cloudflare/DO data object.
- Cloudflare set_record now sends the structured `data` object CAA requires
(a flat `content` string is rejected), and matches on tag+value so it does
not clobber a different CA's existing CAA record.
- DigitalOcean set_record sends flags/tag for CAA too.
- DNSZoneService._cloudflare_sync uses structured data for CAA, so the preset
CAA record syncs correctly to Cloudflare-backed zones.
- ssl_service.obtain_certificate calls the hook and surfaces the result.
Proving tests in tests/test_dns_caa.py (7 cases, all green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- backup_crypto.py: Fernet file encrypt/decrypt using the existing
SERVERKIT_ENCRYPTION_KEY (no new deps) — encrypt_file (-> .enc),
decrypt_file, is_encrypted_backup, encryption_available.
- BackupService: an opt-in `encrypt_backups` config flag; _finalize_backup
encrypts each artifact before _auto_upload (so the stored/remote copy is
encrypted) and records encrypted/path/size; restore_application and
restore_database transparently decrypt a .enc backup to a temp file first.
- 9 tests: byte-exact roundtrip, wrong-key failure, availability reflects
env, is_encrypted_backup, and an encrypted backup_files create+restore.
Notes: in-memory (streaming large archives is a follow-up); key loss =
unrecoverable backups (back the key up out of band). Additional remote
targets (WebDAV/Azure/Dropbox/SFTP) and restore drills remain follow-ups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ContainerScalePolicy model (per-app: enabled, service_name, min/max
replicas, cpu high/low %, cooldown, current_replicas).
- ContainerScaleService: evaluate() scales a compose service up/down by one
replica when average CPU crosses the high/low thresholds, within bounds and
outside the cooldown; scale_to() for manual; sweep() for cron. Uses
`docker compose --scale`; local + scale-capable services only.
- apps.py: GET/PUT /<id>/scale-policy, POST /<id>/scale,
POST /<id>/scale/evaluate, POST /scale-sweep (admin).
- 10 tests: bounds clamping, scale up/down, hold-at-max, cooldown, disabled,
unknown-cpu, manual endpoint, auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- GpuService shells out to nvidia-smi for per-GPU stats (utilization, VRAM +
memory %, temperature, power, fan, driver) and GPU compute processes with
best-effort PID->container resolution via cgroup. Coerces [N/A] to null and
degrades to "no GPUs" when nvidia-smi is absent.
- GET /api/v1/gpu/ returns {available, gpus, processes}.
- 6 tests: row parse, N/A coercion, availability, no-GPU info, endpoint, auth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ContainerSleepPolicy model (per-app: enabled, idle_timeout_minutes,
last_activity_at, asleep) + a sleep badge in Application.to_dict.
- ContainerSleepService: sleep/wake an app (compose_down/up or
stop/start_container, local), record_activity, and sweep_idle() which
stops enabled+awake apps idle past their timeout — and never sleeps an app
with no activity baseline.
- apps.py endpoints: GET/PUT /<id>/sleep-policy, POST /<id>/sleep,
POST /<id>/wake, POST /sweep-idle (admin, cron-drivable).
- 9 tests: policy, sleep/wake, docker-failure, remote guard, sweep
selection, no-baseline skip, endpoints, auth.
Follow-ups: traffic-based idle detection (feed record_activity from the nginx
log) and request-triggered wake-on-demand; remote-server sleep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- POST /api/v1/apps/<id>/image-update/apply pulls the newest image for a
compose-managed app, then recreates it (compose_pull + compose_up),
reusing the existing local/remote deploy path (DockerService /
RemoteDockerService) exactly like start_app. Refreshes the image_update
badge afterward.
- Guarded to docker-compose apps; a standalone container would need its full
run spec to recreate, so those get a clear "update manually" message.
- 4 tests: pull-then-recreate ordering, pull-failure propagation, non-compose
guard, auth gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ImageUpdateCheck model + ImageUpdateService: compare an app's local
RepoDigest against the registry's current index digest (via
`docker buildx imagetools inspect`) for the same tag, and flag
update_available when they differ. Shells out to docker like the rest of
the Docker layer; degrades gracefully when the image is local-only or the
registry is unreachable.
- API: admin POST /api/v1/image-updates/applications/<id>/check and
GET /api/v1/image-updates/applications/<id> for the latest result.
- Application.to_dict gains a lightweight image_update badge (mirrors the
existing image_scan badge).
- 7 tests: digest diff/match/unknown, no-image, badge, endpoint, auth gate.
One-click apply (pull + recreate) to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- DdnsHost model: a hostname bound to a DNS zone plus a secret token; the
token is masked in API responses except when first minted.
- DdnsService.update_ip reuses DNSZoneService so a configured provider
(e.g. Cloudflare) is synced automatically; chooses A vs AAAA by IP family;
no-op when the IP is unchanged.
- API: admin CRUD for hosts + a public, token-authenticated /ddns/update
endpoint that a router or cron calls when its IP changes (IP from ?ip= or
the request source). Wrong token -> 401, bad IP -> 400.
- 8 tests covering the service flow and both endpoints.
UI panel to follow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Auto-generate SERVERKIT_ENCRYPTION_KEY in production when missing.
- Detect canonical domain from Host/X-Forwarded-Host; persist via SystemSettings.
- Expand CORS origins dynamically to include the canonical domain.
- Add SystemTab UI to view/save detected panel domain and HTTPS preference.
- Install script writes key, prompts for PANEL_DOMAIN, and sets nginx IP→domain redirect.
- Add backend tests for key bootstrap and domain detection helpers.
The previous commit added a module-level `from sqlalchemy import event` to
register SQLite speed PRAGMAs. But the test-system-utils workflow runs
tests/test_utils_system.py with only `pip install pytest` (no SQLAlchemy), and
pytest still imports conftest.py — so it broke with
ModuleNotFoundError: No module named 'sqlalchemy'.
Wrap the import + listener in try/except ImportError. Those jobs run no
DB-backed test, so skipping the PRAGMA tuning there is harmless; the full
backend job still gets it.
Verified: tests/test_utils_system.py (41 passed) and the agent e2e roundtrip
both pass; conftest now imports with just pytest + stdlib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_command_roundtrip_via_send_command drives agent_registry.send_command
from a background thread while the main thread handles /poll and /result.
Flask-SQLAlchemy serves :memory: from a single shared connection (StaticPool),
so the two threads issue interleaved transactions on one connection and the
background commit intermittently dies with ObjectDeletedError /
PendingRollbackError (it passed some CI runs, failed others).
Point tests at a temp-file SQLite instead: each thread gets its own connection
with SQLite's file locking, which is safe and deterministic. Add
synchronous=OFF / journal_mode=MEMORY PRAGMAs so the file DB stays ~as fast as
in-memory (throwaway data needs no durability). No production code changed.
Verified: the e2e roundtrip passes 22/22 consecutive runs (was flaky); full
suite green (exit 0, 232 passed / 12 skipped locally).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 4 of docs/REMOTE_ACCESS_ROADMAP.md (the buildable/verifiable parts):
- #20 multiple private peers per edge: allocate a per-edge WireGuard listen
port (tunnel_netutil.pick_listen_port) instead of a hardcoded 51820, so one
edge can front many private peers without UDP-port conflicts. (Multiple
services per tunnel already worked since Phase 2.)
- #21 reachability diagnostic: tunnel_netutil.diagnose_reachability classifies
an up-but-never-handshaking tunnel as likely-blocked outbound UDP; surfaced
in the status-refresh response (diagnostic field) and as a hint on the Remote
Access page. The relay data plane itself is deferred.
#22 (client-VPN) and #23 (identity-aware SSO) remain XL milestones, not built.
Verified: 14 backend tests (incl. port-allocator + diagnostic), eslint + build.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
