Comments cited plan sections as "(§G3)" or "(§8 unification)", which
point into the same git-ignored plans the plan-number citations did.
References next to a real document (docs/REDESIGN_MAP.md §3) stay. Code
and runtime strings are unchanged (AST and token comparison).
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
The token was compared with != (timing leak), and an app with no
webhook_token accepted a request that sent none, since None != None is
false. Require both tokens and use hmac.compare_digest.
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.
The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
A fresh panel no longer lets whoever reaches it first become admin:
registering the first account needs a one-time code the installer
prints (`serverkit setup-code` repeats it). SERVERKIT_ADMIN_EMAIL and
SERVERKIT_ADMIN_PASSWORD create the admin unattended instead.
Git apps now deploy what users expect: a repository with a compose
file deploys every service it declares, "Deploy latest" builds the
branch's newest commit, and push webhooks rebuild the containers.
Also fixes branch switching on single-branch clones, uploads of a
lone file, compose v1 port preflight, and CLI commands that ignored
their flags.
The split point was clean: git_service.py L602-881 (Gitea install/
lifecycle/config) and 18 of git_bp's 32 routes (lifecycle + repo browsing
via GiteaAPIService) move into builtin-extensions/serverkit-git/backend/,
mounted back at the same /api/v1/git prefix when the extension is
installed. GiteaAPIService moves wholesale (its only consumer).
The deploy pipeline stays core untouched: clone/pull/deploy/webhook-verify
on GitService (all 5 core importers keep every method they call), the
webhook routes including the unauthenticated signature-verified receiver,
and the deployment routes. NginxService's gitea-config helpers stay core
as a deliberate two-speed seam the extension calls.
Upgrade parity: serverkit-git joins GATED_BUILTIN_SLUGS (auto-install only
when the panel-managed serverkit-gitea application exists) and
run_backend_acquisition now sweeps gated slugs too, so a panel that
installed the frontend-only extension picks the backend up once. The core
repo pickers keep three fail-soft getGitea* aliases that 404 into a clean
'not installed' state.
Proving suite test_git_split.py (7 tests); live smoke: builtin install
mounts all 18 routes. Render smoke green for the Git + Security pages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 75 §G1, the remaining 33 files. Every raw subprocess call site in
backend/app that can go through a door now does. What is left is what
should be:
8 app/utils/system.py the doors themselves
8 database_service mysqldump|gzip, gunzip|mysql, pg_dump|gzip,
gunzip|psql — a pipe is two processes
7 build/docker/db_sync/ftp streaming output, gzip pipes
1 log_service tail -f
2 serverkit-analytics sibling repo; see below
Extensions were migrated in BOTH trees. builtin-extensions/ is the source
and backend/app/plugins/ the live copy, compared by normalised hash;
test_builtin_extension_drift stays green. serverkit-email has both sides
tracked, serverkit-ftp and serverkit-cloud-provision only the source.
serverkit-analytics (2 calls) is deliberately untouched: it has no source
in builtin-extensions/ at all, and backend/app/plugins/serverkit-analytics
is a gitignored materialised copy. Editing it would be a change that
cannot be committed and is overwritten on the next sync — that fix belongs
in the serverkit-analytics repo.
Exit-code semantics are preserved wherever they carry meaning, with a
comment at each site, because they are the thing a mechanical migration
destroys: clamscan 1 = INFECTED, grype 1 = vulnerabilities FOUND, yara
1 = matches, cron_runner 127 = never launched. Those read
result['returncode'], never result['success'].
run_checked gained two things real call sites needed: it yields when the
caller directs the streams itself (writing straight to a log file), rather
than adding capture_output on top and raising ValueError; and text/errors
pass through, so site_importers/ssh keeps decoding with errors='replace'
instead of gaining a UnicodeDecodeError on a bad byte.
A regression this migration introduced, caught by the suite and fixed:
install_profile_service._docker_usable() is deliberately TRI-STATE —
None means the probe could not run, because inside LXC the docker client
is usually present while the daemon never started, and callers "must not
treat a broken probe as proof of absence". Writing it as
run_checked(...)['success'] turned an exec failure into False, i.e. "docker
is not usable" — a fact nobody established. That is the §A violation this
plan exists to remove, reintroduced by the migration meant to prevent it.
test_probe_honesty_doctor caught it. Every other run_checked(...)['success']
caller was then audited; the rest are genuinely two-state, and their
originals also returned False on a missing binary.
Seven more slips of the same shape were caught before running by an AST
check for `x = run_checked(...)` followed by `x.stdout/.stderr/.returncode`
— a dict read as an object, which raises only on the failure path, so
happy-path tests stay green.
api_controller_boundary_baseline.json: pure deletion. Routing api/docker.py
through DockerService removed the last 6 subprocess crossings, so the
`subprocess` category is gone from the API layer entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five services shipped byte-identical save_config copies and as many the
same get_config skeleton — the next fix (an atomic write, a better
error) would have had to land in every one. load_json_config /
save_json_config own the mechanics; the path, the default, and any
behaviour around the write stay per-service.
Byte-compatible with the copies: plain open(..., 'w'), no locking, no
atomic rename — adding those here would be a behaviour change smuggled
in with a consolidation; if they're ever wanted they now land once.
Divergent services (template cache invalidation, storage secret
merging, email_relay's DB backing, build_service's leaner return dict)
keep their own methods. monitoring_service converts separately — it is
mid-edit on another branch of this plan.
An authenticated user of any role could inject arguments into git
subprocesses running as the backend user (root under systemd):
- repo_url was passed unvalidated to git ls-remote/clone, enabling the
ext:: transport (host command execution on git < 2.52 defaults) and
file:// local-repository disclosure (all git versions).
- app_path on POST /deploy/clone was passed without a '--' terminator,
enabling --upload-pack=<cmd> option injection independent of git
version and protocol.*.allow policy.
- branch flowed into 'git fetch origin <branch>' as an unterminated
refspec.
Fix (new app/utils/git_security.py applied at every affected sink):
- repo_url scheme allowlist (https/http/git/ssh + scp-like SSH);
rejects ext::/file://, local paths, leading '-', whitespace.
- '-c protocol.ext.allow=never -c protocol.file.allow=never' pinned on
every remote-touching git call, plus a restrictive GIT_ALLOW_PROTOCOL
in the subprocess environment.
- '--' positional terminator before URL/path/refspec arguments —
co-load-bearing with protocol pinning, which alone does not stop
option injection such as --upload-pack=<cmd>.
- branch/ref validation (leading '-', ref-format metacharacters) for
clone --branch, fetch refspecs and stored deploy configs.
- POST /deploy/clone now confines app_path to the managed apps root,
mirroring how /apps/from-repository derives it server-side.
Adds 31 regression tests; bumps version to 1.7.86.
Reported-by: CaptBoykin
Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
The React-Flow Workflow Builder is replaced by the Automations extension
(tramo). Remove its core Python engine and port its four event emitters to
the panel event bus so tramo workflows can trigger on them via the events
bridge.
- event_service.py: new EVENT_CATALOG entries health.check_failed, git.push,
monitor.high_cpu, monitor.high_memory.
- docker_service.py: drop the app_stopped emit (app.stopped already fires via
the audit trail -- no double-emit).
- environment_health_service/git_service/monitoring_service: emit the new
panel events via EventService instead of WorkflowEventBus.
- Delete app/api/workflows.py, app/services/workflow_engine.py,
app/services/workflow_service.py; drop the blueprint + startup job
registration; neutralize the cron workflow-scheduler handler to a no-op.
- KEEP app/models/workflow.py + its tables (read-only legacy export lives in
the tramo blueprint; table drop is a future cleanup).
- Remove the two workflow-engine test classes from test_jobs.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both webhook receivers now re-read serverkit.yaml after a deploy: unchanged hash
is a no-op, a changed manifest auto-applies its autoDeploy services and flips the
rest to pending with an admin notification. Register a 'manifest' drift check
that projects live state onto the declared surface (in_sync iff declared items
match; repair re-applies). Emit manifest.applied/pending/drifted/error events.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make email handling case-insensitive (queries + model normalization) to prevent duplicate accounts. Add SiteDomainService.panel_origin() helper and use it across services to build public URLs (Gitea, WordPress, Roundcube) and Django ALLOWED_HOSTS. Add optional custom-domain support when creating WordPress sites and propagate domain to Roundcube install. Improve environment domain fallback to prefer panel/base domains over .localhost. Small Dockerfile healthcheck tweak (use 127.0.0.1). Update install/update/build scripts to always (re)create virtualenv locally for release installs. Remove unused frontend CreateSiteModal and expose domain field in WordPress page.
Introduce creating services from Git repositories and related UI changes.
Backend: add /apps/from-repository endpoint to validate input, clone repositories, detect build/app type (via BuildService), configure build and deploy (via GitService), persist Application records, and perform cleanup on failure. Add helper utilities for slugifying, safe repo URLs, path validation, and attaching deploy configs to app responses. Make git clone command branch optional.
Frontend: add NewService page, route (/services/new), sidebar entry and buttons for creating a new repo-based service, API client method createAppFromRepository, and import new styles. Update Services, ServiceDetail, Deployments, and Servers pages with UI/UX improvements (deploy actions, repo display sanitization, new icons, deployment CTA, and extensive servers rail/refactor). Add _new-service.scss and include it in main styles.
Adds cross-server Fleet Monitor endpoints and models, plus broad security and robustness improvements across agent, backend, and tooling.
Key changes:
- New fleet monitor API (endpoints for heatmap, comparison, alerts, thresholds, anomalies, forecast, search, exports, and Prometheus) and service skeleton.
- New DB models for ServerAlertThreshold and MetricAlert to support metric alerts.
- Agent discovery/registration hardening: HMAC-signed discovery requests, signature/timestamp validation, minimal discovery responses, HMAC-based unregister headers, and enforced command timeout cap in agent command handling.
- WebSocket and agent client TLS handling gated by SERVERKIT_INSECURE_TLS env var; added auth rate limiting on agent gateway.
- Docker-compose: parameterize agent image via AGENT_VERSION and avoid running as root by adding docker group access.
- Backend security & policy: tightened CSP script/style sources, request size limit, 2FA pending-token rejection on non-2FA endpoints, registration flow/guards, admin-only SSO admin routes via middleware, rate limits on sensitive endpoints, and logging for auth/2FA events.
- Database service hardening: identifier validation, avoid passing passwords on CLI (use env vars), safer subprocess usage, and docker exec handling improvements.
- CronService: command validation to prevent shell injection and run commands without shell=True; build_service and other subprocess usages updated similarly.
- Misc: .env.example notes clarified for encryption key generation, invitation token generation improved, agent registry timestamp window reduced, various logging and input validations added, frontend FleetMonitor page and styles stubbed in, and Vite/config/style updates.
Overall this commit implements fleet monitoring scaffolding and a set of security, validation, and operational improvements to make agent discovery, registration, command execution, and backend services safer and more robust.
Introduce a full workflow/automation subsystem: new WorkflowEngine with DAG validation/execution (Kahn's algorithm), logic-if branching, variable interpolation, script sandboxing (timeouts, output limits), retry semantics, notification nodes, and an in-process WorkflowEventBus for event-triggered workflows. Backend API updates include webhook trigger and validate endpoints, auto-generation of webhook IDs on create/update, and services emitting events (git push, health check failures, app stopped, monitoring alerts). Add Alembic migration to create workflow-related tables and bump VERSION to 1.4.0. Also update .gitignore, add GitHub issue/PR templates, quick-start script, and various frontend workflow panel/component adjustments to support the new automation features.
Introduces backend/app/paths.py to centralize all key directory paths, allowing overrides via environment variables for local development. Updates all backend services and APIs to use these centralized paths instead of hardcoded values. Adds dev path override examples to backend/.env.example and updates .gitignore to exclude dev-data. Also changes frontend dev server port to 5274 and updates related scripts and CORS settings for consistency.
Introduces Nginx configuration for serving Gitea at the /gitea path, including creation and removal logic in NginxService. GitService now manages and reports the slug-based URL path, and the frontend Git page uses and displays the new /gitea path if available, falling back to port-based URLs for backward compatibility.
Introduces backend API endpoints and service logic for installing, managing, and uninstalling a Gitea-based Git server. Adds a new frontend page for Git server management, updates the sidebar and routing, and includes supporting API methods and styles for the new feature.
Implements backend and frontend support for listing remote Git branches (by app or repo URL) and selecting them during deployment configuration. Adds webhook log viewing for debugging, with backend endpoints and UI in the deploy tab. Updates webhook verification to support GitHub, GitLab, and Bitbucket, and introduces new styles for the deploy tab.
Introduces new API endpoints and services for backups, monitoring, and Git deployment. Adds security middleware, account lockout on failed logins, and improves domain/SSL management. Updates Dockerfiles, deployment configs, and frontend to support new features and UI components.
Juan Denis