Plan 73 item 1. SSLService hardcoded /usr/bin/certbot, so the install method
certbot itself recommends -- snap, which lands in /snap/bin -- produced a
FileNotFoundError the panel surfaced as an opaque failure. is_command_available
had the same blind spot: it probed /usr/bin, /usr/sbin and the two local dirs
but never /snap/bin, which is routinely absent from a service's $PATH.
resolve_certbot_bin() now tries the CERTBOT_BIN override, then shutil.which,
then the explicit candidate list including the snap path. Positive results are
cached (an install location does not move mid-process) but negative ones are
not, so installing certbot is picked up without a panel restart;
install_certbot clears the cache outright. CERTBOT_BIN as a class attribute is
gone -- it captured the value at import time, which is exactly why an override
set later never took effect -- replaced by certbot_bin() plus CERTBOT_FALLBACK
so a command built on a host with no certbot still fails cleanly.
The second half is the failure the reviewer actually hit on stream: with
certbot present but its nginx plugin missing, `certbot --nginx` dies deep
inside certbot with "the nginx plugin is not working / binary doesn't exist",
and we passed that through raw. obtain_certificate now preflights -- nginx on
the box, then `certbot plugins` for an enabled `* nginx` -- and returns an
actionable error naming python3-certbot-nginx before certbot is ever invoked
for issuance. A failing plugins probe falls back to the distro package check
rather than blocking issuance on a host where the plugin is clearly present.
AdvancedSSLService's wildcard and multi-domain commands go through the same
resolution instead of the bare name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An authenticated user of any role could inject arguments into git
subprocesses running as the backend user (root under systemd):
- repo_url was passed unvalidated to git ls-remote/clone, enabling the
ext:: transport (host command execution on git < 2.52 defaults) and
file:// local-repository disclosure (all git versions).
- app_path on POST /deploy/clone was passed without a '--' terminator,
enabling --upload-pack=<cmd> option injection independent of git
version and protocol.*.allow policy.
- branch flowed into 'git fetch origin <branch>' as an unterminated
refspec.
Fix (new app/utils/git_security.py applied at every affected sink):
- repo_url scheme allowlist (https/http/git/ssh + scp-like SSH);
rejects ext::/file://, local paths, leading '-', whitespace.
- '-c protocol.ext.allow=never -c protocol.file.allow=never' pinned on
every remote-touching git call, plus a restrictive GIT_ALLOW_PROTOCOL
in the subprocess environment.
- '--' positional terminator before URL/path/refspec arguments —
co-load-bearing with protocol pinning, which alone does not stop
option injection such as --upload-pack=<cmd>.
- branch/ref validation (leading '-', ref-format metacharacters) for
clone --branch, fetch refspecs and stored deploy configs.
- POST /deploy/clone now confines app_path to the managed apps root,
mirroring how /apps/from-repository derives it server-side.
Adds 31 regression tests; bumps version to 1.7.86.
Reported-by: CaptBoykin
Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
Runtime-ESM extensions (starting with serverkit-wordpress) need slot
mounting and error containment from the host; additive, versioned in
lockstep frontend/backend.
`contributions.widgets` mounts a component at a host-chosen slot. That is the
wrong shape for the new dashboard, where the user decides what goes on a board
and where. So this adds `dashboard_widgets` alongside it: a registry of
placeable widget TYPES that surface in the widget library, each instance
positioned, sized and configured by the person looking at it.
The two are easy to confuse, so the docstring, the manifest spec and
EXTENSIONS.md all now say plainly which is which, and a test asserts a
dashboard widget never leaks into the fixed-slot list.
SDK_VERSION 1.0.0 -> 1.1.0 (added surface, older extensions unaffected),
bumped in lock-step across the JS export and the backend mirror.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
210 run_privileged() call sites passed no timeout. With output captured
and no terminal attached, a command that never returns takes its caller
with it — silently and forever. That is exactly how the metadata guard's
iptables probe wedged backend startup; the sudo -n fix removed that one
trigger, but the class of bug was still one hung command away.
run_privileged now applies a default timeout when the caller does not
name one, so the fix lands on all 210 sites at once rather than being
reapplied by hand and forgotten on the 211th. TimeoutExpired is a far
better outcome than a wedged worker, and callers that genuinely need
longer already pass their own; `timeout=None` opts out deliberately.
The four direct read-only probes in this module (dpkg -s, rpm -q,
systemctl is-active/is-enabled) bypass that helper, so they get their own
much tighter ceiling — they answer immediately or not at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
privileged_cmd prepended a bare `sudo`. Nothing here runs attached to a
terminal and run_privileged captures output, so on a non-root host a
password prompt is both invisible and unanswerable — sudo blocks forever
and takes its caller with it. This hung backend startup outright: the
metadata guard's `iptables -C DOCKER-USER` probe sat unanswered and the
server never bound its port.
Emit `sudo -n` so the call fails immediately with a non-zero exit, which
every caller already handles. Production runs as root (_needs_sudo() is
false) which is why this stayed hidden, but any non-root install without
passwordless sudo would hang on boot.
An explicitly-sudo command passed in by a caller is still forwarded
untouched; the regression test pins non-interactive sudo across all four
invocation shapes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rate-limit buckets, login lockout and audit-log source IPs hand-parsed the
leftmost X-Forwarded-For token — a client-controlled value — letting an
attacker rotate it to dodge per-IP limits or forge audit trails.
- config: add TRUST_PROXY_HEADERS / TRUSTED_PROXY_HOPS (+ _env_bool/_env_int)
- create_app: wrap in gated Werkzeug ProxyFix (rightmost trusted hops) before
the limiter/handlers so every remote_addr consumer sees the real client
- new app/utils/client_ip.py::get_client_ip() — the single trusted seam
- replace all six leftmost-XFF parsers (rate_limit, audit middleware,
api_key_auth, api_analytics, audit_service, ddns) with get_client_ip()
- deploy: root .env.example ships TRUST_PROXY_HEADERS=true (behind nginx);
backend/.env.example documents it off for dev
- tests: test_trusted_client_ip.py (7) incl. real flask-limiter bucket isolation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/system/version now returns install_dir (new get_install_dir() in the
shared version resolver: SERVERKIT_INSTALL_DIR override -> own tree ->
/opt/serverkit default; abspath not realpath so blue/green boxes report
the stable symlink). The File Manager re-points its 'Stack' quick-access
shortcut from the response, so custom-directory installs jump to the
real tree; the static default remains the fallback while loading or on
older backends. 2 new resolver tests + install_dir asserted in the
endpoint test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The About page's /system/version (and check-update's current-version
read) hardcoded /opt/serverkit first, duplicated in three places:
- app/utils/version.py is now the single resolver: explicit
SERVERKIT_INSTALL_DIR override -> the running code's own tree ->
/opt/serverkit last (a box moved to a custom dir may still have a
stale tree there); system.py's two inline copies now call it
- the systemd unit template renders SERVERKIT_INSTALL_DIR from the
installer's SERVERKIT_DIR (deliberately not the bare SERVERKIT_DIR
env name, which app/paths.py already uses for the /var/serverkit
data root); flows to existing installs via the updater's re-render
- crypto's .env discovery gains the same install-dir hint
- Docker image now ships VERSION (/app/VERSION) — containers showed
the 1.0.0 fallback because no candidate path existed in the image
- 4 proving tests in backend/tests/test_panel_version.py
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both reverse proxies bind 80/443, so pointing the same domain at host Nginx and
a Dockerized proxy stack is an operational footgun. This tags each app with the
plane it expects and surfaces disagreements instead of leaving them implicit.
- Each app carries an `ingress_plane` ('nginx' | 'proxy_stack'); NULL reads as
the host-Nginx default. New pure helpers in app/utils/ingress.py: only
container-based services are proxy-stack eligible, and a requested proxy_stack
is forced back to nginx for PHP/WordPress/static/Python apps. to_dict() also
exposes `ingress_proxy_eligible`.
- ProxyStackService.ingress_audit(server) reports which of a server's apps
disagree with its active proxy mode; fleet_overview now includes per-server
app_count + mismatch_count. New GET /servers/<id>/proxy/ingress-audit.
- Create wizard offers the proxy stack only for Docker/Compose services and
keeps the default on Nginx; app detail shows an Ingress badge; the fleet proxy
dashboard flags mismatch counts; the server proxy panel warns and lists the
specific mismatched apps with reasons.
Migration 035 adds applications.ingress_plane. Backend suite green (incl. 8 new
ingress tests); frontend builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four foundational enhancements toward the deploy-code developer experience:
- Container status aggregator (C7): new ContainerStatusService applies a
deterministic priority hierarchy (degraded > restarting > running:unhealthy >
starting > running:healthy > exited > unknown) over an app's containers,
cached with a short TTL. Exposed at /api/v1/status and pushed over a new
container_status Socket.IO channel; Applications page now renders an
aggregated ContainerStatusPill with sub-status tooltip.
- API token scopes + sensitive data masking (C10): require_scope() decorator
enforces fine-grained scopes for API-key requests (pass-through for JWT),
a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable
sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains
a grouped ApiKeyScopesModal picker and per-key scope pills.
- Server onboarding state machine (C8): new onboarding lifecycle
(pending -> validating -> installing_prerequisites -> installing_docker ->
pairing_agent -> ready/failed) with ServerOnboardingService, a
server_onboarding_logs table, servers.onboarding_* columns, REST endpoints,
and an OnboardingWizard surfaced on the server detail overview.
- Declarative template catalog (C5): documented catalog schema and
magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*)
layered onto the existing YAML template loader without changing behavior for
templates that don't use them; GET /api/v1/templates/catalog/schema.
Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend
builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add app/utils/formatting.py with format_bytes() and delegate 9 duplicated
human-readable byte formatters across services.
- Import admin_required from app.middleware.rbac in 15 API modules instead of
redefining it locally.
- Extend app/utils/slug.py with validate_app_name() and use it for template
install/validation and apps slug normalization.
- Extend app.utils.slug with unique_slug(default=...) and validate_slug helpers
- Migrate workspace, secret-vault, webhook-gateway, site-domain, upload,
environment-domain, environment-pipeline, marketplace, and status-page
services to use the shared helpers
- Removes ~8 duplicate _slugify/_unique_slug implementations
- Marketplace create_extension now resolves duplicate slugs instead of raising
- Backend tests and frontend production build pass
- backup_crypto.py: Fernet file encrypt/decrypt using the existing
SERVERKIT_ENCRYPTION_KEY (no new deps) — encrypt_file (-> .enc),
decrypt_file, is_encrypted_backup, encryption_available.
- BackupService: an opt-in `encrypt_backups` config flag; _finalize_backup
encrypts each artifact before _auto_upload (so the stored/remote copy is
encrypted) and records encrypted/path/size; restore_application and
restore_database transparently decrypt a .enc backup to a temp file first.
- 9 tests: byte-exact roundtrip, wrong-key failure, availability reflects
env, is_encrypted_backup, and an encrypted backup_files create+restore.
Notes: in-memory (streaming large archives is a follow-up); key loss =
unrecoverable backups (back the key up out of band). Additional remote
targets (WebDAV/Azure/Dropbox/SFTP) and restore drills remain follow-ups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Auto-generate SERVERKIT_ENCRYPTION_KEY in production when missing.
- Detect canonical domain from Host/X-Forwarded-Host; persist via SystemSettings.
- Expand CORS origins dynamically to include the canonical domain.
- Add SystemTab UI to view/save detected panel domain and HTTPS preference.
- Install script writes key, prompts for PANEL_DOMAIN, and sets nginx IP→domain redirect.
- Add backend tests for key bootstrap and domain detection helpers.
Encryption-at-rest (#16/#17) for the two provider stores still in plaintext:
- crypto.py: add decrypt_secret_safe (decrypt with plaintext fallback) + is_encrypted.
- DNS providers: encrypt api_key/api_secret on write; decrypt at point-of-use via
_api_key/_api_secret in the Cloudflare/Route53/DigitalOcean/GoDaddy builders.
- Storage: encrypt secrets in save_config; decrypt in get_config.
- Idempotent encrypt_legacy_secrets() migrations run on startup, encrypting any
pre-existing plaintext in place. Legacy values keep working via the decrypt
fallback, so there is no flag day.
Namecheap registrar:
- RegistrarConnection.config_json column for non-secret extras (username, client_ip).
- RegistrarService: Namecheap XML domains.getList (portfolio + expiry), test, and
MM/DD/YYYY -> ISO date parsing.
- Frontend: namecheap tile is live; the registrar form is provider-aware (GoDaddy
key+secret vs Namecheap key + username + whitelisted IP).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds cross-server Fleet Monitor endpoints and models, plus broad security and robustness improvements across agent, backend, and tooling.
Key changes:
- New fleet monitor API (endpoints for heatmap, comparison, alerts, thresholds, anomalies, forecast, search, exports, and Prometheus) and service skeleton.
- New DB models for ServerAlertThreshold and MetricAlert to support metric alerts.
- Agent discovery/registration hardening: HMAC-signed discovery requests, signature/timestamp validation, minimal discovery responses, HMAC-based unregister headers, and enforced command timeout cap in agent command handling.
- WebSocket and agent client TLS handling gated by SERVERKIT_INSECURE_TLS env var; added auth rate limiting on agent gateway.
- Docker-compose: parameterize agent image via AGENT_VERSION and avoid running as root by adding docker group access.
- Backend security & policy: tightened CSP script/style sources, request size limit, 2FA pending-token rejection on non-2FA endpoints, registration flow/guards, admin-only SSO admin routes via middleware, rate limits on sensitive endpoints, and logging for auth/2FA events.
- Database service hardening: identifier validation, avoid passing passwords on CLI (use env vars), safer subprocess usage, and docker exec handling improvements.
- CronService: command validation to prevent shell injection and run commands without shell=True; build_service and other subprocess usages updated similarly.
- Misc: .env.example notes clarified for encryption key generation, invitation token generation improved, agent registry timestamp window reduced, various logging and input validations added, frontend FleetMonitor page and styles stubbed in, and Vite/config/style updates.
Overall this commit implements fleet monitoring scaffolding and a set of security, validation, and operational improvements to make agent discovery, registration, command execution, and backend services safer and more robust.
Allow authenticated (non-admin) users to fetch API key usage by replacing @admin_required with @auth_required in api_analytics.py and importing auth_required. Add optional user support to privileged_cmd and run_privileged in app/utils/system.py so callers can run commands as a specific user (sudo -u). Update wordpress_service to use the new user keyword when invoking privileged commands. Fix tests/conftest.py to put the backend root (parent directory) on sys.path so tests import correctly.
Introduce safer system interactions and improved logging fallbacks across services. Replace direct sudo/subprocess calls with run_privileged/privileged_cmd, add command availability checks (is_command_available) and Linux guards for user management. Enhance LogService with fallback readers (python file I/O, grep fallback, syslog and Windows Event Log support) and standardize responses via sourced_result. Update frontend Terminal to display log source and adjust UI when journalctl is unavailable. Misc: delegate ProcessService log retrieval to LogService, add PackageManager usage in PHP/SSL installs, and add warning logging in DeploymentService when diff generation fails.
Backend: Replace direct sudo/subprocess.run calls with run_privileged and privileged_cmd, add _needs_sudo/privileged_cmd logic and is_command_available checks, and return clearer errors for missing binaries (journalctl/grep/tail/truncate/logrotate/systemctl/service). Add FileNotFoundError handling and use privileged_cmd for Popen.
Frontend: UI and style refinements across Downloads, Security and Terminal pages — simplified Linux icon, added release date and banner actions, redesigned scanner UI into interactive scan cards with custom path and toolbar, handle unavailable journalctl with an empty state, update SVG icons. Styles (.less): switch mono font to JetBrains Mono, adjust spacing/gaps, sidebar text truncation, revamp downloads and security card styles, and improve terminal log fonts, wrapping and processes table layout.
Refactor services to use centralized system helpers (ServiceControl, PackageManager, run_privileged, is_command_available) instead of ad-hoc subprocess/os calls. Updated deployment, firewall, ftp, nginx, php, python, security, and ssl services to call the new utilities and added error/timeout handling improvements and package manager abstraction. Added backend/app/utils/system.py implementing the helpers and backend/tests/test_utils_system.py with unit tests. This centralizes privileged command execution and service/package management, simplifying tests and platform differences.
Adds API key rotation, IP allowlisting, anomaly detection, and replay protection to the ServerKit agent system. Backend changes include new endpoints, models, and services for security alerts, nonce tracking, and anomaly detection, as well as updates to the server model and agent authentication logic. The agent now supports credential rotation and replay protection via nonces. Frontend updates add UI and API methods for managing allowed IPs, viewing security alerts, and rotating API keys. Documentation and environment setup instructions are included.
