Adds cross-server Fleet Monitor endpoints and models, plus broad security and robustness improvements across agent, backend, and tooling.
Key changes:
- New fleet monitor API (endpoints for heatmap, comparison, alerts, thresholds, anomalies, forecast, search, exports, and Prometheus) and service skeleton.
- New DB models for ServerAlertThreshold and MetricAlert to support metric alerts.
- Agent discovery/registration hardening: HMAC-signed discovery requests, signature/timestamp validation, minimal discovery responses, HMAC-based unregister headers, and enforced command timeout cap in agent command handling.
- WebSocket and agent client TLS handling gated by SERVERKIT_INSECURE_TLS env var; added auth rate limiting on agent gateway.
- Docker-compose: parameterize agent image via AGENT_VERSION and avoid running as root by adding docker group access.
- Backend security & policy: tightened CSP script/style sources, request size limit, 2FA pending-token rejection on non-2FA endpoints, registration flow/guards, admin-only SSO admin routes via middleware, rate limits on sensitive endpoints, and logging for auth/2FA events.
- Database service hardening: identifier validation, avoid passing passwords on CLI (use env vars), safer subprocess usage, and docker exec handling improvements.
- CronService: command validation to prevent shell injection and run commands without shell=True; build_service and other subprocess usages updated similarly.
- Misc: .env.example notes clarified for encryption key generation, invitation token generation improved, agent registry timestamp window reduced, various logging and input validations added, frontend FleetMonitor page and styles stubbed in, and Vite/config/style updates.
Overall this commit implements fleet monitoring scaffolding and a set of security, validation, and operational improvements to make agent discovery, registration, command execution, and backend services safer and more robust.
Introduce full agent fleet management and discovery features across backend, agent, and frontend. Backend: add AgentVersion and AgentRollout models, extend Server/ServerGroup/ServerCommand/AgentSession with fleet, queue and latency fields; implement AgentFleetService (bulk upgrades, staged rollouts, queued commands, retries, diagnostics) and DiscoveryService (UDP broadcast scan); add new servers API endpoints for fleet operations. Agent: add UDP discovery responder, agent:update handler to trigger updater, and wire updater.UpdateTo to install specific versions; extend protocol messages with discovery and agent:update types. Registry: deliver queued commands on agent connect and track heartbeat latency. Frontend: add AgentFleet page and route, plus API client updates. Also add SECURITY_AUDIT.md to .gitignore. These changes enable discovery, staged rollouts, offline command queuing with retries, and fleet health/diagnostics.
Introduce a full workflow/automation subsystem: new WorkflowEngine with DAG validation/execution (Kahn's algorithm), logic-if branching, variable interpolation, script sandboxing (timeouts, output limits), retry semantics, notification nodes, and an in-process WorkflowEventBus for event-triggered workflows. Backend API updates include webhook trigger and validate endpoints, auto-generation of webhook IDs on create/update, and services emitting events (git push, health check failures, app stopped, monitoring alerts). Add Alembic migration to create workflow-related tables and bump VERSION to 1.4.0. Also update .gitignore, add GitHub issue/PR templates, quick-start script, and various frontend workflow panel/component adjustments to support the new automation features.
Introduce a full-featured workflow automation system: add WorkflowEngine service to execute workflows, background scheduler to trigger cron workflows, and croniter dependency. Extend Workflow model with automation fields (is_active, trigger_type, trigger_config, last_run_at, last_status) and add WorkflowExecution and WorkflowLog models. Add API endpoints to start executions, list executions, fetch execution details and logs, and persist execution records/logs. Implement frontend components for workflow nodes, config panels, and an execution history UI (multiple new React components) and wire minor builder UI imports. Note: database migrations are required to add new columns and tables.
Add user-configurable sidebar and initial workflow/environment features across backend, frontend, and docs.
Backend: store sidebar_config on User (JSON column), add get_sidebar_config / set_sidebar_config helpers, include sidebar_config in user.to_dict(), and validate/update sidebar_config in the update_current_user API (preset and hiddenItems validation).
Frontend: refactor Sidebar to compute visible items via a shared sidebarItems module and memoized grouping/rendering; add settings UI and sidebarItems (new components) and update settings styles to support sidebar configuration.
Docs & README: update README and ARCHITECTURE to document Workflow Builder, Environment Pipeline, and other new/renamed features; bump roadmap versions and reorder/expand roadmap phases to reflect Visual Designer, Automation Engine, pipeline, and monitoring work.
Overall: wires up end-to-end support for customizable sidebar preferences, surfaces them via API, and updates UI + documentation to reflect new automation and pipeline features.
When creating an admin via the backend CLI, mark setup as complete by calling SettingsService.complete_setup so the web UI won't show the setup wizard. In the frontend, add a sign-in flow to SetupStepAccount when registration is disabled (e.g. an admin was created via CLI): pull login and registrationEnabled from Auth, implement handleLogin, show an informational banner, and keep the original register flow as handleRegister. In AuthContext, make checkSetupStatus retry a few times (with a short delay) if the backend isn't ready, and on exhausted retries assume a fresh install (needsSetup and registrationEnabled = true) to avoid locking out users while the backend initializes.
Major refactor of install.sh and uninstall.sh to improve robustness, cross-distro support, and low-RAM VPS behavior. install.sh: add safety cd, colored helpers, Python detection with fallback to 3.12, RAM check and swap setup, noninteractive apt/dnf and needrestart handling, Docker/Docker Compose/Node installation, clone/update repo, create system dirs and install bundled templates, set up venv and dependencies, generate .env, create self-signed SSL, install systemd service, configure nginx reverse proxy, build frontend on host to avoid Docker memory issues, start services, perform health checks, and send install telemetry. uninstall.sh: formatting/consistency fixes, require root, safer docker compose teardown, remove /etc/serverkit and /var/serverkit, and send uninstall telemetry. Also add CONTRIBUTORS.md and remove the .claude code-review SKILL.md.
Introduce a full Services UI and detailed service pages: add Services list page and ServiceDetail page, plus many service-detail components (CommandsTab, EventsTab, GitConnectModal, GunicornTab, LogsTab, MetricsTab, PackagesTab, SettingsTab, ShellTab). Add hooks (useService, useDeployments), serviceTypes utility, and page-specific styles. Update App routing and Sidebar to use /services and wire routes for service list/detail. Minor UI style update in main.less. Update Dockerfiles to create /var/serverkit/apps directory for app storage. Fix backend analytics endpoint decorator by calling @auth_required() to correctly apply authentication.
Allow authenticated (non-admin) users to fetch API key usage by replacing @admin_required with @auth_required in api_analytics.py and importing auth_required. Add optional user support to privileged_cmd and run_privileged in app/utils/system.py so callers can run commands as a specific user (sudo -u). Update wordpress_service to use the new user keyword when invoking privileged commands. Fix tests/conftest.py to put the backend root (parent directory) on sys.path so tests import correctly.
Update backend/requirements.txt to upgrade Authlib from 1.5.2 to 1.6.9. Verify SSO/OAuth flows and run tests to ensure compatibility with the newer Authlib release.
Add a nosec comment to the http_requests.post call in EventService to suppress Bandit B113 warnings for the subscription post. Update many tests in test_utils_system.py to explicitly patch app.utils.system.os.name as 'posix' and app.utils.system.shutil.which to simulate presence of /usr/bin/sudo (and include sudo in apt/dnf side effects). Adjust test function signatures to accept the new mock and ensure package/service tests run as a non-root POSIX environment with sudo available.
Bump current development version to v1.5.0 and reorganize the roadmap to reflect completed work, reprioritize upcoming work, and add several new planned phases. Marked many items as completed (Team & Permissions, API Enhancements, SSO/OAuth, Database Migrations/Alembic, Multi-Server, Git Deployment, Advanced Security) and replaced older phase entries with new high-priority initiatives such as the New UI & Services page, Customizable Sidebar & Dashboard Views, Migration Wizard frontend, Container Logs & Monitoring UI, DNS management, Nginx advanced config, and expanded planned phases through v3.0. Also updated the version table to show completed and current releases and adjusted the contribution priority list to highlight UI, DNS, status page, and Nginx work. These changes keep the roadmap accurate with recent progress and surface critical next steps (notably merging the new-ui branch and migration UX).
Introduce team invitation support and a revamped email subsystem plus admin features. Adds Invitation model and API endpoints (create, list, revoke, resend, validate) and integrates invitation validation into user registration. Replaces old email account model with richer email models (EmailDomain, EmailAccount, EmailAlias, EmailForwardingRule, DNSProviderConfig) and updates models __init__. Refactors email API to manage domains, accounts, aliases, forwarding, DNS providers, spamassassin, Roundcube/Postfix services and queue/log endpoints. Adds several backend services (invitation, DNS provider, DKIM, Dovecot, Postfix, Roundcube, SpamAssassin, permission service) and a DB migration for permissions/invitations. Enhances admin API with permission management and activity dashboard endpoints, extends audit log actions, and adds a permission_required RBAC decorator. Includes frontend additions for settings/invitations UI and miscellaneous config/service updates and tests.
Introduce programmatic API features and observability: add API Key management, API usage analytics, webhook event subscriptions, and OpenAPI docs. New models (ApiKey, ApiUsageLog, ApiUsageSummary, EventSubscription, EventDelivery), services (api_analytics, api_key, event, openapi), and blueprints (/api-keys, /api-analytics, /event-subscriptions, /docs) were added and registered in create_app. Middleware for API key authentication, buffered analytics logging (with a flush thread), and enhanced rate limiting (custom key function and response headers) were implemented; RBAC was updated to accept API key auth via an auth_required decorator. Background threads for hourly aggregation and event retry were added; security CSP updated to allow resources for Swagger UI. Frontend settings UI and styles were added/updated for API keys and webhooks, and requirements were updated.
Add full SSO (Google, GitHub, generic OIDC, SAML) support and integrate Alembic-based DB migrations. Introduces OAuthIdentity model, sso_service, SSO API blueprint (authorize/callback/link/unlink/admin/config/test), and settings to enable/configure providers and enforce SSO-only logins. Integrates Flask-Migrate and a MigrationService with API endpoints for status/backup/apply/history and includes Alembic scaffolding (migrations/versions/001_baseline.py). Update app init to register SSO and migrations blueprints and initialize Migrate; replace fragile auto-column hack with MigrationService.prepare/check. Update User model (nullable password_hash, auth_provider, has_password/to_dict fields), add AuditLog SSO actions, add frontend SSO components/pages and a migration UI, and update ROADMAP and requirements. Notes: token encryption uses Fernet derived from SECRET_KEY; migrations may add the new auth_provider column at startup.
Introduce a full email management feature: adds EmailService (Postfix/Dovecot/SpamAssassin/OpenDKIM management, account creation, forwarding, webmail, queue/logs, service control), new email API blueprint with admin-protected endpoints, and EmailAccount DB model. Register the email blueprint in app init. Frontend: add Email page, route and sidebar link, preload accent color + theme to prevent FOUC, new email styles and a dashboard layout hook. Also update ROADMAP.md to include Phase 21 SSO/OAuth and bump roadmap last-updated date.
Introduce safer system interactions and improved logging fallbacks across services. Replace direct sudo/subprocess calls with run_privileged/privileged_cmd, add command availability checks (is_command_available) and Linux guards for user management. Enhance LogService with fallback readers (python file I/O, grep fallback, syslog and Windows Event Log support) and standardize responses via sourced_result. Update frontend Terminal to display log source and adjust UI when journalctl is unavailable. Misc: delegate ProcessService log retrieval to LogService, add PackageManager usage in PHP/SSL installs, and add warning logging in DeploymentService when diff generation fails.
Backend: Replace direct sudo/subprocess.run calls with run_privileged and privileged_cmd, add _needs_sudo/privileged_cmd logic and is_command_available checks, and return clearer errors for missing binaries (journalctl/grep/tail/truncate/logrotate/systemctl/service). Add FileNotFoundError handling and use privileged_cmd for Popen.
Frontend: UI and style refinements across Downloads, Security and Terminal pages — simplified Linux icon, added release date and banner actions, redesigned scanner UI into interactive scan cards with custom path and toolbar, handle unavailable journalctl with an empty state, update SVG icons. Styles (.less): switch mono font to JetBrains Mono, adjust spacing/gaps, sidebar text truncation, revamp downloads and security card styles, and improve terminal log fonts, wrapping and processes table layout.
Backend: filter out virtual/pseudo filesystem types and skip noisy mount prefixes (e.g. /snap/, /var/lib/docker/, /run/), and deduplicate mounts by device so only a single mount per device is reported for disk usage. Frontend: make the file manager sidebar scrollable with a max-height (calc(100vh - 120px)), disable that height in the small-screen modal, and remove max-height restriction for the last child to prevent clipping.
Backend: add PUT /cron/jobs/<id> endpoint and CronService.update_job to update job metadata and (on Linux) modify the system crontab; includes schedule validation and error handling.
Frontend: add useTabParam hook and update routes/PageTitle logic to support optional :tab URL segments for many pages, enabling deep-linked tabs. Update CronJobs UI to unify create/edit modal, add edit-on-click, run-output modal, improved SVG icon attributes, and wire updateCronJob in the API client. Styles: global SVG button rules, hover for cron items, and run-output styles.
Add backup dirs in Dockerfiles and adjust ownership. Change server routes to use a dynamic tab param and update components to support tabbed ServerDetail pages. Improve metrics handling by adding serverId support, more resilient field parsing, and server-specific metrics API calls. Revamp ServerDetail: introduce token modal and AgentRegistrationSection, reorganize Metrics, Settings and Security areas (IP allowlist, API key rotation, security alerts), move delete/generate token flows and wire up navigation. Update Servers and Docker pages to handle different API response shapes (Array checks) and tweak install modal copy/UX. Rename backend API call for registration token to /regenerate-token. Add many UI/Icon/CSS adjustments to support the new layouts and interactions, and fix various small bugs and data-shape assumptions.
Replace raw WebSocket handling with Socket.IO/Engine.IO protocol support in agent/internal/ws: build Socket.IO URL, perform Engine.IO OPEN handshake, connect to namespace, emit/parse Socket.IO events for auth, adapt read/write loops, add ping loop and reconnection handling, and map events to protocol messages. Update agent registration endpoint to /api/v1/servers/register. Add config file handling in cmd/agent (respect --config flag, set key file relative to config path) and import filepath. Update install scripts to new download API paths for Windows and *nix. Adjust frontend servers page copy to show registration tokens expire in 24 hours. Also add a Contributors section to the create-pr skill guidance.
Introduce six new .claude skill definitions for developer workflows: audit-subprocess, code-review, create-pr, db-inspect, deploy, and new-feature (each added under .claude/skills/). Update .gitignore to stop globally excluding the .claude directory (so skills can be tracked) and add ignores for generated artifacts and local settings: /.pr, /.reviews, and .claude/settings.local.json.
Add cross-platform development launchers (dev.sh, dev.ps1) and a frontend ESLint flat config; remove legacy scripts and update docs to use ./dev.sh. Harden backup and storage logic: require admin for backup endpoints, use realpath checks to ensure backup paths stay inside the backup dir, prevent path traversal during archive extraction, and generate schedule IDs with UUIDs. Improve StorageProviderService with masked-secret handling, set restrictive file perms on config, validate S3/B2 endpoint URLs to mitigate SSRF (urlparse + ipaddress), avoid following symlinks during uploads, and adjust MD5 usage. Update Bandit config to skip expected noisy checks and tweak .gitignore entries and contributing docs to target the dev branch and the new dev scripts.
Move the backend into docker-compose and switch the frontend to a multi-stage Docker build. Key changes:
- Added backend/.dockerignore to ignore virtualenv, pycache, env files, instance, etc.
- Updated backend Flask app to safely serve index.html only if it exists and return JSON status/errors when frontend assets are absent.
- Removed docker-compose.dev.yml and expanded docker-compose.yml to run both backend and frontend in Docker (backend service, healthcheck, restart/logging, new ports 5849:5000 & 3847:80, and named volumes).
- Frontend changes: remove file ignores that prevented in-container builds, replace nginx-only image with a Node builder stage + nginx runtime (builds assets in Docker), and update nginx proxy to target the backend service (proxy_pass http://backend:5000).
- Updated dev script (scripts/dev/dev.bat) to use plain `docker compose` commands and the new service/container names and ports.
These changes enable fully containerized local/dev runs where the frontend is built inside Docker and the backend runs as a service in the compose network.
Update backend/requirements.txt to upgrade Flask from 3.0.0 to 3.1.3 and Werkzeug from 3.1.5 to 3.1.6. Applies minor/patch updates for bug fixes and dependency compatibility in the backend.
Replace hardcoded repository URL from serverkit/serverkit to jhd3197/ServerKit across agent Dockerfile, README, packaging scripts (deb/rpm) and install scripts so downloads and metadata point to the correct repo. Backend install endpoints now replace the repo placeholder with the GITHUB_REPO constant, and the frontend Windows install command was adjusted to use the -Server parameter instead of -ServerUrl. These changes ensure install scripts, packages, and generated docs reference the intended repository and that generated installer commands match the agent's expected flags.
Replace hardcoded placeholder host with window.location.origin when generating Linux and Windows install scripts so the scripts use the current deployment origin at runtime. Also adjust the Linux install flag from `--server-url` to `--server` to match the expected install script parameter. These changes ensure generated installer commands point to the correct server and include the registration token.
