Commit Graph
106 Commits
Author SHA1 Message Date
Juan Denis 27d4f839f4 Auth key usage, sudo user option, test path fix
Allow authenticated (non-admin) users to fetch API key usage by replacing @admin_required with @auth_required in api_analytics.py and importing auth_required. Add optional user support to privileged_cmd and run_privileged in app/utils/system.py so callers can run commands as a specific user (sudo -u). Update wordpress_service to use the new user keyword when invoking privileged commands. Fix tests/conftest.py to put the backend root (parent directory) on sys.path so tests import correctly.
2026-03-04 03:20:23 -05:00
Juan Denis ca87d1d54a Silence Bandit and mock sudo/posix in tests
Add a nosec comment to the http_requests.post call in EventService to suppress Bandit B113 warnings for the subscription post. Update many tests in test_utils_system.py to explicitly patch app.utils.system.os.name as 'posix' and app.utils.system.shutil.which to simulate presence of /usr/bin/sudo (and include sudo in apt/dnf side effects). Adjust test function signatures to accept the new mock and ensure package/service tests run as a non-root POSIX environment with sudo available.
2026-03-04 02:54:14 -05:00
Juan Denis b7fab1152b Add invitations, email models, admin APIs
Introduce team invitation support and a revamped email subsystem plus admin features. Adds Invitation model and API endpoints (create, list, revoke, resend, validate) and integrates invitation validation into user registration. Replaces old email account model with richer email models (EmailDomain, EmailAccount, EmailAlias, EmailForwardingRule, DNSProviderConfig) and updates models __init__. Refactors email API to manage domains, accounts, aliases, forwarding, DNS providers, spamassassin, Roundcube/Postfix services and queue/log endpoints. Adds several backend services (invitation, DNS provider, DKIM, Dovecot, Postfix, Roundcube, SpamAssassin, permission service) and a DB migration for permissions/invitations. Enhances admin API with permission management and activity dashboard endpoints, extends audit log actions, and adds a permission_required RBAC decorator. Includes frontend additions for settings/invitations UI and miscellaneous config/service updates and tests.
2026-03-04 02:34:49 -05:00
Juan Denis c5710e88b9 Add API keys, analytics, webhooks, and middleware
Introduce programmatic API features and observability: add API Key management, API usage analytics, webhook event subscriptions, and OpenAPI docs. New models (ApiKey, ApiUsageLog, ApiUsageSummary, EventSubscription, EventDelivery), services (api_analytics, api_key, event, openapi), and blueprints (/api-keys, /api-analytics, /event-subscriptions, /docs) were added and registered in create_app. Middleware for API key authentication, buffered analytics logging (with a flush thread), and enhanced rate limiting (custom key function and response headers) were implemented; RBAC was updated to accept API key auth via an auth_required decorator. Background threads for hourly aggregation and event retry were added; security CSP updated to allow resources for Swagger UI. Frontend settings UI and styles were added/updated for API keys and webhooks, and requirements were updated.
2026-03-04 01:53:46 -05:00
Juan Denis 08d5c4bade Add SSO/OAuth and DB migration framework
Add full SSO (Google, GitHub, generic OIDC, SAML) support and integrate Alembic-based DB migrations. Introduces OAuthIdentity model, sso_service, SSO API blueprint (authorize/callback/link/unlink/admin/config/test), and settings to enable/configure providers and enforce SSO-only logins. Integrates Flask-Migrate and a MigrationService with API endpoints for status/backup/apply/history and includes Alembic scaffolding (migrations/versions/001_baseline.py). Update app init to register SSO and migrations blueprints and initialize Migrate; replace fragile auto-column hack with MigrationService.prepare/check. Update User model (nullable password_hash, auth_provider, has_password/to_dict fields), add AuditLog SSO actions, add frontend SSO components/pages and a migration UI, and update ROADMAP and requirements. Notes: token encryption uses Fernet derived from SECRET_KEY; migrations may add the new auth_provider column at startup.
2026-03-04 01:37:22 -05:00
Juan Denis 9b2f20b07e Add email server backend, model and UI
Introduce a full email management feature: adds EmailService (Postfix/Dovecot/SpamAssassin/OpenDKIM management, account creation, forwarding, webmail, queue/logs, service control), new email API blueprint with admin-protected endpoints, and EmailAccount DB model. Register the email blueprint in app init. Frontend: add Email page, route and sidebar link, preload accent color + theme to prevent FOUC, new email styles and a dashboard layout hook. Also update ROADMAP.md to include Phase 21 SSO/OAuth and bump roadmap last-updated date.
2026-03-04 00:31:18 -05:00
Juan Denis 3491e88877 Add log fallbacks, privilege helpers, and checks
Introduce safer system interactions and improved logging fallbacks across services. Replace direct sudo/subprocess calls with run_privileged/privileged_cmd, add command availability checks (is_command_available) and Linux guards for user management. Enhance LogService with fallback readers (python file I/O, grep fallback, syslog and Windows Event Log support) and standardize responses via sourced_result. Update frontend Terminal to display log source and adjust UI when journalctl is unavailable. Misc: delegate ProcessService log retrieval to LogService, add PackageManager usage in PHP/SSL installs, and add warning logging in DeploymentService when diff generation fails.
2026-03-04 00:02:52 -05:00
Juan Denis 7037ef345a Run commands via privileged helpers; UI tweaks
Backend: Replace direct sudo/subprocess.run calls with run_privileged and privileged_cmd, add _needs_sudo/privileged_cmd logic and is_command_available checks, and return clearer errors for missing binaries (journalctl/grep/tail/truncate/logrotate/systemctl/service). Add FileNotFoundError handling and use privileged_cmd for Popen.

Frontend: UI and style refinements across Downloads, Security and Terminal pages — simplified Linux icon, added release date and banner actions, redesigned scanner UI into interactive scan cards with custom path and toolbar, handle unavailable journalctl with an empty state, update SVG icons. Styles (.less): switch mono font to JetBrains Mono, adjust spacing/gaps, sidebar text truncation, revamp downloads and security card styles, and improve terminal log fonts, wrapping and processes table layout.
2026-03-03 23:12:27 -05:00
Juan Denis c4591c405e Ignore virtual mounts, dedupe devices; tweak sidebar
Backend: filter out virtual/pseudo filesystem types and skip noisy mount prefixes (e.g. /snap/, /var/lib/docker/, /run/), and deduplicate mounts by device so only a single mount per device is reported for disk usage. Frontend: make the file manager sidebar scrollable with a max-height (calc(100vh - 120px)), disable that height in the small-screen modal, and remove max-height restriction for the last child to prevent clipping.
2026-03-03 22:31:27 -05:00
Juan Denis 7aa7642586 Add cron job editing and tabbed route support
Backend: add PUT /cron/jobs/<id> endpoint and CronService.update_job to update job metadata and (on Linux) modify the system crontab; includes schedule validation and error handling.

Frontend: add useTabParam hook and update routes/PageTitle logic to support optional :tab URL segments for many pages, enabling deep-linked tabs. Update CronJobs UI to unify create/edit modal, add edit-on-click, run-output modal, improved SVG icon attributes, and wire updateCronJob in the API client. Styles: global SVG button rules, hover for cron items, and run-output styles.
2026-03-03 22:11:30 -05:00
Juan Denis b992220f79 Add dev launcher; harden backups & storage
Add cross-platform development launchers (dev.sh, dev.ps1) and a frontend ESLint flat config; remove legacy scripts and update docs to use ./dev.sh. Harden backup and storage logic: require admin for backup endpoints, use realpath checks to ensure backup paths stay inside the backup dir, prevent path traversal during archive extraction, and generate schedule IDs with UUIDs. Improve StorageProviderService with masked-secret handling, set restrictive file perms on config, validate S3/B2 endpoint URLs to mitigate SSRF (urlparse + ipaddress), avoid following symlinks during uploads, and adjust MD5 usage. Update Bandit config to skip expected noisy checks and tweak .gitignore entries and contributing docs to target the dev branch and the new dev scripts.
2026-03-02 22:48:29 -05:00
Piya f0e640237a feature: add backups storage provider
Align backups API, services, frontend, and dev scripts for new backup storage functionality.

Made-with: Cursor
2026-03-03 02:56:51 +07:00
Juan Denis 8921b6ff0a Compose: containerize backend & build frontend
Move the backend into docker-compose and switch the frontend to a multi-stage Docker build. Key changes:

- Added backend/.dockerignore to ignore virtualenv, pycache, env files, instance, etc.
- Updated backend Flask app to safely serve index.html only if it exists and return JSON status/errors when frontend assets are absent.
- Removed docker-compose.dev.yml and expanded docker-compose.yml to run both backend and frontend in Docker (backend service, healthcheck, restart/logging, new ports 5849:5000 & 3847:80, and named volumes).
- Frontend changes: remove file ignores that prevented in-container builds, replace nginx-only image with a Node builder stage + nginx runtime (builds assets in Docker), and update nginx proxy to target the backend service (proxy_pass http://backend:5000).
- Updated dev script (scripts/dev/dev.bat) to use plain `docker compose` commands and the new service/container names and ports.

These changes enable fully containerized local/dev runs where the frontend is built inside Docker and the backend runs as a service in the compose network.
2026-03-01 14:30:21 -05:00
Juan Denis f0eb8b8cd9 Update GitHub repo references and install params
Replace hardcoded repository URL from serverkit/serverkit to jhd3197/ServerKit across agent Dockerfile, README, packaging scripts (deb/rpm) and install scripts so downloads and metadata point to the correct repo. Backend install endpoints now replace the repo placeholder with the GITHUB_REPO constant, and the frontend Windows install command was adjusted to use the -Server parameter instead of -ServerUrl. These changes ensure install scripts, packages, and generated docs reference the intended repository and that generated installer commands match the agent's expected flags.
2026-03-01 13:39:03 -05:00
Juan Denis 06de896175 Use system utils for commands and services
Refactor services to use centralized system helpers (ServiceControl, PackageManager, run_privileged, is_command_available) instead of ad-hoc subprocess/os calls. Updated deployment, firewall, ftp, nginx, php, python, security, and ssl services to call the new utilities and added error/timeout handling improvements and package manager abstraction. Added backend/app/utils/system.py implementing the helpers and backend/tests/test_utils_system.py with unit tests. This centralizes privileged command execution and service/package management, simplifying tests and platform differences.
2026-02-10 08:09:38 -05:00
Juan Denis caa799ba1b Replace jwt_required with RBAC, use socket auth
Replace generic @jwt_required() usage with role-based decorators and switch Socket.IO token handling to the auth payload.

- Replace jwt_required with viewer_required/admin_required across API modules (backend/app/api/cron.py, firewall.py, ftp.py, git.py, metrics.py) and update imports accordingly.
- Require JWT for previously public system/private endpoints (backend/app/api/system.py, private_urls.py).
- Change Socket.IO server connection handler to accept auth payload and read token from auth (backend/app/sockets.py) to avoid token leakage in query strings.
- Update frontend clients to send token via socket auth instead of query (frontend/src/pages/WordPressProject.jsx, frontend/src/services/socket.js).

These changes add finer-grained RBAC control for endpoints and improve socket auth security by removing tokens from URLs/logs.
2026-02-08 23:14:13 -05:00
Juan Denis e36c3247b3 Add multi-site WordPress management and UI overhaul
Introduces backend endpoints and service logic for managing multiple WordPress sites and their environments (production, staging, development). Updates frontend to display a grid of WordPress sites, enable site creation via modal, and refactors the WordPress page UI for multi-site support. Adds environment creation controls, improves environment modal logic, and enhances styles for the new layout. Also adds wp-cli.phar to backend for WordPress management.
2026-01-28 22:15:57 -05:00
Juan Denis 5113c85e99 Add configurable dev paths and centralize path management
Introduces backend/app/paths.py to centralize all key directory paths, allowing overrides via environment variables for local development. Updates all backend services and APIs to use these centralized paths instead of hardcoded values. Adds dev path override examples to backend/.env.example and updates .gitignore to exclude dev-data. Also changes frontend dev server port to 5274 and updates related scripts and CORS settings for consistency.
2026-01-28 21:12:55 -05:00
Juan Denis f0ccfea160 Add lightweight auto-migration for missing DB columns
Introduces an _auto_migrate_columns function in app initialization to automatically add missing columns to existing tables. Refactors the CLI migrate_db command to use a unified expected columns list and checks both 'applications' and 'wordpress_sites' tables for missing columns, improving consistency and reducing manual migration steps.
2026-01-27 22:07:59 -05:00
Juan Denis a706829a50 Add onboarding wizard and developer mode features
Introduces a multi-step onboarding wizard for initial server setup, including admin account creation, use case selection, resource tier display, and summary. Adds backend support for onboarding completion and use case storage. Implements developer mode toggle in settings, exposing an icon reference panel. Updates styles and refactors related frontend logic for setup and settings pages.
2026-01-27 21:25:15 -05:00
Juan Denis cf7ccf8af3 Add WordPress environment pipeline API and models
Introduces a comprehensive REST API for WordPress multi-environment management, including endpoints for project listing, environment CRUD, promotion, sync, comparison, locking, activity logs, container logs, resource limits, basic auth, WP-CLI, health checks, disk usage, bulk operations, and auto-sync scheduling. Adds new models for environment activity, promotion jobs, and sanitization profiles, and integrates auto-sync scheduling into the app startup. Updates model imports and registration to include the new models.
2026-01-27 19:35:21 -05:00
Juan Denis 06d63cbc52 Add standalone WordPress and SSL management UI
Implements backend endpoints and service logic for managing standalone WordPress via Docker, including install, uninstall, start, stop, and restart. Adds Nginx integration for WordPress at /wordpress. Introduces a new frontend page for SSL certificate management with Certbot install, obtain, renew, and revoke actions. Updates the WordPress frontend page to support one-click install, status, and lifecycle management. Updates documentation and install instructions for improved clarity.
2026-01-25 22:48:12 -05:00
Juan Denis c7444a88f5 Add resource tier enforcement and UI for WordPress creation
Introduces backend logic to determine server resource tier and block WordPress site creation on insufficient resources (lite tier). Adds ResourceTierContext and ResourceGate component to the frontend, displaying a UI warning and disabling creation actions when requirements are not met. Updates WordPress page and styles to integrate the new resource gating mechanism.
2026-01-25 20:01:07 -05:00
Juan Denis 6bcbcab56a Add unified dev scripts and improve login UX
Moved and improved development helper scripts for better cross-platform support, including WSL and Docker. Updated backend login to accept username or email, and updated frontend login form accordingly. Enhanced backend and frontend dev server compatibility for WSL. Updated documentation for streamlined setup and usage.
2026-01-25 18:33:25 -05:00
Juan Denis f4ed11ceaf Add WordPress site management API and models
Introduces backend API endpoints for WordPress site management, including site creation, environment handling, database snapshots, sync, and Git integration. Adds new models for WordPressSite, DatabaseSnapshot, and SyncJob, and registers the new blueprint. Updates frontend with new WordPress pages, components, and styles to support these features.
2026-01-25 16:47:04 -05:00
Juan Denis d2e5fdd6d6 Update default GitHub repo for agent releases
Changed the default value of SERVERKIT_GITHUB_REPO from 'serverkit/serverkit' to 'jhd3197/ServerKit' to point to the new repository for agent release checks.
2026-01-25 14:28:08 -05:00
Juan Denis 4cdcff598d Remove compare_server_metrics endpoint
Deleted the /metrics/compare API endpoint and its handler from servers.py. This endpoint allowed comparison of metrics across multiple servers but is no longer needed.
2026-01-22 19:55:22 -05:00
Juan Denis bca7e29b57 Implement security features: key rotation, IP allowlist, anomaly detection
Adds API key rotation, IP allowlisting, anomaly detection, and replay protection to the ServerKit agent system. Backend changes include new endpoints, models, and services for security alerts, nonce tracking, and anomaly detection, as well as updates to the server model and agent authentication logic. The agent now supports credential rotation and replay protection via nonces. Frontend updates add UI and API methods for managing allowed IPs, viewing security alerts, and rotating API keys. Documentation and environment setup instructions are included.
2026-01-22 18:11:22 -05:00
Juan Denis fd90e2eead Add full-stack remote terminal/PTY support
Implements remote terminal functionality across agent, backend, and frontend. Adds PTY session management in the agent using creack/pty, new protocol actions, and terminal manager. Backend exposes terminal session APIs and session tracking via TerminalService. Frontend introduces a RemoteTerminal component using xterm.js and related APIs for interactive remote shell access.
2026-01-22 16:44:06 -05:00
Juan Denis 78fee0ada0 Add server historical metrics API and service
Implements backend API endpoints and a new ServerMetricsService for historical metrics, retention policies, aggregation, and multi-server comparison. Updates frontend ApiService with methods for metrics history, aggregation, comparison, retention stats, and cleanup. Updates integration plan to mark monitoring & metrics as complete.
2026-01-22 08:51:31 -05:00
Juan Denis 4a34349ed4 Add remote Docker Compose management support
Implements full remote Docker Compose project management, including listing, starting, stopping, restarting, pulling images, and viewing logs for compose projects. Updates the agent, backend API, service layer, frontend UI, and API client to support these operations for both local and remote servers.
2026-01-22 08:31:27 -05:00
Juan Denis 35deeb8882 Add remote Docker container logs streaming support
Implements backend and frontend support for streaming Docker container logs from remote servers. Adds new API endpoints, agent handler, and UI controls for selecting log line count. Updates documentation to reflect the completed feature.
2026-01-21 23:23:29 -05:00
Juan Denis 4e9767a6ca Add agent auto-update mechanism and backend endpoints
Implemented agent auto-update functionality, including periodic update checks, manual update command, and configuration options. Added backend API endpoints for agent version checking, update triggering, and download redirection. Updated documentation to reflect completion of the auto-update system.
2026-01-21 23:07:20 -05:00
Juan Denis 528c86d640 Add agent install scripts and release automation
Introduced Linux (install.sh) and Windows (install.ps1) installation scripts for ServerKit agent, and exposed new backend endpoints to serve these scripts and installation instructions. Added a GitHub Actions workflow (agent-release.yml) for automated agent builds and releases. Updated ROADMAP.md and SERVERS_INTEGRATION_PLAN.md to reflect progress on installation and distribution features.
2026-01-21 22:57:40 -05:00
Juan Denis 009fb12435 Add multi-server management and agent gateway
Introduces backend models, API endpoints, and services for remote server and agent management, including WebSocket agent gateway, server CRUD, metrics, command history, and remote Docker operations. Updates frontend to support server listing, detail pages, and related UI components and styles.
2026-01-21 22:47:42 -05:00
Juan Denis 39aac19f59 Add advanced security management features
Implemented backend and frontend support for Fail2ban management, SSH key management, IP allowlist/blocklist, security audit reports, Lynis vulnerability scanning, and automatic security updates. Added new API endpoints and service methods for these features in the backend, and introduced corresponding tabs and UI components in the frontend Security page for comprehensive server security administration.
2026-01-21 21:58:59 -05:00
Juan Denis d4b44b582f Integrate firewall management into Security page
Moved firewall management UI from its own page into a new tab within the Security page for unified security management. Updated backend firewall service to improve detection of installed firewalls. Adjusted sidebar navigation and routing to remove the standalone Firewall page. Added new styles for the integrated firewall tab and updated roadmap to reflect these changes.
2026-01-21 21:44:07 -05:00
Juan Denis 5597521148 Add git webhook and deployment management APIs
Introduces models and services for managing Git webhooks, deployment history, and Gitea repository interactions. Adds comprehensive API endpoints for webhook CRUD, repository browsing, deployment triggering, rollback, and log retrieval. Updates ROADMAP.md to remove version targets from planned features.
2026-01-21 21:31:57 -05:00
Juan Denis 5139fac360 Add Nginx /gitea path support for Gitea and update UI
Introduces Nginx configuration for serving Gitea at the /gitea path, including creation and removal logic in NginxService. GitService now manages and reports the slug-based URL path, and the frontend Git page uses and displays the new /gitea path if available, falling back to port-based URLs for backward compatibility.
2026-01-21 20:56:13 -05:00
Juan Denis 39fc113eb7 Add integrated Git server (Gitea) management UI and API
Introduces backend API endpoints and service logic for installing, managing, and uninstalling a Gitea-based Git server. Adds a new frontend page for Git server management, updates the sidebar and routing, and includes supporting API methods and styles for the new feature.
2026-01-21 18:25:50 -05:00
Juan Denis 3b59478b7c Enhance dashboard and workflow with real app support
Added system info API for admin, improved system metrics with cached memory and network I/O stats, and updated frontend dashboard with refresh controls and live/polling status. Workflow builder now supports loading full server infrastructure, adding real apps, and improved node palette. Docker app nodes and config panels display more details for real apps, and styles updated for new UI elements.
2026-01-19 20:57:42 -05:00
Juan Denis 5fe76cd811 Add support for template file creation and bind mounts
Introduces handling of a 'files' section in templates, allowing creation of files and automatic updating of docker-compose.yml to use bind mounts for these files instead of named volumes. Also updates nginx config generation to avoid using 'default_server' for private URL servers, delegating that responsibility to serverkit.conf.
2026-01-19 19:49:12 -05:00
Juan Denis f05d056678 Improve port variable handling and add app cleanup CLI
Port variables in templates are now hidden and always auto-generated, never using user-supplied values. The TemplateService's port assignment checks both database and Docker for used ports, and removes obsolete Docker Compose version fields. Adds new CLI commands for cleaning up all applications, performing a factory reset, and listing applications. Minor update to AuthContext to mark setup as complete after registration.
2026-01-19 16:56:57 -05:00
Juan DenisandClaude Opus 4.5 4a5c474e4b fix(workflow): Start containers and pass DB credentials on deploy
- Add compose_up() call to start containers after creating docker files
- Update app status to 'running' on successful container start
- Pass database password (DB_PASSWORD) to app environment variables
- Store credentials in created_resources for proper propagation
- Return warning if containers fail to start (app still created)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 15:27:02 -05:00
Juan DenisandClaude Opus 4.5 f55e399021 feat(22-3): Add WebSocket container log streaming
- Add subscribe_container_logs event for real-time log streaming
- Add unsubscribe_container_logs event to stop streaming
- Stream logs via docker logs --follow subprocess
- Parse log lines with timestamp and level detection
- Emit container_log events to room logs_{app_id}
- Handle container_log_ended when container stops
- Clean up processes on disconnect or unsubscribe
- Support service selection for compose apps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 14:45:50 -05:00
Juan DenisandClaude Opus 4.5 744c626edb feat(22-2): Add container-logs REST API endpoint
- Add GET /apps/{id}/container-logs with tail, since, timestamps, format params
- Add GET /apps/{id}/containers to list containers for compose apps
- Support 'raw' and 'json' output formats
- Include container state and available services in response
- Cap log lines at 10000 for performance
- Handle service selection for multi-container compose apps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 14:44:51 -05:00
Juan DenisandClaude Opus 4.5 f3010d9983 feat(22-1): Add streaming logs and container resolution to DockerService
- Add stream_container_logs() for real-time log streaming via subprocess
- Add get_app_container_id() to resolve container ID from app model
- Add get_all_app_containers() for compose apps with multiple containers
- Add parse_log_line() to extract timestamp and detect log level
- Add parse_logs_to_lines() for batch parsing
- Add get_container_state() to check if container is running

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 14:44:05 -05:00
Juan Denis 46b6d5e7ad Add server time and timezone management features
Introduces backend API endpoints and service methods for retrieving and setting server time and timezone (admin only). Updates the dashboard to display current server time and timezone, and adds a timezone selector to the settings page for admins. Includes supporting API methods and frontend styles for the new UI components.
2026-01-19 14:06:44 -05:00
Juan Denis 85831b9b54 Add metric filter toggles and improve validation UX
Introduces clickable filter buttons to toggle CPU, memory, and disk metrics in MetricsGraph, replacing the static legend. Refines app name validation in both backend and frontend for template installation, allowing 2+ character names and improving error handling. WorkflowBuilder now handles missing workflow data gracefully and supports a fullscreen layout. Updates related styles for new filter legend and fullscreen workflow view.
2026-01-19 13:43:46 -05:00
Juan Denis 83ba62bb08 feat(21-2): Add workflow deploy API endpoint 2026-01-19 12:38:12 -05:00