POST /servers/register needs no login and ran a slow PBKDF2 check
against every pending server per request, so a few IPs under the
5/min limit could keep the single worker's CPU busy. Tokens carry 256
random bits, so store a SHA-256 digest and find the row directly;
hashes issued before this still verify until they expire.
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
An audit found 102 FK columns with no index — every join, per-parent
listing, and the delete cascades from _delete_cascade_policy scanned
whole tables on them (deployments.app_id, audit_logs.user_id,
dns_records.zone_id, webhook_logs.webhook_id, ...). All FK columns now
declare index=True; migration 096 creates the same indexes on existing
installs, deriving the list from the live schema at upgrade time (so
it is idempotent and cannot drift from what an install actually has).
Ratchet: test_every_fk_column_is_indexed pins the rule with no
allowlist — a new FK column is born indexed or the build fails.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every NOT NULL FK from the KNOWN_DANGLING_ON_DELETE audit now declares
its parent-side relationship, so the delete-cascade policy door covers
it: AI conversations/pending actions, API keys, dashboards, event
subscriptions, login links, sent invitations, resource grants (users);
env vars + history, previews + settings, WAF policy (applications);
surveys, doctor results, proxy stack, tunnels (servers); manifests
(projects); DDNS hosts (zones); exposed services (tunnels); drills
(backup policies); rollouts (agent versions); site plugin installs
(WordPress sites); queue messages (groups). The ratchet set is now
EMPTY and stays that way.
One new deliberate exception: Workspace.projects — projects carry
environments and app groupings, so delete_workspace now refuses (409)
while any remain instead of silently destroying them (previously it
crashed with IntegrityError).
Migration 095 sweeps rows already orphaned on existing installs by
pre-fix hard-deletes — including an application's leftover env-var
secrets, which a reused SQLite row id could have attached to a future
app.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Migrations 065/068 have always created servers.management_mode and
servers.allow_agent_update_observed, but the model fields,
MANAGEMENT_MODES, is_managed, and agent_registry.observed_blocked_count
were lost in the 2026-07 data loss — POST /management-mode and
GET /observed-status raised AttributeError, and a mode flip that got
past validation would land on a transient Python attribute and
evaporate. test_survey_api carried skip-stubs documenting the hole.
Restored: the two columns (specs mirror the migrations), the mode
constant and is_managed property, to_dict emission, and a minimal
observed-block counter on the registry. The four skip-stubs are now
real tests, including one proving the flip persists across sessions.
The dispatch-level Observe guard remains hollow and is tracked in
plan 82 §H. events_json on notification_preferences is likewise
re-mapped (column carried by migration 059; its resolution surface is
still lost) so pre-loss override data survives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Opt-in to_dict over __table__.columns: auto-isoformat datetimes,
*_encrypted always dropped, __serialize_exclude__ for redacted columns,
serialize_extra() for computed fields (named to dodge ServerMetrics'
literal 'extra' column). 20 purely-mechanical to_dicts across 15 files
convert; genuinely custom shapes keep their hand-written methods (plan
75 §F's ruling stands). Field selection stays api/_query.py's job.
Gate: key-for-key parity against the pre-conversion key sets frozen
from git HEAD (only the Timestamp pair may be additive).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TimestampMixin (byte-identical to the hand-written pair — zero schema
migrations) adopted by 52 classes across 44 model files; the utcnow
tz-aware migration is now a one-file edit. uuid_pk() replaces the 13
pasted String(36) uuid4 lambdas across 7 files (ai.py's hex variant
stays grandfathered, documented). Deviant shapes (indexed created_at,
sanitization_profile's default-less updated_at, created-only models)
deliberately keep their own columns.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One crypto path for model-owned *_encrypted columns: assignment encrypts
and RAISES on failure (server.py's accessors print()-ed the error and
silently stored nothing — an agent could 'pair' yet never verify a
signature); read decrypts tolerantly (None + logged warning). is_set()
standardizes the has_secret masking flag. Server adopts it for
api_secret / api_secret_pending; legacy method names kept as delegates.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every model with a JSON-in-Text column hand-rolled the same try/except
json.loads pair — get_details/get_context/get_headers/get_payload and a
dozen more, each a place the next fix would have to land N times. The
mixin owns the mechanics (empty-is-default, corrupt-is-default so a
hand-edited row never 500s an endpoint, optional shape check,
falsy-on-write); the per-model parts — accessor name, column, default —
stay per-model.
Conversion is behaviour-preserving: identical fallbacks, identical
on-write empty forms (NULL vs '[]' vs '{}'). Deliberately NOT converted:
accessors that let json.loads raise on corrupt data (collapsing those
would be a behaviour change, not a consolidation), and inline parses in
to_dict (per-entity serialization choice). A handful of 'except
Exception' reads narrow to the dominant (JSONDecodeError, TypeError) —
the only exceptions json.loads raises for str input.
Second endpoint after /apps, same pattern. Server.to_dict grows a `fields`
parameter so $select skips the derived work rather than only trimming the
payload: `onboarding_progress` is a JSON walk that is ~29% of a row and is read
by exactly one screen, and `group_name` is a relationship load. A caller that
needs {id, name} -- Domains and Backups both do -- now pays for neither.
`group_name` is eager-loaded when it IS wanted, so the list stops issuing one
SELECT per server for it.
Two details this route needed that /apps did not: it answers with a bare JSON
array, so no paging envelope is emitted (that would break every caller), and an
explicit $orderby now wins over the hardcoded by-name sort while the default is
preserved when absent.
3 more tests; the 293 existing server/fleet/agent tests still pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The agent (serverkit-agent bbaf4e2) now sends install_dir and
config_dir in system_info. Panel side:
- servers.agent_install_dir + agent_config_dir columns (migration 047,
idempotent guard like its siblings) with the same coalesce contract
as the other system_info fields — a transient payload gap never
wipes a stored value
- exposed in Server.to_dict and /servers/available; TargetPicker
passes them on the agent target
- the File Manager rail prefers the reported config dir over the
installer convention (backslash-normalized for Windows agents;
agent-logs link derives <config_dir>/logs)
- proving test: full system_info -> DB -> available round trip +
coalesce survival
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four foundational enhancements toward the deploy-code developer experience:
- Container status aggregator (C7): new ContainerStatusService applies a
deterministic priority hierarchy (degraded > restarting > running:unhealthy >
starting > running:healthy > exited > unknown) over an app's containers,
cached with a short TTL. Exposed at /api/v1/status and pushed over a new
container_status Socket.IO channel; Applications page now renders an
aggregated ContainerStatusPill with sub-status tooltip.
- API token scopes + sensitive data masking (C10): require_scope() decorator
enforces fine-grained scopes for API-key requests (pass-through for JWT),
a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable
sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains
a grouped ApiKeyScopesModal picker and per-key scope pills.
- Server onboarding state machine (C8): new onboarding lifecycle
(pending -> validating -> installing_prerequisites -> installing_docker ->
pairing_agent -> ready/failed) with ServerOnboardingService, a
server_onboarding_logs table, servers.onboarding_* columns, REST endpoints,
and an OnboardingWizard surfaced on the server detail overview.
- Declarative template catalog (C5): documented catalog schema and
magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*)
layered onto the existing YAML template loader without changing behavior for
templates that don't use them; GET /api/v1/templates/catalog/schema.
Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend
builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cross-cutting backbone the rest of the #33 agency-scale milestone needs,
landed as a contained, narrow-only foundation rather than a 36-route rewrite.
- Nullable workspace_id FK on applications + servers (Alembic 015 off 014) with
an idempotent raw-SQL backfill: create a Default workspace, attach every
workspace-less row, and make every existing user a member so nobody loses
visibility once scoping is activated. Also creates the workspace_id indexes
that upgraded installs would otherwise lack.
- Centralized scope helper in WorkspaceService: ensure_default_workspace,
resolve_workspace_id (validates a requested workspace; membership-checked with
admin bypass; rejects deactivated accounts on the workspace path), and
scope_query.
- Opt-in + narrow-only: a request filters by workspace only when it carries an
X-Workspace-Id header / ?workspace_id=. scope_query INTERSECTS the workspace
filter with the existing per-user ownership filter, so it can only narrow
visibility, never broaden it — with no context the behavior is byte-identical
to before (admin->all, owner->own, servers global). This closes the footgun
where the backfill would otherwise let a non-admin enumerate all resources via
the Default workspace.
- Wired into apps.py + servers.py (list + create): replaces the duplicated
ownership branch and stamps new resources with the active-or-default workspace.
Deferred to follow-on slices: broad route adoption, per-site ACL, the
workspace<->User role reconciliation, server-side branding, and a frontend
workspace selector.
Verified: 10 new pytest tests (scope-narrowing, a can't-broaden-access
regression, deactivated-user gate, raw-SQL backfill against real SQLite +
idempotency), full suite 120 passed, a real end-to-end alembic upgrade to 015 on
a fresh DB (columns + indexes), and a 2-lens adversarial review (3 findings, all
fixed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two defects in the agent control plane found during the deep-dive review.
1. verify_agent_auth() failed OPEN on a missing secret (critical). The HMAC
check was wrapped in `if api_secret:`, so when get_api_secret() returned
None (NULL ciphertext, or a decryption error it swallows) the signature
block was skipped and execution fell through to "authenticated" — reducing
agent auth to agent_id + api_key_prefix, both non-secret observable values.
Now mandatory: no decryptable secret => log + track_auth_attempt(False) =>
return None. Also added Server.get_pending_api_secret() and select the
secret matching whichever prefix matched, so signatures actually verify
during a key-rotation window instead of hitting the old bypass.
2. Heartbeat reaper evicted freshly-reconnected agents (TOCTOU race).
_check_heartbeats snapshotted dead server_ids under the lock then released
it; _handle_agent_timeout popped unconditionally. An agent that reconnected
in that window (new ConnectedAgent, new socket_id) got clobbered and flipped
offline right after reconnecting. The snapshot now carries socket_id, and
_handle_agent_timeout re-validates under the lock (same socket AND still
stale) before evicting. Extracted HEARTBEAT_TIMEOUT so the scan and the
re-check can't drift; added the missing rollback on error.
Adds tests/test_agent_registry_security.py (5 tests). Full suite: 80 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three wins exposed by the redeployed panel showing real data:
1. Permission expansion now covers every namespace, not just docker
and system. Extend _expand_permission_scope so any action with a
read-style verb (list, list_installed, list_units, status, info,
inspect, search, logs, stats, available, current) accepts a
`<ns>:read` or `<ns>:<sub>:read` candidate. Mutating verbs add a
`<ns>:write` candidate. This unblocks docker:image:list,
system:info, packages:list_installed, systemd:list_units, and the
pyenv read paths for users on read-style profiles.
2. Go runtime probe was using `--version` (a flag), but Go takes
`version` as a subcommand. The agent's runtime detector tries
each form and falls back gracefully; Go is now correctly
identified by `go version` and the panel shows a real version
string instead of the "go flag provided but not defined" error
text.
3. SystemInfo gains Platform + CPUModel fields (pushed alongside
the existing hostname/cores/memory/disk). OverviewTab now reads
from the persisted server row first and falls back to the
synchronous getRemoteSystemInfo response — so CPU and memory
render even when the system:info action is permission-denied or
the agent is offline.
metrics.GetSystemInfo also gets a runtime.NumCPU() fallback so
CPU cores never reports 0 on Windows hosts where gopsutil's WMI
queries flake.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Refresh button on the System Status card was returning 403
because the per-server permissions allowlist (e.g. 'docker_manager'
profile) doesn't include 'agent:recapabilities' and the scope
expander has no rule for the agent: namespace. The action is purely
panel-driven discovery — it doesn't mutate the host — so it
shouldn't be subject to the same ACL as install/exec actions.
- Add Server.CONTROL_PLANE_ACTIONS bypass set: agent:recapabilities,
agent:update. has_permission() returns True for these unconditionally;
JWT auth at the API layer is what gates access.
- /poll endpoint now logs the keys it received in metrics / system_info
/ capabilities so we can debug "panel says N/A" failure modes from
the panel logs without instrumenting the agent.
- Agent's sendCapabilities now logs at INFO with sudo mode + capability
count + runtime count so the agent.log shows whether the periodic
resend is firing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The agent reports a rich capability payload (sudo mode, runtime
managers, detected runtimes, allowed paths, systemd JSON support)
that the panel was discarding once the WebSocket disconnected. The
Overview tab also gave no visibility into what the agent had
discovered, which left users guessing why a feature tab was missing
or a Docker call returned empty.
Backend:
- Server model gains seven nullable columns: cached_capabilities,
cached_runtimes, cached_runtime_managers, cached_allowed_paths,
cached_sudo, cached_systemd_json, capabilities_at. Migration 005
is idempotent (checks existing columns before adding) so it's safe
on already-bootstrapped databases.
- agent_registry.update_capabilities() now persists a snapshot to the
Server row whenever the agent reports state. Wrapped in try/except
so a DB hiccup doesn't drop the in-memory update.
- /servers/<id> falls back to the cached snapshot when the agent is
offline and exposes capabilities_stale + capabilities_at so the UI
can badge stale data.
Frontend:
- New SystemStatusCard renders sudo mode, package manager, runtime
managers, detected runtime versions, capability map (as colored
chips), and file-access roots. "Stale" badge + relative timestamp
+ Refresh button (calls agent:recapabilities).
- DockerTab now distinguishes "agent online but no docker daemon"
from "no containers found", with concrete remediation hints. Load
errors surface as a banner instead of being swallowed to console.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add robust credential handling and migration for agent encryption keys: surface LoadError on startup, log clear guidance when creds fail to decrypt, try legacy (pre-1.6.14) machine key fallback and transparently re-encrypt under the new host-stable key. Fix machine-key derivation (make host-stable instead of user-stable) and encapsulate decryption/migration helpers.
Fix SQLite autoincrement issue: change ServerMetrics.id from BigInteger to Integer so SQLite will autoincrement rows and metrics are recorded.
Ensure background heartbeat checker can touch the DB by capturing the Flask app in AgentRegistry and entering an app_context in the heartbeat thread (with a test-friendly fallback for when no app is available).
Frontend UI: simplify Tabs component classes and update pages to use a unified .page-container wrapper; replace many manual tab button implementations with Tabs/TabsList/TabsTrigger and add improved tab styles (disabled/focus states). Misc: add docs/PLAN_AGENT_FLEET.md to .gitignore.
Add end-to-end support for removing remote Docker networks and volumes and normalize remote API responses. Server: new Flask endpoints to DELETE remote networks/volumes and helpers to compute external base/ws URLs and v1 API paths. Agent: register new handler docker:network:remove and Client.RemoveNetwork implementation. Backend service: RemoteDockerService.remove_network uses agent registry for remote servers and falls back to local DockerService. Models: expand Server permission logic to map agent actions to profile/legacy scopes (DOCKER_READ_ACTIONS, scope expansion and wildcard matching) and add unit tests covering permission mappings. Frontend: handle agent-backed response shapes (unwrapRemoteData, normalizeListResponse), call new removeRemoteNetwork/removeRemoteVolume API methods, update download/install URLs to /api/v1, and tweak several UI texts. Install scripts: change config/log directories, pass --config when registering/starting agent, and update systemd/service/unit and Windows service install paths. README: small wording updates about fleet/remote deployment and monitoring.
Broad refactor and hardening across backend and frontend:
- Standardize API error responses by removing legacy `success` flags and returning concise error payloads.
- Replace inline admin decorators with centralized middleware import (admin_required) in DB and Docker APIs.
- DatabaseService: add safer MySQL execution helpers and secure user-creation flow (hex-encoding, env-pwd usage) and improve pg drop to validate identifiers.
- Models: add eager-loading (subquery) for relationships and add indexes to frequently queried columns to reduce N+1 and improve performance.
- DockerService: add structured logging and error messages for better observability when Docker commands fail.
- Config: tighten ProductionConfig cookie settings (secure, httponly, samesite).
- Frontend: migrate many .less styles to .scss, add new security UI components and API client modules, update package metadata accordingly.
- Misc: add APP_IMPROVEMENTS.md to .gitignore.
These changes improve security, reliability, observability, and frontend styling consistency.
Introduce full agent fleet management and discovery features across backend, agent, and frontend. Backend: add AgentVersion and AgentRollout models, extend Server/ServerGroup/ServerCommand/AgentSession with fleet, queue and latency fields; implement AgentFleetService (bulk upgrades, staged rollouts, queued commands, retries, diagnostics) and DiscoveryService (UDP broadcast scan); add new servers API endpoints for fleet operations. Agent: add UDP discovery responder, agent:update handler to trigger updater, and wire updater.UpdateTo to install specific versions; extend protocol messages with discovery and agent:update types. Registry: deliver queued commands on agent connect and track heartbeat latency. Frontend: add AgentFleet page and route, plus API client updates. Also add SECURITY_AUDIT.md to .gitignore. These changes enable discovery, staged rollouts, offline command queuing with retries, and fleet health/diagnostics.
Adds API key rotation, IP allowlisting, anomaly detection, and replay protection to the ServerKit agent system. Backend changes include new endpoints, models, and services for security alerts, nonce tracking, and anomaly detection, as well as updates to the server model and agent authentication logic. The agent now supports credential rotation and replay protection via nonces. Frontend updates add UI and API methods for managing allowed IPs, viewing security alerts, and rotating API keys. Documentation and environment setup instructions are included.
Introduces backend models, API endpoints, and services for remote server and agent management, including WebSocket agent gateway, server CRUD, metrics, command history, and remote Docker operations. Updates frontend to support server listing, detail pages, and related UI components and styles.
Juan Denis