Plan 55 task 7, backend half. A manifest's `permissions` array was
declaration-only: the install dialog showed it and nothing ever compared it
against what the extension went on to do.
The capability gate now records every call through it -- including the
refusals, which are the interesting ones -- so `usage_report(slug)` can answer
declared-versus-observed per extension. Observations are in-memory and
per-process by design: the question is "what has this extension done since the
panel started", and persisting it would outlive the evidence and start reading
like an audit trail, which this is not. `has()` deliberately does not record; it
is a question, not a use, and counting it would fabricate activity for callers
that only checked.
The larger half of this change is admitting what the panel cannot see. Task 7
was specified as hooks at "the shell/docker/db helper entry points", and those
do not exist: the SDK exposes no shell, docker, filesystem or network helper at
all, and hands out `db` as raw SQLAlchemy. An extension wanting any of them
imports the host module directly, where no in-process gate is involved. The one
genuinely gated capability is `agent.command:<action>`, because the SDK is the
only way to reach an agent.
So observability is data, not prose: OBSERVABLE_PREFIXES marks what the gate
truly mediates, and the report marks every other declared permission
`observable: false` rather than showing it as "never used". Absence of evidence
is not evidence of absence here, and a UI claiming otherwise would be inventing
enforcement the panel does not have -- exactly the honesty the plan's D2
decision asks for.
GET /plugins/<id>/permissions returns the report, admin-only: it describes what
third-party code has been reaching for.
The UI surface and the docs follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 72 item A.2. FleetDoctorService, FleetRepairService, the FleetDoctorResult
model, migration 063 and their tests were all complete and reachable from
nothing: no routes, register_jobs() never called, no seeded schedule. The
frontend was further along than the backend -- getFleetDoctorReport() and
runFleetSweep() have been calling /doctor/fleet and /doctor/fleet/run, two
endpoints that did not exist, so the multi-server health story the agent
platform advertises 404'd on contact.
Nothing in those services was redesigned. The one addition is a read-side
aggregator: they could run a sweep and persist rows but had no way to hand
back a per-server report, so fleet_report() composes one. It lists every
registered server rather than only connected ones, which is what makes an
unreachable box a reported state -- last known rows plus connected: false --
instead of a hole in the report. Persisted rows key on check_key while the
UI's field-driven renderer keys on key, so the report emits both.
Three admin-only routes, matching api/doctor.py's gating exactly. The sweep
returns 202 with a job id and never runs on the request thread: it fans out
across agents over the network, and the agent gateway registry is
single-worker and in-memory. Repair is a thin passthrough to FleetRepairService,
which already owns the allowlist, capability gate and audit trail; refusals
come back as per-item results inside a 200, so one offline server cannot fail
a batch. The batch is bounded at 25 so a single request cannot fan an
unbounded number of remote mutations at that single worker.
The schedule is daily like the host doctor but starts 30 minutes behind it,
so the fleet fan-out contends with neither the local sweep nor the boot-time
agent reconnect storm.
The frontend gets a Fleet Doctor section rather than losing its dead methods.
The per-check article was extracted into one component now shared by the host
and fleet lists, so fleet findings render through the same
repairable/repair_ref path instead of a parallel implementation; the confirm
modal says plainly that a fleet repair acts on a remote server through its
agent and is audited.
test_jobs' builtin schedule count moves 21 -> 22 for the new schedule. The
builtin.* kind count is untouched -- this kind is doctor.fleet.run.
Note for anyone reading the suite: tests/test_fleet_doctor_dns_rotation.py is
skipped at module level and proves nothing here. Its rotating-DNS-window API
did not survive the recovery rebuild; that is pre-existing and unrelated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 5. The Domains SSL modal asked for a contact email on every
single certificate, and the backend hard-required one per call. The app-detail
Settings tab did remember it -- in localStorage, so the memory was per browser
and invisible to the other surface. The two never agreed, and neither could
help the paths with no form at all: domain attach passed `admin@<host>` when
it had nothing else, an address that usually does not exist, which is where
every expiry notice for those certificates went to die.
There is now a single panel-wide ACME contact in system settings.
get_acme_contact() resolves an explicit address first and falls back to the
stored one; remember_acme_contact() writes it back whenever a request carries
an explicit address. Both are best-effort on the settings side by design: a
settings failure must not break issuance, and failing to remember an address
must not turn a successful certificate into an error. An empty input never
clears a stored contact, since the automatic paths pass whatever they were
given and that is frequently nothing.
Wired into every issuance entry point -- the domain SSL enable route, the
generic certificate route, and domain attach, which now consults the stored
contact before falling back to the admin@<host> guess. The address is only
remembered after issuance actually succeeds; remembering one Let's Encrypt
rejected would be worse than remembering none.
Prefill needed its own endpoint. The settings routes are all admin-gated, but
issuing a certificate for an app you can edit is not an admin action, so
GET /ssl/acme-contact is jwt_required only -- it returns an address the
operator chose for notices, not a credential. The Domains modal and the
Settings tab both prefill from it, and the Settings tab keeps its localStorage
seed so a browser that already has the value shows it without a flash.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 7. Let's Encrypt prefers IPv6 whenever a AAAA record exists, so a
domain whose A record points at this server perfectly still fails HTTP-01
issuance when its AAAA points somewhere else. That shape looked healthy
everywhere in the panel, which is what made it expensive: the operator saw a
verified domain, a green doctor report, and a certificate error naming nothing
either of them had mentioned.
Both DNS surfaces were blind to it for different reasons. The doctor's sweep
returned ok as soon as the server IP was *among* the resolved addresses, so
the extra IPv6 answer never registered -- and that branch is reached precisely
when the A record is correct, i.e. exactly the case worth reporting.
verify_domain used socket.gethostbyname, which is IPv4-only and cannot see a
AAAA record at all.
Both now compare every resolved address against this server: the configured
public IP plus the host's own interface addresses, so a correct dual-stack
setup stays quiet and only a genuine stray is named. Comparison is by parsed
value rather than by spelling, since an expanded 2001:0db8:... form and its
compressed equivalent are the same address, and psutil's zone ids (fe80::1%eth0)
are stripped first. Interface enumeration happens once per doctor sweep
instead of once per domain.
The doctor reports this as a warn, not a fail: the site itself is reachable
and it is a stray record rather than a broken one. Both messages say what to
do -- remove the AAAA record or point it at this server's IPv6 address --
rather than only naming the symptom.
verify_domain keeps its existing response shape. ip_address still carries a
single IPv4 address for the callers that predate this, with the full set added
alongside as ip_addresses, plus aaaa_conflict and a warning string. The
Domains page needed the last part to matter: it printed a green "Domain
verified!" toast for anything that resolved, so a conflict the API now
reports would have gone straight back to being invisible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 3. The quick-access rail links the local target at /etc/nginx,
but /etc was never in ALLOWED_ROOTS, so the shortcut returned 403 for every
user including admins -- a dead button, and our own regression: it shipped
after the review that flagged the surrounding SSL story.
/etc/nginx is now an allowed root, but browse-only. The panel generates those
vhosts itself, so a hand edit is silently overwritten on the next domain
change, and a writable /etc/nginx would hand anyone holding files.write a
root-owned config that the panel itself reloads -- a much larger grant than
"let me look at the effective config", which is all the shortcut is for.
READ_ONLY_ROOTS is a second tier under ALLOWED_ROOTS rather than a special
case for this one path: is_path_writable() = allowed AND not read-only, and
every mutating entry point now gates on it instead of on is_path_allowed, so
adding a future root to the list is enough to make the whole write surface
refuse it. Both comparisons resolve with realpath first, so a symlink planted
in a writable root cannot launder a write into the subtree.
copy is deliberately asymmetric -- copying a vhost out to keep only reads the
source, so only the destination is gated -- while move removes its source and
therefore checks both ends. The upload endpoint repeats the check because it
writes to disk directly instead of going through FileService's methods.
Only this subtree is opened, not /etc: /etc/passwd and /etc/shadow stay
denied, and the panel-internal PROTECTED_ROOTS exclusion is untouched
(GHSA-rm3m-9mvw-68fh).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The two worst endpoints in the backend, both measured in plan 69 phase 6.
**/status/apps** ran one `docker compose ps` per app plus a `docker inspect`
per container -- roughly 30 apps x 3 containers = ~120 process spawns per
request -- and it is refilled on a timer, so the cost never amortised. It is
now ONE host-wide `docker ps` whose output is indexed by the compose
project/service/working-dir labels Docker already stamps on every container,
shared between callers through a 3-second in-memory snapshot.
The snapshot is deliberately module memory rather than CacheService: it must be
droppable synchronously, because a status that survives the action that changed
it reads as a broken panel. So `invalidate(app_id)` is called from start, image
apply, deploy and rollback. 3s is short enough that a missed hook self-corrects
within one poll.
**/admin/activity/summary** ran two 90-day loops issuing one COUNT per day =
180 sequential queries. Now one `GROUP BY func.date(...)` per series with the
gaps zero-filled in Python, so the response shape is byte-identical. `func.date`
is portable across the SQLite and PostgreSQL this app supports. A third N+1 fell
out on the way: the top-users block did one SELECT per user, now a single IN.
New tests assert the zero-fill specifically -- a day with no rows must still
appear and the series length must match the window, which is exactly what a
GROUP BY drops if you forget.
1140 backend tests pass. The 3 failures in that run (test_fail2ban_jail_service,
test_url_swap, test_site_routing) all reproduce on a clean tree -- the last is
the known "serverkit-wordpress extension not installed" class.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second endpoint after /apps, same pattern. Server.to_dict grows a `fields`
parameter so $select skips the derived work rather than only trimming the
payload: `onboarding_progress` is a JSON walk that is ~29% of a row and is read
by exactly one screen, and `group_name` is a relationship load. A caller that
needs {id, name} -- Domains and Backups both do -- now pays for neither.
`group_name` is eager-loaded when it IS wanted, so the list stops issuing one
SELECT per server for it.
Two details this route needed that /apps did not: it answers with a bare JSON
array, so no paging envelope is emitted (that would break every caller), and an
explicit $orderby now wins over the hardcoded by-name sort while the default is
preserved when absent.
3 more tests; the 293 existing server/fleet/agent tests still pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Field selection did not exist anywhere in the backend (`fields=`/`select=`: 0
hits across 90+ API files) and neither did eager loading (joinedload/
selectinload/subqueryload: 0 matches). /apps sends ~2,319 B per row where the
list UI reads ~411, over an unbounded `query.all()` -- for an admin with no
workspace header that is literally SELECT * FROM applications.
Ports the author's conuco_v2 query layer as app/api/_query.py, with two
deliberate differences:
- ORM, not Core. conuco resolves fields through `table.c`; here they resolve
against the mapper's column set, and that set is the ALLOWLIST -- an unknown
field is a 400, never a getattr into arbitrary model attributes.
- $select is not only a column narrowing. That is where the win actually is:
Application.to_dict() costs ~3 extra queries per row, and two of them
(image_scans, image_update_checks) are lazy='dynamic', so the attribute IS a
Query and NO loader option can hoist them. Declining a field is the only way
not to pay for it. to_dict(fields=...) now skips the loads it was not asked
for, so $select=id,name costs zero extra queries per row instead of three.
/apps also eager-loads server/project/environment/linked_app -- but only when
the request actually asked for the names they resolve, since a joinedload for
a field nobody selected is a wider row for nothing.
Every parameter is optional and to_dict(fields=None) is the full legacy
payload, so adopting this changed no existing caller's response. Paging meta
appears only once a caller pages.
$view is deliberately not implemented: ServerKit's saved views are per-user
rows already resolved in the browser, so a second server-side copy of the same
concept would only be somewhere for the two to disagree.
19 new tests. The 2 failures in the wider run (test_fail2ban_jail_service,
test_url_swap) reproduce on a clean tree -- pre-existing, not from this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Making Domain soft-deletable silently changed the meaning of every existing
`Domain.query`: before, it could only return live rows. An audit of all 16 call
sites found ten that were now wrong, several with side effects outside the
panel.
The ones that reached the outside world:
setup_reconcile_service.py:267 the only is_primary row after deleting the
primary IS a tombstone -- and it drives a WP-CLI search-replace across
the whole WordPress database. It would have repointed a live site at a
host nginx no longer serves.
setup_reconcile_service.py:83 DNS backfill enumerated deleted domains and
called ensure_a_record() -- creating a real record at Cloudflare/Route53
for a domain the user removed.
api/domains.py:367 (enable_ssl) a real ACME order for a host that is in no
server_name: the HTTP-01 challenge cannot be served, burning Let's
Encrypt failed-validation limits for the whole registered domain, and on
success installing a wrong-CN cert on the LIVE vhost.
doctor_service.py:267 permanent red "does not resolve" rows that no
action could clear, and the same set is the allowlist for one-click DNS
repair -- so Repair created a provider record for a deleted domain.
And the ones that broke the feature itself:
domain_attach_service.py:79 a tombstone counted as a name clash, so you
could not move a hostname to another app after removing it. Migration 083
made the unique index partial precisely to free the name; this re-imposed
the burn in application code.
domain_attach_service.py:71 attach found the tombstone, reported success,
created no live row, published nothing -- and then mutated a recycle-bin
record.
site_domain_service.py:600 same for give_subdomain: success, a URL, a DNS
record, and a site that was never published.
api/domains.py:532 a second vhost renderer (admin Regenerate)
writing tombstones back into server_name.
Application.domains still includes tombstones ON PURPOSE -- filtering it with a
primaryjoin would, with delete-orphan, make SQLAlchemy HARD-delete the row the
recycle bin exists to keep. `Application.live_domains` is the accessor for
"what this app actually serves", and to_dict() uses it so the /apps payload no
longer lists deleted domains.
Per-domain routes now 404 a tombstone instead of editing it.
Seven tests cover the leaks that had real consequences.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
create_domain and delete_domain each called NginxService.create_site directly,
passing no ssl_cert / ssl_key / micro_cache. create_site takes those as
arguments, so the regenerated vhost came out HTTP-only and cache-less: on a
site covered by a base wildcard cert, touching ANY domain silently knocked the
whole site — every other domain on it — back to plain HTTP, and dropped its
micro-cache directives. Invisible until someone loaded the site.
Both now publish through SiteDomainService.write_app_vhost via one local seam,
so add and remove cannot disagree about what the served config should be. That
also fixes two lesser asymmetries: the rewrite was gated on app_type == docker,
so php/static/flask/wordpress vhosts kept advertising a domain after it was
removed; and it never called enable_site.
ALSO FIXES A REGRESSION I INTRODUCED WITH SOFT DELETE. app_vhost_kwargs read
`Domain.query`, which before soft delete could only return live rows and now
returns tombstones too. The vhost renderer would have written a deleted domain
straight back into server_name — the domain would keep being served after
deletion, and drift detection (which renders through this same function) would
report correct config as drifted. Now query_active().
Three tests, each failing against the previous code: the wildcard cert and
micro-cache survive in the resolved kwargs, a soft-deleted domain is absent
from them, and an end-to-end delete leaves the surviving domain on HTTPS.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting is the only action in a control panel that repeating cannot undo, so
records a person destroys by hand now leave a tombstone and the Recycle Bin
hands them back.
SoftDeleteMixin adds deleted_at/deleted_by_id plus soft_delete/restore and
query_active()/query_deleted(). recycle_bin_service is a REGISTRY: a model
registers with a label function and an optional on_restore hook, so the API
lists and restores across types without knowing what a domain is. Adding the
next soft-deletable model is one register() call.
Applied to Domain (a real, destructive case -- deleting one used to lose the
record outright) and SavedView. DELETE /domains/<id> now tombstones and still
tears the vhost down; restoring a row does NOT re-do side effects unless the
type supplies on_restore, which is the honest default.
API: GET /recycle-bin (all kinds, newest first), POST
/recycle-bin/<kind>/<id>/restore, DELETE /recycle-bin/<kind>/<id> (purge,
admin-only -- the one irreversible action), POST /recycle-bin/purge-expired
with a 30-day default retention.
THE PART THAT NEEDED CARE: both tables had a plain UNIQUE covering what is now
a tombstone, so deleting a domain would have permanently burned its name and
deleting a view would have blocked re-creating one by the same name. Migration
083 replaces each with a PARTIAL unique index predicated on deleted_at IS NULL
-- supported by SQLite 3.8+ and Postgres. Tests pin exactly that: a deleted
name can be reused, and the live row stays unique.
Also gives SavedView a `slug` (backfilled), the handle the upcoming
?view=<slug> links resolve against. A tombstoned view does not reserve its
slug, so deleting a view frees the link name too.
Migration is idempotent on both columns and indexes: `deleted_at` is
index=True on the mixin, so a database built from the models already has that
index, and a bare create_index would abort the upgrade on exactly the installs
we most want to keep working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CPU/Memory/Disk gauges read `server.metrics`, the same key the
per-server status endpoint uses (servers.py:757). The list endpoint never
populated it -- to_dict() defaults to include_metrics=False, and even when
asked it emits `latest_metrics` -- so on a live panel every gauge cell fell
back to the "no data" dash.
It survived because the screenshot fixtures DO supply `metrics`
(sections.mjs:294): the UI looked correct in every capture and was empty in
production. The fixture had encoded the intended contract; the backend never
implemented it.
Batched deliberately: Server.metrics is lazy='dynamic', so the obvious
per-row lookup costs one extra SELECT per server. Newest row per server =
highest id (autoincrement on insert), which also avoids the tie a
max(timestamp) join hits for same-second rows -- two queries total,
regardless of server count.
Tests pin all three properties, and each fails without the fix: the key and
field names the table reads, absence (not {}) for a server that never
reported, and that the query count stays flat as servers are added.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An authenticated user of any role could inject arguments into git
subprocesses running as the backend user (root under systemd):
- repo_url was passed unvalidated to git ls-remote/clone, enabling the
ext:: transport (host command execution on git < 2.52 defaults) and
file:// local-repository disclosure (all git versions).
- app_path on POST /deploy/clone was passed without a '--' terminator,
enabling --upload-pack=<cmd> option injection independent of git
version and protocol.*.allow policy.
- branch flowed into 'git fetch origin <branch>' as an unterminated
refspec.
Fix (new app/utils/git_security.py applied at every affected sink):
- repo_url scheme allowlist (https/http/git/ssh + scp-like SSH);
rejects ext::/file://, local paths, leading '-', whitespace.
- '-c protocol.ext.allow=never -c protocol.file.allow=never' pinned on
every remote-touching git call, plus a restrictive GIT_ALLOW_PROTOCOL
in the subprocess environment.
- '--' positional terminator before URL/path/refspec arguments —
co-load-bearing with protocol pinning, which alone does not stop
option injection such as --upload-pack=<cmd>.
- branch/ref validation (leading '-', ref-format metacharacters) for
clone --branch, fetch refspecs and stored deploy configs.
- POST /deploy/clone now confines app_path to the managed apps root,
mirroring how /apps/from-repository derives it server-side.
Adds 31 regression tests; bumps version to 1.7.86.
Reported-by: CaptBoykin
Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
AuditService.get_logs gains a target_id filter (paired with target_type) and
/admin/activity/feed passes both through — the per-resource activity
timeline can now ask 'what happened to THIS app'.
SavedView model (user/page/name/state JSON/is_default, unique per
user+page+name), idempotent 082 migration, saved_view_service with
one-default-per-page enforcement, and a JWT-scoped /api/v1/views CRUD
blueprint. Covered by 9 endpoint tests.
VALID_TARGET_TYPES is now provider-driven; the wordpress_site branch
moves behind a provider the WP extension registers. ProtectionPanel /
RestoreDrawer read available types from the API; policies whose provider
is absent surface a clear missing-provider state instead of crashing
(plan 52 phase 3 task 13).
Proactive hardening from an internal audit of the same broken-authorization
class as GHSA-6w78-q5vm-rfmh (authentication checked, app/workspace-level
authorization skipped):
- deployment_jobs: job detail/logs/retry resolve the job and gate on
can_access_app / can_operate_app (app-less jobs: requester or admin);
global job listing is admin-only
- secrets_webhooks: every vault/secret/webhook-endpoint/delivery by-id
route resolves to its workspace and 404-seals non-members; secret
reveal and signing-secret regeneration require workspace owner/admin;
dead local admin_required placeholder removed
- shared_resources: variable-group CRUD and attach/detach gate on the
resolved scope (workspace role / can_edit_app / can_operate_app),
closing deploy-time env injection into foreign apps
- queue_bus: message receive/complete/fail/list on system-owned groups
are admin-only; user-owned groups restricted to their owner
- docker: raw container inspect (exposes env vars) and container logs
are admin-only, matching Decision 7; per-app logs stay app-gated
- databases: raw SQL query endpoints, host-level metadata reads, and
process lists are admin-only; per-app managed database flows unchanged
Adds 4 persona-matrix regression suites (74 tests) and bumps the
test-count ratchet.
GHSA-6w78-q5vm-rfmh: four git deployment/webhook read endpoints checked
authentication but skipped app-level authorization, letting any active
user (even a foreign viewer) read cross-tenant deployment history and
logs that can contain secrets from pre/post-deploy scripts.
- git.deployments/<id>: resolve deployment -> app, 404/403 via
app_access_tier before returning any log bytes
- git.deployments/webhook/<id>: gate on _webhook_visible_to (app-linked
webhooks follow app visibility; unlinked stay panel-wide, plan 29 #11)
- deploy.history & deploy.webhook-logs: app_id queries require
app_access_tier; global (no app_id) queries are panel-admin only
Adds backend/tests/test_git_deployment_authz.py (9 persona-matrix
regression tests) and bumps the test-count ratchet.
Co-authored-by: Tyler Boykin <25166954+CaptBoykin@users.noreply.github.com>
- install_profile_service: record what install.sh put on the box
(minimal/standard/full), recommended from detected hardware and
overridable; thresholds mirrored by recommend_profile in install.sh
- setup wizard: replace the hard ResourceGate/SetupStepTier with
SetupStepCapacity + SetupStepSecurity
- ResourceAdvisory: inline capacity warning at the point of action
(WordPress/Docker/Services/Deployments), backed by headroom.fits
- RequiresDocker component for docker-dependent surfaces
- install.sh/update.sh: profile detection + tests
(test_install.sh, test_update.sh, test_server_capacity.py)
_maybe_attach_certificate throttled on last_check_at, which is the probe clock.
An active monitor is checked every 30s, so that condition was never older than
the 6h refresh window: the certificate would be read once and then never
refreshed, leaving a stale expiry date on the detail page forever.
Adds cert_checked_at (migration 081) and throttles on that instead. The stamp
is written even when the read fails, so an unreachable TLS endpoint is retried
on the same cadence rather than on every single probe.
Also serves the Monitors list sparkline from one bulk query per page instead of
one per row, bounded by a time window so a busy monitor's history cannot crowd
out a quiet one's the way a single global LIMIT would.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A StatusComponent was already a full synthetic check (http/tcp/dns/smtp/ping,
target, interval, timeout, HealthCheck history, uptime windows, auto-incidents)
— it was just framed as "a component of a status page" and its engine lived
inside the serverkit-status extension, so watching a site required publishing a
status page. Promote it:
- migration 081 makes status_components.page_id and status_incidents.page_id
nullable, and adds the columns backing the monitor UI (is_paused,
check_method, expected_status, keyword, follow_redirects, verify_tls, retries,
consecutive_failures, cert_issuer, cert_expires_at)
- new core app/services/monitor_service.py owns the check engine; the extension
keeps page publishing and delegates
- new /api/v1/monitors (CRUD, check, pause, history, uptime buckets, incidents)
- new builtin.monitor_check sweep on a 30s tick — until now checks only ran when
someone pressed "Check now", so a configured monitor never noticed an outage
Check-engine defects fixed while moving it:
- ping ignored the process return code and could only ever report 'up'
- ping used Linux-only flags with no guard, breaking the Windows dev server
- http ignored its own method/expected_status/follow_redirects/verify_tls
- smtp was documented on the column but unimplemented; keyword did not exist
- run_check never recomputed uptime and never opened or resolved an incident —
only the WordPress health-sync path did, so network monitors never paged
- TLS issuer/expiry are now captured (throttled to one read per 6h)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-ups from the GHSA-4wqh-7f4f-5qmx / GHSA-mc93-rc3x-fpgq review:
- File Manager read endpoints (browse/info/read/search/disk-*/analyze/
type-breakdown/download + s3/browse, s3/read, s3/download-url) only
required a valid JWT, so a user whose files.read permission is revoked
via custom permissions could still read and download files. They now
require permission_required('files', 'read'), matching the write side.
- LogService._read_syslog() interpolated the unit name into a bash -c
string using subprocess.list2cmdline(), which is cmd.exe-style quoting
and does not stop $(...)/backtick expansion under bash. grep now runs
as a plain argv list (with --) and the tail happens in Python.
Reachable only on non-systemd hosts by admins, but the same bug class
as the Postfix advisory.
Regression tests: revoked files.read 403s every read endpoint; the
syslog filter never invokes a shell and passes payloads verbatim.
Mutating File Manager endpoints (write/create/mkdir/delete/rename/copy/
move/chmod/upload and the S3 equivalents) only required a valid JWT, so a
viewer account (files.write=false) could write and delete files. They now
require the files.write permission via permission_required(). Read
endpoints are unchanged. Adds regression tests covering both advisories.
Co-authored-by: AruvasgaChithan <AruvasgaChithan@users.noreply.github.com>
`GET /databases/engines/<app_id>/extensions` carried only `@jwt_required()`, so
any signed-in user could enumerate any instance on the panel — the image it
runs, its databases, and what it can load. All facts about someone else's
application.
It gates on `can_access_app` now, mirroring the per-app docker DB read next to
it: owner, admin, or an explicit grant. Read access rather than admin-only is
deliberate — a developer who can already see an app should be able to see what
that app can load. The POST was already admin-gated and is unchanged.
The file also declares its `# Bucket: PER-APP` header (plan 29 #9), which is
what the structural sweeps read to know a per-app route file has been thought
about at all.
Found by the repo's own authz sweeps in a full-suite run — four failures, one
cause. Worth noting they fired on a route that has no feature test capable of
catching this: every functional test authenticates as someone entitled to the
data. That is precisely what a structural sweep is for.
Four tests pin the behaviour: denied without access, still visible to the owner,
404 rather than 403 for an app that does not exist (conflating them would make
the endpoint useless to its own caller), and 401 anonymous.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The third deferred item. An extension installs INTO a running engine rather than
getting its own container, so it lives in its own namespace:
`backend/templates/extensions/<id>.yaml` with a top-level `extension:` block,
discovered by the same operator-wins two-directory scan the app templates use.
Dropping a YAML in makes it available with no code change, and a test pins that.
Not nested inside the engine template, because pgvector applies to any
PostgreSQL-speaking engine rather than to `postgresql.yaml` — nesting would force
duplication per host. Not a third `kind:` in the app catalog either: an
extension has no compose, port or volume and produces no Application, so every
consumer of `list_all_templates` would have had to learn to skip it.
The hard part is not the SQL. `CREATE EXTENSION vector` is valid and fails on
stock `postgres:16` with "could not open extension control file", because that
image does not ship the library. Two layers, and the statement is never sent
speculatively: an extension declares which images provide it, so a listing marks
an instance unsupported BEFORE anything is clicked; then at install
`pg_available_extensions` is queried first and a negative answer is a 409 with a
remedy, not an attempt. The live probe overrules the image name in both
directions — a private `registry.internal/ops/pg-with-vector` is allowed, and a
`pgvector/pgvector` tag that does not actually carry the library is still
refused.
ServerKit does not recreate a container behind the operator's back. The remedy
instead names an engine template that IS capable, found by matching the
extension's images against each engine template's compose — hence
`postgresql-pgvector.yaml`. TimescaleDB deliberately ships no bundled host and
says so rather than implying one exists.
Compatibility also needed correcting mid-build: matching on wire protocol alone
claimed CockroachDB could host pgvector. It answers on pgwire and cannot load a
single extension. Engines opt in with `engine.extensions: true`, which excludes
it without any Python naming it.
Two fixes fell out. `DatabaseService` had no psql twin of `docker_mysql_execute`
— `managed_db_user_service` carried a private copy with a comment saying as
much; there is now one implementation and it delegates. And that twin sets
`ON_ERROR_STOP=1`, without which psql reports SUCCESS for a statement the server
rejected.
57 tests, covering the zero-code-change property, compatibility, the image
problem in both probe directions, admin gating, and refusals that must not
execute anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
You could not install a database from ServerKit. The tree's four roots were
literals, only MySQL and PostgreSQL reported real state, and no install path
existed anywhere in the backend.
Rather than add one, this makes a database engine an ordinary app template that
declares a top-level `engine:` block — family, wire protocol, admin user, which
variable holds the password, what it calls a "table". Presence of that block is
the only thing that lists a template in the Databases catalog, and installing it
is the template pipeline that already existed: same deployment job, same
streaming console.
The consequence is the point. Adding CockroachDB is adding a YAML file. So is
adding something we have never heard of — author it locally or sync a template
repo — and the install form still works, because it is generated from the
template's own `variables[]` rather than hand-written per engine. There is no
engine model, no engine table, no engine install mechanism to keep in step.
`test_dropping_a_new_engine_yaml_lists_it` is the design; if it ever needs code
to pass, this went wrong.
Twelve engines ship: postgresql, mysql, mariadb, mongodb, redis, valkey,
cockroachdb, clickhouse, influxdb, neo4j, meilisearch, couchdb. Each pins its
image, binds to loopback, names a data volume, and was checked against the
upstream image's actual environment contract — a wrong variable name is a
container that boots with authentication silently disabled.
Two safety notes worth reading. Engines bind 127.0.0.1 unless explicitly
exposed, and the flag is applied after caller variables so a handcrafted payload
cannot quietly publish one. And CockroachDB single-node runs `--insecure`
because password auth needs a certificate-backed cluster: for engines with no
credentials at all the loopback bind is the only control, so public exposure is
refused outright rather than warned about. A test pins the passwordless list to
exactly cockroachdb, so a future template that drops its password variable
fails loudly instead of opening a hole.
Also surfaced at last: `ManagedDatabase.owner_application_id` has been accepted
by the create endpoints for ages with no UI to set it. Creating a database now
offers "Attach to application".
Deferred deliberately: new-table builder, SQL dump import, engine extensions
(pgvector, TimescaleDB) — each a data-mutation surface deserving its own round.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dashboard's layout lived in localStorage as `{id, visible}` — a visibility
toggle wearing a layout's clothes. It carried no geometry, so widget width was
baked into SCSS (`.dash-kpi { grid-column: span 3 }`) and "make the chart wider"
was a code change. And it stayed on one machine.
`DashboardBoard` gives a user real boards: named, ordered, each holding a JSON
array of placed widgets with x/y/w/h and per-instance config. Three defaults
ship — overview, infrastructure, apps & deploys — seeded on a user's first GET
and restorable per board.
Validation is deliberate about the boring cases: geometry must fit the 12-column
grid, ids must be unique within a board, bools are rejected as coordinates
(`bool` subclasses `int`), and a rejected PUT leaves the stored board untouched
rather than half-applied. A board belonging to another user is a 404, not a 403
— there is no reason to confirm it exists.
The shipped defaults are checked by rebuilding their occupancy map cell by cell,
so editing DEFAULT_BOARDS into an overlap fails the suite instead of shipping a
broken first-run layout.
31 tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`contributions.widgets` mounts a component at a host-chosen slot. That is the
wrong shape for the new dashboard, where the user decides what goes on a board
and where. So this adds `dashboard_widgets` alongside it: a registry of
placeable widget TYPES that surface in the widget library, each instance
positioned, sized and configured by the person looking at it.
The two are easy to confuse, so the docstring, the manifest spec and
EXTENSIONS.md all now say plainly which is which, and a test asserts a
dashboard widget never leaks into the fixed-slot list.
SDK_VERSION 1.0.0 -> 1.1.0 (added surface, older extensions unaffected),
bumped in lock-step across the JS export and the backend mirror.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two more of the same fault as the logs: the frontend reading fields the backend
does not send at that level.
Resource Usage showed 0.0% CPU, 0.0% memory and N/A for everything else on a
healthy container. The stats route answers {stats: {...}} carrying docker's own
CLI keys, and the caller read the top level of the envelope — so every field was
undefined and fell through to its zero. It unwraps `stats` now, which is where
CPUPerc, MemUsage, NetIO and PIDs have been all along.
Related Resources sat on "Loading…" permanently because GET /apps/<id>/related
was never implemented. The client has always called it, the 404 was swallowed by
an empty catch, and the card reads null as "still loading". The endpoint now
exists — domains, managed databases, backup policy and deploy count, scoped by
the same access check as the rest of the app routes, each section gathered
independently so a missing table degrades that row instead of failing the card.
The catch also settles to an empty object now, so a future failure shows "no
related resources" rather than spinning forever.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploying checked three things — the name is free, the server row exists, the
directory isn't taken. Nothing asked whether the box could carry the thing, so
a 2 GB app landed on a 1 GB VPS at full speed and the first sign of trouble was
the OOM killer.
Every piece was already here and unconnected: templates describe what they cost
(`requirements: memory / storage`), servers report what they have
(Server.total_memory / total_disk) and what they are using (the newest
ServerMetrics sample). This is the join, answering in the shape a person asks
it — does it fit, will it be tight, or can't we tell?
Advisory on purpose. It reports; it never refuses. An estimate is an estimate,
real usage depends on configuration and traffic, so the operator keeps the
final say and "tight" is a sentence to read rather than a wall to argue with.
A full server still returns valid:true, and a test asserts that so this cannot
quietly become a gate.
Three judgements worth stating:
- The reserve is a floor-and-fraction pair (10% of the box, at least 256 MB RAM
/ 2 GB disk). A bare percentage is wrong at both ends: 10% of 1 GB is
nothing, 10% of 64 GB is more than most apps need.
- `unknown` outranks `ok`. No recent sample, or a template that declares
nothing, says so — a confident wrong answer is worse than no answer.
- Free memory comes from psutil's `available`, not `free`: cache and buffers
are reclaimable, and counting them as used calls every healthy Linux box
full.
Catalog coverage goes from 52/106 to 106/106. The 54 estimates are each app's
typical idle-to-light-use footprint, taken from the project's own documented
minimum where it publishes one and rounded to the values the existing 52
already use — sonarqube 4 GB because that is its documented heap floor, immich
4 GB for the ML models, prometheus 20 GB because retention dominates, traefik
256 MB.
Surfaced two ways: a capacity block alongside (never inside) `errors` on
validate-install, and a read-only GET the deploy drawer can poll while the
operator is still choosing a target, so the answer updates as they switch
servers instead of waiting for a submit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An extension with something worth backing up had to hand-roll it — no policy,
no cron mirror, no run history, no retention, no offsite copy, no verification,
no restore, and no place in the Protection panel. A registered kind inherits
every one of those, because everything downstream of the target_type switches
is already generic: the run lifecycle, retention (count, age, incremental-chain
ancestry), the API blueprint, and the panel itself, which threads target_type
through without ever switching on its value.
No migration: target_type is an unconstrained String, and the allowed set lived
purely in a Python tuple. A kind supplies resolve() and execute(), plus an
optional restore().
Four switches had to learn about the registry, and the order matters in three
of them:
- validate_target_type is the keystone — every policy route funnels through it,
so accepting a registered kind opens the whole blueprint at once.
- _resolve_target and _execute_backup check the registry BEFORE their core
branches, because both end in an application fallthrough that swallows any
unrecognised type.
- run_restore_job likewise. This is the sharp one: without it a plugin's
archive would be unpacked over an app directory by core's else-branch.
A kind that didn't implement restore is refused at request time with a reason,
rather than queueing a job that takes the policy lock and runs a safety backup
before failing.
The registry normalises what plugins hand back — target_type is forced rather
than defaulted, so a kind cannot describe itself as an 'application' and be
restored as one; execute() must return (path, size, meta) with a real path; and
the metadata retention and the panel read is filled in.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dev-gated (DEMO_DEPLOYS_ENABLED) scripted DeploymentJobs — success/fail-build/
long/ansi/slow scenarios driven through RunLogStream so the Deploy Console is
fully exercisable locally with no Docker or server. GET/POST
/api/v1/deployment-jobs/simulate; retry + unified-job wiring; 12 proving tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review feedback + follow-up on the review-stamps feature:
- The 409 trust gate now distinguishes reason 'unreviewed' (community
code not vetted) from 'unverified' (no pinned checksum, possible even
for first_party), each with an accurate message; the risk dialog
carries the reason and matches its title/body to the actual case
instead of always claiming 'unreviewed community extension'
- Unreviewed registry entries are now developer-stage content: hidden
from the Marketplace catalog and refused at the install endpoint
(404) unless the panel runs in a development context (Flask debug /
testing config, or the dev_mode setting). Hidden means not
installable; the 409 acknowledgment flow only exists where they list
- dev_mode setting description mentions the Marketplace effect
- tests: reason assertions on both 409 cases; catalog visibility and
404-on-hidden-install coverage (38 passing)
The registry index (schema v3) can now carry a review object per entry
(reviewer, date, sha256, optional commit/notes) asserting that someone
inspected the exact artifact being listed. The stamp is bound to the
artifact hash: review.sha256 must equal the entry sha256, so any change
to the released zip invalidates the review.
- registry_service: keep review through normalization (validated),
derive trust per entry: first_party, reviewed (stamp matches the
pinned hash), or unreviewed
- marketplace install route: unreviewed (or sha256-less) entries return
409 requires_acknowledgment unless the admin posts
acknowledge_risk: true; acknowledged installs record trust in the
audit log. Consent gate, consistent with ADR 0002
- Marketplace UI: Reviewed badge (reviewer + date tooltip) and
Unreviewed warning badge on registry cards, plus a risk confirmation
modal that retries the install with acknowledge_risk
- tests: trust derivation and the install gate (409 + acknowledge path)
- docs: EXTENSIONS_REGISTRY.md documents the field and the flow
Wire the server half of runtime-frontend delivery: get_active_contributions()
now emits panel sdk_version and a 'frontends' descriptor map {slug: {entry,
hashes, sdk_version}} for every active extension whose frontend_entry is a .mjs
ESM bundle. The bundle sha256 is pinned at install time into the plugin config
under the reserved _frontend_hashes key (no migration — reuses config_json);
update_plugin_config() and the config API preserve/hide reserved keys. Add the
extensions.runtime_frontend kill switch (default on). The client loader already
consumed this shape, so an installed extension's UI now renders on a pre-built
panel with no Vite rebuild. Un-skips the restored plan-25 recovery suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
POST /builds/apps/<id>/deploy now enqueues an app_deploy job and returns 202 +
{deploy_job_id} so the UI opens the live Deploy Console; ?wait=true preserves the
old synchronous contract for CLI/tests (plan 51 D7). Deploy options (no_cache,
version_tag) ride the job plan through to the async runner.
Add POST /deployment-jobs/<id>/retry: clones a FAILED job (both kinds) and
enqueues a fresh run on a new console URL (D8). Extend GET /deployment-jobs/<id>
with ?plan=true so the console can render the step rail.
Tests cover the socket emit helpers (room + payload), the subscribe_deploy auth
guard, the 202/wait contract, retry guards + clone fields, plan-in-GET, and the
streaming-compose variant incl. a failure persisting a real stdout-bearing tail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lets a panel link itself to a master ServerKit panel as a worker WITHOUT
installing the standalone Go agent: the panel registers via the master's
existing token flow, then runs an embedded agent client (HMAC auth + HTTP
long-poll transport) that dispatches agent commands (file:write,
docker:compose:up/ps/logs, system:info/metrics) onto its own local
services. The master needs zero changes — a linked panel is just another
agent-backed Server row with scoped, revocable credentials.
- linked_panel_config table (migration 077) + Fernet-encrypted secret
- embedded agent client: connect/poll/result loop, heartbeat metrics,
per-command threads, allowed-roots guard on file:write
- API: GET/POST/DELETE /api/v1/linked-panel (+ startup resume hook)
- Servers page: LinkedPanelCard (link form, live status, unlink)
- deployment_job_logs.id: BigInteger -> with_variant(Integer, 'sqlite');
BIGINT PK never autoincrements on SQLite, so the first log insert crashed
every fresh install (jobs stuck 'running' at step 0 with no logs)
- migration 076 rebuilds the table on existing SQLite databases
- runner: first status/log/telemetry inside try, non-raising log(),
rollback-safe failure marking — any failure now fails the job visibly
- service: guard enqueue, crash-proof run_job, fail jobs left 'running'
across restarts at boot
- repo-based services (POST /apps/from-repository) now enqueue a real
'app_deploy' DeploymentJob (build + start via DeploymentService.deploy)
and return deploy_job_id
- compose override injects restart: unless-stopped for services without a
policy, so apps come back after reboot
- GET /deployment-jobs gains an app_id filter
- frontend: shared DeploymentJobProgress component (timeout + visible poll
errors) used by the Templates InstallModal and the service Logs tab;
repo wizard navigates to the Logs tab to watch the deploy live; install
success lands on the Logs tab; failure toasts surface real server errors
Introduces a full-stack Test Sandbox feature to run distro matrix tests in Docker from the UI: new backend API/service, sandbox_runs model + migration, run/log endpoints, and a new frontend page with mode selection, run history, live polling, cancel, and per-distro logs.
Also adds ARM64 scripts CI coverage (ubuntu-24.04-arm) and strengthens install/test automation for minimal images and overlay rebuilds: early curl/bootstrap index refresh, stricter venv validation, release-venv glibc fallback, Node reprovisioning on source fallback, rsync --delete overlay behavior, npm bootstrap in VM install, Windows tar pinning in PowerShell harness, plus updated test docs and distro coverage notes.
