The SSL-enable and server-overview changes put session and query
calls in the routes, tripping the controller-boundary ratchet. Move
them into SiteDomainService and WorkspaceService, raise a typed error
for bad allowed_ips, point the ACME-contact test at the new vhost
door, and regenerate the baseline and inventory (476 -> 453).
Enabling SSL on a custom domain edited the vhost in place, but every
later rewrite (deploy, slot switch, domain attach, drift repair,
restore) rebuilt it from Domain rows that only knew wildcard certs,
so HTTPS silently disappeared while the row still said enabled.
Rebuilds now resolve each domain's own cert and render one server set
per certificate when domains differ. Enable/attach publish through
write_app_vhost and only claim SSL when nginx accepted it, the key
path is stored under its real column (legacy rows fall back to
certbot's privkey.pem), disable drops the HTTPS block, and a repeat
add_ssl_to_site swaps the cert instead of breaking nginx -t.
328 more routes used a JSON body as a dict straight from
request.get_json(), so a body of null, [] or a bare string crashed them
into a 500. Bare reads become json_object(); the
'get_json(silent=True) or {}' reads of action routes become
json_object(optional=True), which still treats a missing body as {}.
Routes that take arrays or pass the body through untouched are left
alone. The full backend suite shows no new failures.
Also carries three files' bare section citations (apps, backups,
notifications) and the extended citation guard, which needed them.
Eleven routes passed request.get_json() straight to data.get(...), so
a body of null, [] or a bare string crashed them into a 500. They now
read the body through json_object(), which answers a typed 400.
Comments, docstrings and a few strings cited plan numbers ("plan 29
#9", "plan 86 §B2") from a git-ignored folder, so they pointed at
nothing; one reached users in a bottleneck hint. Code and frontend
strings are unchanged apart from those citations (checked by AST and
token comparison). A vhost still carrying the old asset comment reads
as undrifted, and a guard test keeps new citations out.
Toasts showed backend prose verbatim, with no code to translate
against and "application" where the UI says "service". Not-found,
missing-field, permission, conflict and agent-offline errors now come
from factories in app/exceptions.py that pair one fixed message with
a dotted code (not_found.<resource>, validation.required,
permission.denied, conflict.exists, agent.offline), so the client can
translate them.
The shared app-loading helpers and the apps, domains, servers,
databases and backups routes raise those errors instead of hand
shaping bodies, which drops the hand-shaped body count from 1132 to
850. Five routes picked an HTTP status by comparing error text; they
now key off a result code or a sentinel so rewording can't change a
status. Five broad excepts that echoed str(exc) now log the crash and
return the request id instead.
Plan 73 item 5. The Domains SSL modal asked for a contact email on every
single certificate, and the backend hard-required one per call. The app-detail
Settings tab did remember it -- in localStorage, so the memory was per browser
and invisible to the other surface. The two never agreed, and neither could
help the paths with no form at all: domain attach passed `admin@<host>` when
it had nothing else, an address that usually does not exist, which is where
every expiry notice for those certificates went to die.
There is now a single panel-wide ACME contact in system settings.
get_acme_contact() resolves an explicit address first and falls back to the
stored one; remember_acme_contact() writes it back whenever a request carries
an explicit address. Both are best-effort on the settings side by design: a
settings failure must not break issuance, and failing to remember an address
must not turn a successful certificate into an error. An empty input never
clears a stored contact, since the automatic paths pass whatever they were
given and that is frequently nothing.
Wired into every issuance entry point -- the domain SSL enable route, the
generic certificate route, and domain attach, which now consults the stored
contact before falling back to the admin@<host> guess. The address is only
remembered after issuance actually succeeds; remembering one Let's Encrypt
rejected would be worse than remembering none.
Prefill needed its own endpoint. The settings routes are all admin-gated, but
issuing a certificate for an app you can edit is not an admin action, so
GET /ssl/acme-contact is jwt_required only -- it returns an address the
operator chose for notices, not a credential. The Domains modal and the
Settings tab both prefill from it, and the Settings tab keeps its localStorage
seed so a browser that already has the value shows it without a flash.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 73 item 7. Let's Encrypt prefers IPv6 whenever a AAAA record exists, so a
domain whose A record points at this server perfectly still fails HTTP-01
issuance when its AAAA points somewhere else. That shape looked healthy
everywhere in the panel, which is what made it expensive: the operator saw a
verified domain, a green doctor report, and a certificate error naming nothing
either of them had mentioned.
Both DNS surfaces were blind to it for different reasons. The doctor's sweep
returned ok as soon as the server IP was *among* the resolved addresses, so
the extra IPv6 answer never registered -- and that branch is reached precisely
when the A record is correct, i.e. exactly the case worth reporting.
verify_domain used socket.gethostbyname, which is IPv4-only and cannot see a
AAAA record at all.
Both now compare every resolved address against this server: the configured
public IP plus the host's own interface addresses, so a correct dual-stack
setup stays quiet and only a genuine stray is named. Comparison is by parsed
value rather than by spelling, since an expanded 2001:0db8:... form and its
compressed equivalent are the same address, and psutil's zone ids (fe80::1%eth0)
are stripped first. Interface enumeration happens once per doctor sweep
instead of once per domain.
The doctor reports this as a warn, not a fail: the site itself is reachable
and it is a stray record rather than a broken one. Both messages say what to
do -- remove the AAAA record or point it at this server's IPv6 address --
rather than only naming the symptom.
verify_domain keeps its existing response shape. ip_address still carries a
single IPv4 address for the callers that predate this, with the full set added
alongside as ip_addresses, plus aaaa_conflict and a warning string. The
Domains page needed the last part to matter: it printed a green "Domain
verified!" toast for anything that resolved, so a conflict the API now
reports would have gone straight back to being invisible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.
**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:
DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed,
cron suspended, row tombstoned. All reversible.
RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left
STOPPED -- "put it back" must not silently start processes or open
ports, the same reasoning that stops domain restore re-issuing certs.
PURGE volumes removed, uploaded source deleted, cron detached.
register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.
A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.
**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.
**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().
Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.
Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.
Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Making Domain soft-deletable silently changed the meaning of every existing
`Domain.query`: before, it could only return live rows. An audit of all 16 call
sites found ten that were now wrong, several with side effects outside the
panel.
The ones that reached the outside world:
setup_reconcile_service.py:267 the only is_primary row after deleting the
primary IS a tombstone -- and it drives a WP-CLI search-replace across
the whole WordPress database. It would have repointed a live site at a
host nginx no longer serves.
setup_reconcile_service.py:83 DNS backfill enumerated deleted domains and
called ensure_a_record() -- creating a real record at Cloudflare/Route53
for a domain the user removed.
api/domains.py:367 (enable_ssl) a real ACME order for a host that is in no
server_name: the HTTP-01 challenge cannot be served, burning Let's
Encrypt failed-validation limits for the whole registered domain, and on
success installing a wrong-CN cert on the LIVE vhost.
doctor_service.py:267 permanent red "does not resolve" rows that no
action could clear, and the same set is the allowlist for one-click DNS
repair -- so Repair created a provider record for a deleted domain.
And the ones that broke the feature itself:
domain_attach_service.py:79 a tombstone counted as a name clash, so you
could not move a hostname to another app after removing it. Migration 083
made the unique index partial precisely to free the name; this re-imposed
the burn in application code.
domain_attach_service.py:71 attach found the tombstone, reported success,
created no live row, published nothing -- and then mutated a recycle-bin
record.
site_domain_service.py:600 same for give_subdomain: success, a URL, a DNS
record, and a site that was never published.
api/domains.py:532 a second vhost renderer (admin Regenerate)
writing tombstones back into server_name.
Application.domains still includes tombstones ON PURPOSE -- filtering it with a
primaryjoin would, with delete-orphan, make SQLAlchemy HARD-delete the row the
recycle bin exists to keep. `Application.live_domains` is the accessor for
"what this app actually serves", and to_dict() uses it so the /apps payload no
longer lists deleted domains.
Per-domain routes now 404 a tombstone instead of editing it.
Seven tests cover the leaks that had real consequences.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
create_domain and delete_domain each called NginxService.create_site directly,
passing no ssl_cert / ssl_key / micro_cache. create_site takes those as
arguments, so the regenerated vhost came out HTTP-only and cache-less: on a
site covered by a base wildcard cert, touching ANY domain silently knocked the
whole site — every other domain on it — back to plain HTTP, and dropped its
micro-cache directives. Invisible until someone loaded the site.
Both now publish through SiteDomainService.write_app_vhost via one local seam,
so add and remove cannot disagree about what the served config should be. That
also fixes two lesser asymmetries: the rewrite was gated on app_type == docker,
so php/static/flask/wordpress vhosts kept advertising a domain after it was
removed; and it never called enable_site.
ALSO FIXES A REGRESSION I INTRODUCED WITH SOFT DELETE. app_vhost_kwargs read
`Domain.query`, which before soft delete could only return live rows and now
returns tombstones too. The vhost renderer would have written a deleted domain
straight back into server_name — the domain would keep being served after
deletion, and drift detection (which renders through this same function) would
report correct config as drifted. Now query_active().
Three tests, each failing against the previous code: the wildcard cert and
micro-cache survive in the resolved kwargs, a soft-deleted domain is absent
from them, and an end-to-end delete leaves the surviving domain on HTTPS.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting is the only action in a control panel that repeating cannot undo, so
records a person destroys by hand now leave a tombstone and the Recycle Bin
hands them back.
SoftDeleteMixin adds deleted_at/deleted_by_id plus soft_delete/restore and
query_active()/query_deleted(). recycle_bin_service is a REGISTRY: a model
registers with a label function and an optional on_restore hook, so the API
lists and restores across types without knowing what a domain is. Adding the
next soft-deletable model is one register() call.
Applied to Domain (a real, destructive case -- deleting one used to lose the
record outright) and SavedView. DELETE /domains/<id> now tombstones and still
tears the vhost down; restoring a row does NOT re-do side effects unless the
type supplies on_restore, which is the honest default.
API: GET /recycle-bin (all kinds, newest first), POST
/recycle-bin/<kind>/<id>/restore, DELETE /recycle-bin/<kind>/<id> (purge,
admin-only -- the one irreversible action), POST /recycle-bin/purge-expired
with a 30-day default retention.
THE PART THAT NEEDED CARE: both tables had a plain UNIQUE covering what is now
a tombstone, so deleting a domain would have permanently burned its name and
deleting a view would have blocked re-creating one by the same name. Migration
083 replaces each with a PARTIAL unique index predicated on deleted_at IS NULL
-- supported by SQLite 3.8+ and Postgres. Tests pin exactly that: a deleted
name can be reused, and the live row stays unique.
Also gives SavedView a `slug` (backfilled), the handle the upcoming
?view=<slug> links resolve against. A tombstoned view does not reserve its
slug, so deleting a view frees the link name too.
Migration is idempotent on both columns and indexes: `deleted_at` is
index=True on the mixin, so a database built from the models already has that
index, and a bare create_index would abort the upgrade on exactly the installs
we most want to keep working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Wire the base-domain registry into HTTP + the cert flow:
- SitesHttpsService.setup(provider_id, email, base) now issues the wildcard cert
for a specific base domain and persists HTTPS state to THAT base's registry row
(recording the provider used), so example.com on Cloudflare and toto.com on
Route53 each get their own cert; falls back to the legacy sites_* settings for
a single-base install. status() now also returns the full `bases` registry.
- Admin API: POST/DELETE /admin/sites-https/base-domains[/<domain>] and
.../<domain>/default to register, remove, and set the default base; setup
accepts an optional `base`.
- GET /api/v1/domains/base-domains lists the bases a new site can use (registry,
or the single legacy base) with the pre-selected default — the create-flow
picker's data source. give-subdomain + suggest-subdomain accept a `base`.
- WordPress create accepts baseDomain end to end (API -> service -> subdomain_for),
and picks the covering base's HTTPS state for the baked-in site URL.
Proven in test_site_base_domains.py: the list endpoint (registry + legacy),
admin add/set-default/remove, and per-base HTTPS setup persisting to one row
while the other base stays HTTP. WordPress suite (66) still green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#12 SiteDomainService.give_subdomain(app, label): one-click publish an app at
<slug>.<base> — primary Domain row + nginx vhost + (per-site mode) an auto-created,
guarded/logged A record. Endpoints: POST /domains/give-subdomain and
GET /domains/suggest-subdomain.
#14 GET /api/v1/dns/managed: every record ServerKit owns across all provider zones,
enriched with the app that triggered each (from the ownership ledger).
#13 inline propagation reuses the existing /dns/propagation/<domain> check.
Tests: test_dns_give_subdomain.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Add app/utils/formatting.py with format_bytes() and delegate 9 duplicated
human-readable byte formatters across services.
- Import admin_required from app.middleware.rbac in 15 API modules instead of
redefining it locally.
- Extend app/utils/slug.py with validate_app_name() and use it for template
install/validation and apps slug normalization.
Finish #33's two named workspace-scoping slices, backend-only (no schema
migration, no frontend changes):
- Domains: get_domains scopes through the parent Application via scope_query
(own + granted, workspace-narrowed); also surfaces granted apps' domains,
closing a gap vs the per-domain routes. Admin-no-context path unchanged.
- Databases: the app-linked /docker/databases enumeration scopes via
scope_query; host-level MySQL/PG/SQLite stay global system resources
(documented inline).
- Role reconciliation: WorkspaceService.effective_role + can_write_in_workspace
intersect global User.role with per-workspace WorkspaceMember.role,
narrow-only (a workspace role can only reduce capability; system admin never
capped). Enforced at apps/servers create + reassign seams (viewer -> 403,
only when a workspace context is active). Reconciled role surfaced on
GET /workspaces (my_role/my_effective_role).
6 new tests; full backend suite 193 passed / 12 skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A grant now reaches every app-related blueprint, not just apps.py + WordPress, so
a shared application is fully manageable by its grantees.
- Promoted the access helpers to a shared seam: ResourceGrantService.can_access_app
(owner/admin/any grant -> reads) and can_edit_app (owner/admin/editor grant ->
mutations); apps.py now delegates to them.
- Swept 33 ownership checks across 6 blueprints (categorized read vs mutate via a
parallel mapping workflow): builds.py (12), domains.py (7, incl. SSL + domain
CRUD), deploy.py (6), private_urls.py (5), templates.py (2), databases.py (1).
Reads -> any grant; mutates -> editor grant.
- Fixed the same latent int-vs-str ownership bug in all 33 (now user.id).
Deferred: python.py's single get_app_or_404 helper gates read+mutate+delete
together and needs per-method splitting before it's safe to make grant-aware.
Verified: cross-blueprint integration test (grantee reaches builds deployments;
stranger 403), full suite 131 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces handling of a 'files' section in templates, allowing creation of files and automatic updating of docker-compose.yml to use bind mounts for these files instead of named volumes. Also updates nginx config generation to avoid using 'default_server' for private URL servers, delegating that responsibility to serverkit.conf.
- Require Docker apps to have a port configured before adding domains
- Check port accessibility and include warning if port not reachable
- Provides actionable error messages with hints for resolution
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New endpoints for debugging domain/container routing:
- GET /api/v1/domains/debug/diagnose/{app_id}: Full diagnostic report
with app config, domains, nginx status, docker containers, health
assessment and actionable recommendations
- POST /api/v1/domains/debug/test-routing/{app_id}: Active routing
tests to verify traffic flow
Both endpoints require admin access.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Automatically create, update, and delete nginx configs for Docker applications when domains are added or removed. Adds an endpoint to manually regenerate nginx configs for Docker apps. Updates TemplateService to detect port from multiple possible variable names (PORT, HTTP_PORT, WEB_PORT) when creating Docker app records.
Introduces new API endpoints and services for backups, monitoring, and Git deployment. Adds security middleware, account lockout on failed logins, and improves domain/SSL management. Updates Dockerfiles, deployment configs, and frontend to support new features and UI components.
Add backend and frontend foundations for ServerKit, including Flask API structure, SQLAlchemy models, service layers, and React app with authentication context and service hooks. Implement core API endpoints for authentication, application management, Docker, databases, and system operations. Update README with project overview, tech stack, and setup instructions. Add development roadmap and example environment configuration.
Juan Denis