The deploy paths already know which host ports are taken (app rows,
slot rows, Docker mappings, a bind test) but only used that to pick a
port silently. Exposing it lets the shared port input say "8001 is
used by shop-api" and offer the next free port, instead of failing
at deploy time.
Vela holds one ServerKit API key as a host-side secret and polls the
fleet list, so this read route moves from bare @jwt_required() to
auth_required() + require_scope('servers:read') — a deliberate,
per-route jwt_only_census exception, not a sweep. JWT callers are
unchanged; a restricted key without servers:read gets 403 from the
scope middleware, no credentials get 401.
Scope choice: the existing servers:read catalog entry already covers
'list servers, agents, and metrics', and the serverkit-gui read routes
(capabilities/frame/surface, sibling repo) take the same scope, so one
key scope serves the whole Vela polling surface.
The census ceiling ratchets down to 600 and the migration inventory is
regenerated, as both ratchets require. tests/test_servers_api_key_scope.py
pins the four cases: scoped key 200, wrong-scope key 403, JWT 200,
anonymous 401.
Session keys must name their purpose, and both loopback legs skip
environment proxies so the grant and bearer token never leave the box.
In the tunnel the API always goes through the relay, which attaches the
grant, and the two raw root anchors become router Links so they stay
under the /t/<device> prefix.
The connect-session route authenticates by the loopback-only Cloud
grant rather than a session, and the tunnel's Socket.IO bridge starts
two raw threads; both guards failed CI until they were recorded.
Until now the connect client refused every http and sio stream, so
Open panel in ServerKit Cloud showed a blank page. The client now
bridges them to the loopback listener, the panel serves its page
under the relay's /t/<device>/ prefix, and the SPA prefixes its
router, API and Socket.IO paths from it. The Cloud grant is checked
against Cloud's JWKS before signing in the panel user with the same
email; panel two-factor still applies.
Plan 87 phase E, compose part. For a compose app on slots, the live
release is a slot project, not the project its own compose file names.
Status maps to the live project only (a stopped standby never reads as
a degraded app), start/stop/restart and sleep/wake act on it, container
listings show the live and standby projects, and an image update runs
as a queued slot deploy instead of recreating the stack in place.
Plan 87 phase C. A compose app on slots runs each slot as its own
project, <name>-a / <name>-b, rendered from the merged compose config:
container_name stripped, only the web service published, on the
slot's loopback port, and every named volume pinned to the name it has
today. Without the pin compose prefixes volumes with the project name
and slot B would start on empty ones; the real-Docker leg checks this.
Stateful services (a database, cache or queue with a named volume)
cannot run twice. They move once, on an explicit previewed split, into
a shared <name>-data project that both slots reach by the same names.
The webhook, template-update and rollback paths deploy through the
engine instead of stopping the stack. The --scale web=2 rolling
restart and the Webhook.zero_downtime flag are deleted: it assumed a
service named web, ignored its own health result, and no UI set it.
Compose slot images get immutable per-deployment tags and are pruned
with the rest.
Plan 87 phase D. Both slots share the database, so what changes it
runs once, before the switch, while the old release still serves:
the app's managed databases are dumped (a warning, never a blocker),
then the release command runs in a one-off container of the new
image. A failed release aborts the deploy with the live slot
untouched. The command comes from the app's setting, the Procfile
release: line, or serverkit.yaml; a rollback never re-runs it.
Rolling code back never rolls the database back: after a switch
back the slot card offers the pre-deploy snapshot as an explicit
second step with a data-loss warning. stop_old_before_release is
the announced-downtime fallback for schema changes that cannot stay
backward-compatible for one deploy.
Plan 87 phase F. A slot deploy job lists its real stages (Preflight,
Build, Boot slot X, Health gate, Switch, Watch) and advances through
them as the engine reaches each one, so PipelineStrip renders them
unchanged. The overview gets a slot card: which release is live,
which is the standby and how long it stays warm, with Switch back;
when slots are off it lists why the app does or does not qualify.
Settings gains the opt-in, watch window, standby minutes and the
shared-volume confirmation.
Plan 87 phase E, single-container part. Start, stop and restart act
on the live slot's container (stopping the app also stops its warm
standby), container listings show the live slot as web and the
standby separately, and status, stats, logs and the terminal follow
app.container_id, which the switch now updates.
A vhost restore point records which slot the captured file pointed
at, read from the file itself, and restoring it starts that slot
first, so a restore never points nginx at a stopped standby.
Plan 87 phase B. An app that opts in keeps two slots, like the
panel updater's two installs. A deploy boots the new release in the
idle slot on its own loopback port while the live one serves, holds
it to the health gate, then switches nginx by moving app.port to the
new slot and rewriting the vhost, so every app.port reader follows
without edits. A watch window switches back on its own if the
release falls over; the old slot stays warm for an instant switch
back, then is stopped (not removed) by a standby sweep.
Opting in adopts the running container as slot a with no restart.
Eligibility is shown with reasons and re-checked on every deploy;
opting out folds the slots back on the next in-place deploy. Slot
images are never pruned and slot ports stay reserved.
Plan 87 phase A. Fixes the deploy bugs that make a rollback or a
health check lie:
- builds get an immutable tag per deployment, so rolling back runs
the image that actually ran instead of whatever :latest became;
old tags are pruned so this is not a new disk leak
- the deployment records the commit after the pull, not before
- the health wait is a real gate: a 4xx fails, it takes three passes
in a row, an exited container fails at once, and it raises
- a per-app deploy lock makes a second deploy wait for the first
- a compose rollback rebuilds and recreates its containers, and a
rollback with no version finds the previous release again
- a failed template update says when restoring the old stack failed
Health path, timeout and the 4xx rule are editable in Settings.
Replica counts could be changed, but nginx only proxied to one port
and the auto-scale sweep was never scheduled, so neither the
Auto-scale section nor its five routes could work; the nginx
upstream builder had no UI at all. With one live copy per app they
are gone, and the scale-policy table is dropped. Running containers
are not touched. The scaling spec now says replicas are not
pursued.
A container with a restart policy that keeps dying looked running.
A one-minute builtin tick counts each app's container restarts;
three within ten minutes opens a crash-loop incident through the
monitor incident path and puts a hint on the Overview, and ten quiet
minutes resolve it. Build-pack workers count too.
Webhooks were the only way to hear a core event, and app.deployed
sat in the catalog without ever being emitted. EventService now
calls registered in-process listeners on every emit (best-effort,
torn down with the extension), and a successful deploy emits
app.deployed with the app's hostnames, which a CDN extension uses to
purge exactly those hosts.
The Docker vhost set no caching at all. An opt-in rule serves files
whose name carries a content hash with a one-year immutable
Cache-Control, proxying exactly like location /. A name only counts
as hashed with 8+ characters and a digit, so app-settings.js is never
pinned. Off by default: the panel cannot know an app's asset layout.
A rule-based check reads the app's request metrics, its container
CPU and the CPU and top queries of the databases it references, and
names what fits: page cache for a busy app over an idle database,
a cache or index when one query dominates a busy database, slot
deploys for errors during deploys, and TTL for a low hit ratio. Each
hint links to the fix and states its failure mode; nothing is added
automatically.
An installed PostgreSQL can switch on a transaction-mode PgBouncer
sidecar through its managed override, with no host port, on the
shared network. fromService gains pooledUrl; apps keep the direct
connectionString unless they switch, because transaction pooling
breaks prepared statements in some drivers, LISTEN and advisory
locks, and the card says so.
A new collector template receives OTLP; Attach → Tracing gives an
app the standard OTEL_EXPORTER_OTLP_* variables, and a Jaeger
attached to the collector as its trace store receives the spans.
ServerKit does not instrument apps; it only wires the path. A docker
leg sends a span from an app and finds it in Jaeger.
The image runs unprivileged and a new named volume at /badger is
root-owned, so badger could not create its directories and every
Jaeger install exited with mkdir /badger/key: permission denied.
It now runs as root inside its own container.
The Prometheus, Grafana and Loki templates installed but connected
to nothing. A Grafana install now takes Metrics and Logs
attachments: the panel creates the data source through Grafana's
API with the recorded admin login, and a metrics attach also imports
a dashboard of the panel's fleet metrics. An attached app joins the
shared network, so Grafana reaches the source by name.
The metrics endpoint only accepted a PROMETHEUS_TOKEN env var that
almost no install sets, and the Prometheus template's config never
reached its container. The panel now keeps its own scrape token in
an internal vault, hands it to templates as SERVERKIT_METRICS_TOKEN,
and the template ships a prometheus.yml with a serverkit job.
The explorer showed live sessions but nothing about which queries
cost the most or whether the cache was working. A container's new
Insights tab lists top queries by total time (pg_stat_statements,
performance_schema digests) with connections in use and the buffer
cache hit ratio, and says why a list is empty. PostgreSQL's
pg_stat_statements is switched on through the config tuner, keeping
existing preloads, so its rollback undoes it.
Installed Redis and a running app were unrelated until someone wrote
the env by hand. The Overview's Attached services card reuses an
installed service or installs one, writes the fromService references
(REDIS_URL, AMQP_URL/BROKER_URL, the S3_* set), shows that kind's
failure mode, and offers a redeploy. Detach removes exactly what the
attach wrote.
RabbitMQ joins the catalog with an amqp:// fromService URL. A
manifest service of type redis now installs the Redis engine under
its own name instead of logging declared but not provisioned, so
its consumers' fromService refs bind to it. The bundled index also
gains the Garage entry the storage commit left out.
Only the Procfile web: line was read, so worker: and scheduler:
never ran. Each now runs as serverkit-app-<id>-<process> from the
same image with the app's env and volumes, no port, and a restart
policy; start, stop and restart follow the web container, and a line
removed from the Procfile goes on the next deploy. Single-container
apps also join the shared service network, which only compose apps
did, and the app's container list now shows the web and its workers.
MinIO's images no longer pull, so object storage is now the Garage
template. Attach storage creates a bucket named for the app and a key
that can read and write only that bucket, keeps the secret in an
internal vault, and wires S3_* and AWS_* env through fromService and
fromSecret. Detach revokes the key and removes the env but keeps the
bucket and its objects. A docker leg proves the scope against a
real Garage.
passive_deletes left removal to ON DELETE CASCADE, which SQLite
does not enforce here (foreign_keys is never switched on), so a
hard-deleted app left its rollup rows behind.
A docker-builds leg runs the bundled Redis template with its managed
override and a separate project that only has the resolved URL; it
answers PING, and without the override it cannot resolve the host.
The probe is redis-py: redis-cli does not percent-decode URL
passwords, while app client libraries do.
fromService gave full connection details only for managed databases;
a Redis or MinIO sibling resolved to a bare http:// URL on a network
the app could not reach. Engine templates now offer protocol URLs and
their parts, any template can declare properties in a connection:
block, and apps on either side join a shared serverkit-services
network through the managed compose override. Secrets come from the
recorded install variables, never the running container.
Each site now sets its own TTL (10s default, 300s cap), and serves
the last good page while the app errors or the entry refreshes, with
one upstream request per missing entry. The cache key carries the
host, so purge clears only this site's files instead of every site's
cache. The settings panel shows the 24h hit ratio.
The route's access check moved into the service, where the per-app
authz sweeps can't see it; it is back in the route. The rollup table
now has a parent-side cascade so a purged app takes its rows, and
reads its histogram through the JSON column mixin.
Nothing configured gzip beyond Debian's stock text/html-only line,
and the .htaccess converter claimed compression was global. A conf.d
snippet now adds gzip (and brotli when the module is loaded), holding
only the directives the running config lacks: a repeated http-level
gzip directive fails nginx -t. A snippet nginx rejects is removed, so
compression can't block a vhost write.
A one-minute builtin tick tails each app's timed access log from a
stored byte offset and rolls it into minute and hour rows: status
classes, cache HIT/MISS/BYPASS/STALE and a latency histogram, so
p50/p95/p99 merge exactly across ticks and into hours. Minute rows
go after 48 h and hour rows after 30 days on the same tick. The
Overview tab gets a Traffic card.
Per-app latency, error rate and cache status were invisible: the
vhosts logged plain combined lines. The serverkit_timed format keeps
the combined fields first so bandwidth accounting and the fail2ban
filters still match, and appends request/upstream time, cache status
and host. Legacy access_log lines read as equivalent for drift so an
upgrade doesn't flag every site.
Adds GET /system/storage to the API surface, records the Storage
tab's form block in the migration inventory, and seeds the history
retention test through make_application.
Deploys had no disk check, so a pull or build on a full disk failed
mid-way. The compose preflight now refuses under 512 MB free, before
any container stops, and warns under max(2 GB, 5% of the disk), both
pointing at Storage.
Host disk alerts were off by default and Doctor's headroom failure
notified nobody, so a box reached 100% silently. A 15-minute builtin
now notifies admins once when the panel host disk crosses 85%, again
at 95%, and resets below 80% (storage.disk_alert_percent, 0 = off).
GET /system/storage measures where the panel host's disk goes (app
images and volumes, build cache, panel database, backups, logs, the
previous install) and returns the retention settings that bound
ServerKit's own share, for the Storage page.
- Panel start writes a logrotate file for the panel's logs and a
default json-file size cap into Docker's daemon.json (merged, never
restarting Docker); the retention tick keeps the newest 10 update
logs and 10 build logs per app.
- The previous blue/green slot is removed once the current one has
been live storage.previous_slot_hours (24), never mid-update.
- Disks under 50 GB get shorter retention defaults once per install;
values the operator changed are left alone.
Juan Denis