Backend CI was the entire pipeline wait. Measured on the last dev push:
Extensions CI 12s
Version Bump 18s
Test System Utilities 35s
Security Scan 41s
Scripts CI 66s
Release Build Smoke 68s
Backend CI 21m 50s <- everything else finished in a minute
Step timings put all of it in the test run itself (install deps 22s, ratchet
9s), so there is no caching win to take — it is 3173 tests against a
function-scoped `app` fixture that rebuilds the Flask app and all 80+ tables
per test. Fixing that fixture is the real repair and is planned separately;
this change buys the wall-clock back now without touching a single test.
Split over 4 runners via pytest-split (794/794/794/791). Sharding rather than
pytest-xdist is deliberate: each shard is its own VM, so the process-shared
state that makes in-process parallelism unsafe here (templates.json, APPS_DIR
— see the per-PID DB dance in tests/conftest.py) simply isn't shared, and no
test code has to change.
The ratchet moves to its own job because it must see the whole suite; a shard
only collects its quarter. It runs beside the shards and costs no wall-clock.
Two details:
- The shard command is scoped to `tests` rather than a bare `pytest`.
Identical in CI (3173 either way, backend/dev-data/ being gitignored), but a
bare pytest on a dev box tries to collect the locally deployed apps under
backend/dev-data/ and dies during collection. The line is now copy-pasteable
for debugging a red shard locally.
- With no .test_durations file, pytest-split balances by test count, not time,
and these tests range from 0.1s to 10s+. If one shard becomes the new
critical path, run once with --store-durations and commit the file.
main also drops out of `push` here: release.yml gates itself on this workflow
via ci-gate, so listing main ran the whole suite a second time on every merge.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`npm run lint` was defined in frontend/package.json and documented in
CLAUDE.md, but no workflow ever invoked it. Grepping the whole workflow dir for
`npm run lint`/`eslint`/`npm run build` returned nothing, so three project-
specific checkers chained behind eslint were silently unenforced:
check-settings-index every Settings tab has a search-index entry
check-theme-tokens the theme-token whitelist stays in 3-way sync
check-html-sinks every raw-HTML sink is sanitized or annotated (XSS)
Lint currently passes: 926 warnings, 0 errors, and all three checkers green —
so this is safe to add as a blocking check today. eslint exits 0 on warnings,
so the gate is on errors only; add --max-warnings to freeze the count if the
warning drift ever needs a ratchet of its own.
Lint only, no build step: the frontend is already compiled in CI by Release
Build Smoke Test, whose scripts/build-release.sh runs `npm ci && npm run
build`. backend/app/** is in the paths because check-html-sinks scans it too,
for `|safe`, `Markup(` and `render_template_string`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These five workflows triggered on both `push: [dev, main]` and
`pull_request: [dev, main]`, so one change was validated up to three times: on
the dev push, again on the dev->main PR, and a third time on the post-merge
push to main.
Narrow them to `push: [dev]` + `pull_request: [main]`:
- `pull_request: [dev]` was dead config. Every PR this repo has had targets
main (checked back through #77, dependabot's included), so it never produced
a run.
- `push: [main]` was redundant for release-smoke: release.yml's build-release
job runs that same scripts/build-release.sh for real, on the same commit,
moments later.
Also drops test-system-utils' `unit-tests` job. It ran
`pytest tests/test_utils_system.py` (44 tests) that Backend CI's bare `pytest`
already collects — there is no pytest.ini, addopts or collect_ignore narrowing
collection. The distro matrix and the raw-subprocess audit stay; those are the
parts Backend CI genuinely cannot do.
Coverage of main is unchanged: a pull_request run tests the merge result, and
release.yml still gates itself on the full backend suite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
10 from the template repo/sync suite added alongside the DEFAULT_REPOS fix;
the rest accumulated since the last ratchet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reasoning about a redirect validator is how they ship broken, so this was
driven by a probe that asks the URL parser directly: for ~50 candidate
strings, does `new URL(candidate, base)` leave the origin, and does
sanitizeRedirect agree? Zero leaks before or after -- the origin property was
never broken. What the probe did surface was three ways to be wrong that are
not origin escapes:
1. Dot segments were accepted. `/..//evil.com` normalizes to `//evil.com`,
which stays same-origin only because it is resolved against a base;
assigned to window.location.href it leaves the site. Rather than audit
every present and future consumer of the stored value, the gadget is now
refused outright -- plain and percent-encoded, in any position. A dot
inside a segment name (/files/.env) is untouched.
2. Auth routes were matched as exact strings, so /login/ and, worse,
/login/callback/<provider> got through -- the latter re-runs the SSO
callback with no code and lands the user on an error page right after a
successful login. Now a prefix match on a "/" boundary, case-insensitive,
which still leaves /logins and /login-help as valid destinations.
3. No expiry. A destination parked early in a tab session fired on any later
login in that tab: park an install link, abandon it, log in normally half
an hour later, get taken somewhere you had forgotten about. Now stamped and
good for 30 minutes -- long enough for a password manager, a TOTP prompt,
or an SSO round trip that includes signing up at the provider.
Also a 2048-char cap, and consume now refuses a missing, non-numeric or
future timestamp rather than trusting it: rememberRedirect is the only writer,
so anything else did not come from this module.
Tests go 9 -> 19, and now cover the remember/consume round trip (single use,
expiry, tampered storage, storage unavailable) against a fake sessionStorage
so the whole module still runs under plain node.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DEFAULT_REPOS pointed at raw.githubusercontent.com/serverkit/templates --
the org `serverkit` does not exist (the registry is jhd3197/serverkit-templates,
and its default branch is master, not main). So it has 404'd for its entire
existence: no panel has ever fetched a template remotely, and every /templates
page has been showing only the 118 bundled backend/templates/*.yaml.
Now points at https://serverkit.ai/templates, which proxies the registry and
was built for this consumer -- it serves <repo_url>/index.json and the
<repo_url>/templates/<id>.yaml path this class derives, behind a TTL cache
with last-good fallback. Using the product domain also means a branch rename
upstream cannot silently empty every panel's catalog, which is the exact
failure this is fixing.
Fixing the default alone would only help fresh installs, because get_config
returns whatever a saved templates.json holds. Known-dead URLs are therefore
healed on read. Nothing is lost by rewriting a URL that never once resolved.
The checksum half is here rather than in a follow-up because this commit
ACTIVATES a download path that has never run: sync_templates wrote whatever
came back straight to disk, ignoring the sha256 the index pins for every one
of the 106 official entries. Turning on an unverified fetch days after the
panel learned to verify ed25519 signatures on extensions would be shipping a
known downgrade. Mismatch is now a hard refusal that never reaches disk;
absent hash is allowed and counted, mirroring unsigned-vs-invalid for
extensions. Content is written as bytes so the file on disk is exactly what
was hashed.
Note get_template's in-memory remote read (it parses without saving) is not
covered -- it has no index entry on hand and would need an extra fetch.
DEPLOY ORDER: serverkit.ai must ship the master-branch fix before this helps.
Until then the proxy 502s and sync finds nothing, same as today.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The counterpart to Templates.jsx's ?install=<id>, which has worked for a
while; the extensions page never grew one, so serverkit.ai install links and
README badges had nowhere to land.
It opens the extension's detail modal rather than installing. A URL arriving
from another site must not be able to install anything on its own -- the
operator still presses Install, and the trust gates behind it (unreviewed,
unverified checksum, untrusted publisher key) all still fire exactly as they
do from a card click. An unknown slug says so instead of failing silently.
Deliberately reads the param on /extensions and not /marketplace: the old
path resolves through a <Navigate>, which drops the query string, so a link
to /marketplace?install=x would arrive with nothing to act on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PrivateRoute redirected to /login with no record of where the visitor was
going, and every post-auth path then navigated to '/'. Any deep link opened
without a session was silently discarded -- you landed on the dashboard with
no explanation of why the thing you clicked did not happen.
That was survivable when deep links were rare. It is not now: serverkit.ai
install links (/extensions?install=<slug>, /templates?install=<id>) arrive
from README badges, and by construction they are clicked by people who may
have no open session. Every first-time click hit this path.
The destination goes in sessionStorage rather than react-router location
state because the SSO flow leaves the origin entirely for the identity
provider and returns through /login/callback/<provider>; router state cannot
survive that. One mechanism covering all four post-auth exits (login link
redemption, password, 2FA, SSO) beats two each covering half.
sanitizeRedirect is the pure half and is validated on write AND on read.
Login is exactly where an open redirect hurts, so `//evil.com` and
`/\evil.com` -- both of which browsers read as protocol-relative -- are
rejected along with control characters and the auth routes themselves,
which would loop. Proving test: node --test
src/utils/__tests__/redirectAfterLogin.test.mjs (9 tests).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Piped execution leaves BASH_SOURCE unset, so the sourceability guard's bare
${BASH_SOURCE[0]} was an unbound variable under `set -u` and aborted the
install before main() ever ran:
bash: line 2107: BASH_SOURCE[0]: unbound variable
This broke the one-liner README/DEPLOYMENT document, on every distro — not
just the Debian 13 it was reported on (reproduced on debian:13 bash 5.2.37,
ubuntu:22.04 bash 5.1.16, almalinux:9 bash 5.1.8). CI never saw it because
every job runs `bash install.sh` from a checkout, where BASH_SOURCE IS set.
Use the same non-empty-first guard scripts/install.sh already carries, and
add the proving test the panel installer was missing (T41) — the agent
installer has had its twin since the same fix landed there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The full WordPress suite now lives at github.com/jhd3197/serverkit-wordpress
as a standalone runtime-ESM extension (backend blueprints/services, game
dashboard UI, tests, CI). Installed from the registry or offered during
onboarding from the bundled snapshot. WP data models stay core per D1
(documented core-data-seam).
Runtime-ESM extensions (starting with serverkit-wordpress) need slot
mounting and error containment from the host; additive, versioned in
lockstep frontend/backend.
Create/update validation fails loudly naming the missing extension
instead of orphaning monitors; health sweep no-ops with zero WP sites
(plan 52 phase 3 task 17). StatusComponent.wordpress_site_id stays core
per D1.
The wordpress template kind + wordpress-external-db preflight move behind
a template-provider hook the extension supplies; the backend filters the
catalog so the WP card simply does not appear when WP is absent — no
frontend change needed (plan 52 phase 3 task 16).
Extensions register event types at load; wordpress.* types now come from
the WP extension instead of a hardcoded core catalog (plan 52 phase 3
task 15). emit_wp stays core as a dumb emitter helper.
VALID_TARGET_TYPES is now provider-driven; the wordpress_site branch
moves behind a provider the WP extension registers. ProtectionPanel /
RestoreDrawer read available types from the API; policies whose provider
is absent surface a clear missing-provider state instead of crashing
(plan 52 phase 3 task 13).
Extensions can register providers into core engines at load (backup
targets, event types, template kinds) via a core_hooks module; idempotent,
process-lifetime like blueprint registration. WP extension supplies its
entries through builtin-extensions/serverkit-wordpress/backend/core_hooks.py
(plan 52 phase 3, D4).
App.jsx /wordpress* title fallbacks removed (manifest page_titles cover
the routes); sidebar preset hiddenItems no longer hardcode the wordpress
slug — nav sources from extension contributions.
5 WordPress* pages + components/wordpress (40 files) + 3 _wordpress*.scss
partials move from core into builtin-extensions/serverkit-wordpress/
frontend/ (synced to frontend/src/plugins via the baked-glob pattern).
200 imports rewritten to serverkit-sdk; remaining @/ imports only for
genuinely host-internal modules. main.scss drops the WP partials — the
extension bundle imports its own styles. npm build + lint green.
TabGroupLayout, Modal, ConfirmDialog, Spinner/LoadingState, EmptyState,
Skeleton(SkeletonBoundary), and the ui/* form kit (Button, Input, Select,
Switch, ...) as generic re-exports so an extension page can mirror core
page chrome without deep imports. Proven out by the WordPress frontend
relocation (plan 52 phase 4).
Unknown categories already derived/filtered gracefully; this adds
games to the canonical order and gives it a lucide Gamepad2 icon
ahead of serverkit-minecraft landing in the registry.
The four minecraft test files now live in serverkit-minecraft/tests/
(panel-hosted via SERVERKIT_MINECRAFT_DIR loader). Ratchet floor
3280 -> 3197 (-83 minecraft tests; collected 3222, floor intact).
The Minecraft game-server extension now lives at
github.com/jhd3197/serverkit-minecraft as a standalone runtime-ESM
extension (analytics/gpu repo pattern). It is installed from the
registry like any first-party extension, no longer bundled in-tree.
scripts/sign-extension.mjs signs a bundle with an ed25519 author key;
EXTENSIONS.md documents the format, key model, and verdict behaviors.
Generated private keys live in gitignored scripts/keys/.
Consent card shows permissions, version gate, and signature state
(Signed/Unsigned/Unknown signer/Invalid) with honest copy; 'Install
anyway' destructive path for acknowledged risk; invalid = disabled.
Proactive hardening from an internal audit of the same broken-authorization
class as GHSA-6w78-q5vm-rfmh (authentication checked, app/workspace-level
authorization skipped):
- deployment_jobs: job detail/logs/retry resolve the job and gate on
can_access_app / can_operate_app (app-less jobs: requester or admin);
global job listing is admin-only
- secrets_webhooks: every vault/secret/webhook-endpoint/delivery by-id
route resolves to its workspace and 404-seals non-members; secret
reveal and signing-secret regeneration require workspace owner/admin;
dead local admin_required placeholder removed
- shared_resources: variable-group CRUD and attach/detach gate on the
resolved scope (workspace role / can_edit_app / can_operate_app),
closing deploy-time env injection into foreign apps
- queue_bus: message receive/complete/fail/list on system-owned groups
are admin-only; user-owned groups restricted to their owner
- docker: raw container inspect (exposes env vars) and container logs
are admin-only, matching Decision 7; per-app logs stay app-gated
- databases: raw SQL query endpoints, host-level metadata reads, and
process lists are admin-only; per-app managed database flows unchanged
Adds 4 persona-matrix regression suites (74 tests) and bumps the
test-count ratchet.
Juan Denis