The new Git deploy tests built Application rows by hand, which the
seed ratchet forbids; they use make_application now. A test that
enables foreign keys could leave a pooled connection enforcing them,
failing unrelated tests depending on shard order; every checkout now
starts with enforcement off.
test_buildpacks.py asserts on Dockerfile TEXT, which cannot see a bug that
only fails at build or boot time -- the `--omit=dev` dead fallback shipped
`vite: not found` to a user's deploy. New @docker_builds leg (gated on
SERVERKIT_DOCKER_BUILDS=1, sibling of @real_binaries) runs the full path
per stack: detect() -> generate_dockerfile() -> `docker build` ->
`docker run` -> HTTP 200. Fixtures: vite (the regression), express with no
build step (prod-only install must still boot), flask, static. Wired as a
backend-ci job beside real-binaries.
Writing the flask fixture surfaced the same dead-at-boot class one layer
up: detection invents `gunicorn app:app` / `uvicorn main:app` as the start
command but never installed the server, so a repo that doesn't declare
gunicorn built green and died at boot. The build command now installs the
invented server (a no-op when the repo already pins it); Procfile-authored
start commands remain the author's contract.
All four stacks verified locally: build + boot + HTTP 200 in 53s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
builtin-extensions/serverkit-cloudflare-ops + the tracked frontend live
copy removed; FLAGSHIP_SLUGS is now empty (machinery kept);
plugins-manifest.json empties; the bundled registry entry flips to
bundled:false with the local zip's sha256. cloudflare_ops_bridge resolves
the copy-installed package FIRST so a registry install works without the
builtin registration path.
Tests: new conftest.mount_sibling_extension mirrors a real install from
the sibling checkout (the _wp_support.ensure_plugin analogue) and a
cf_extension fixture forces a private app; the zone-settings and round2
suites mount it autouse (74/74 green), the extraction suite pins the
lean-panel shape, and the flagship-era onboarding tests prove the
machinery with a fabricated member instead.
OPERATOR: publish jhd3197/serverkit-cloudflare-ops, sign,
sync-registry-index.sh.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
builtin-extensions/serverkit-email + both tracked live copies removed; the
bundled registry entry flips to bundled:false with the local zip's sha256;
the gated auto-install (_email_was_configured) moves from
GATED_BUILTIN_SLUGS to REGISTRY_GATED_SLUGS so mail users still get it back
once the entry is published.
Panel-core tests re-home onto the sibling checkout via the new
conftest.sibling_extension_dir helper (skip cleanly when absent): the
extraction suite installs from the serverkit-email repo, the dovecot-probe
suite loads its module from there. All 39 affected tests green with the
sibling present.
OPERATOR: publish jhd3197/serverkit-email, sign, sync-registry-index.sh.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The panel's crontab/nginx code only ever met the fake_subprocess stub —
the 8-distro matrix runs stubs and the container legs run 12 detection
tests, so a syntax the real parser rejects failed on a user's box, not
in CI (the #117 class).
- new `real_binaries` marker (registered in conftest); the tests also
gate on SERVERKIT_REAL_BINARIES=1 so ordinary suite runs never mutate
a machine's crontab by surprise. Local loop (WSL works):
SERVERKIT_REAL_BINARIES=1 pytest tests -m real_binaries
- test_real_crontab_roundtrip: add -> list -> toggle -> remove through
the real `crontab -`/`crontab -l`, operator lines preserved, name
injection blocked; user crontab saved/restored around each test
(verified green on WSL against the real binary)
- test_real_nginx_vhost_syntax: every render_site_config flavor through
a real `nginx -t` in a tmp wrapper conf, plus a garbage probe so the
harness can't go vacuous
- backend-ci gains a real-binaries job (ubuntu runner, apt nginx+cron)
- scripts-ci gains the probe-matrix job: scripts/test/probe-matrix.sh
(real ufw/nginx/firewalld in 4 container shapes) existed since plan
75 but was wired into NO workflow; the ubuntu-root-sbinless leg was
re-verified green locally before wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
G3: session-scoped route_rules serves url_map checks without booting a
second app (each boot re-exposes the plan-64 state-leak classes);
test_fixture_scope_guard freezes the 13 remaining direct create_app
callers as a shrinking baseline (config_checkpoint_alias migrated as
proof). G4: assert_requires_auth/assert_admin_only promoted into
tests/factories.py; test_raw_infra_authz delegates to the shared shape.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tests/factories.py owns make_user/headers_for/make_application/
make_server/make_workspace; conftest personas (scoping_rbac,
new viewer_headers/developer_headers fixtures) ride them. The
byte-identical _seed_app pair (container scale/sleep), two variants
(image_update, waf) and three server fixtures (fleet doctor/repair)
migrate as proof. Ratchet freezes the 44-file hand-minted-JWT and
65-file direct-Application() populations.
Note: test_fleet_doctor.py::test_dns_fail_when_unresolved fails
pre-existing on this machine (warn vs fail), unrelated to this change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plan 75 §G7. The suite's hand-rolled stubs were entrenching the very
duplication the plan removes: _FakeProc pasted byte-identical in four
modules, ~25 fake_run closures re-implementing argv dispatch, and zero
tests patching the wrappers — every stub targeted raw subprocess inside
one service module, so each new service test recreated the raw seam.
tests/subprocess_stub.py is that seam. Tests script argv → result; when
G1 moves services onto run_checked() the retarget happens here once
instead of in twelve files.
Two properties are deliberate:
- An unscripted command raises instead of inventing returncode=0. It is a
BaseException for the same reason SbinPathError is — services wrap
subprocess in `except Exception`, so an AssertionError would be eaten
by exactly the handlers the kit exists to test.
- Stubbed execs are still sbin-guarded. Patching subprocess removes the
§B2 runtime guard, so the kit re-applies it; popen_guard grows a public
guard_argv() as the one door for that check.
Matching normalises away a sudo prefix and argv[0] absolutisation, so one
scripted rule matches on a root box, a sudo box and a dev box alike.
Mutation-checked: removing the guard call fails the guard test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The static sbin guard reads literal argv lists — 164 of 288 raw
subprocess sites (57%). The other 43% (variables, f-strings,
concatenation) is where plan 74's outage class can still hide: a call
that works in dev (PATH has sbin) and dies under the panel unit (PATH
has none).
An autouse conftest fixture now routes every subprocess.Popen the suite
makes through GuardedPopen, which sees the actual argv at call time and
refuses a bare name that resolves under the current PATH but not under
one with the sbin dirs removed. A binary simply absent on the test
machine still raises the honest FileNotFoundError — the guard judges
PATH shape, never machine contents. The refusal is a BaseException:
this bug class ships because blanket except handlers convert exec
failures into false facts, so an AssertionError would be swallowed by
exactly the handlers it hunts.
Coverage is 'whatever the suite exercises' — honest, measurable, and
growing for free with every new test. Full suite green with the guard
live (3791 passed; 2 pre-existing load flakes pass in isolation).
20 tests across five files failed on any checkout with the serverkit-wordpress
sibling repo present, with the message "the serverkit-wordpress extension is not
installed". That message is what made this hard to see: the extension WAS
available, and the suite is explicitly built so a fresh clone passes without it.
The cause is that wordpress_bridge.ensure_loadable() gates on an ACTIVE
InstalledPlugin ROW, not on importability. That is deliberate (audit F2): a
DISABLED extension's modules stay importable so its blueprints can be
re-mounted, but its services must not be reachable. When that check landed,
wp_extension_package quietly stopped being able to deliver what its docstring
promised -- it only registers the package in sys.modules, and it took no `app`
fixture, so it could not create a row even in principle. Its sibling
wp_extension seeds one, which is exactly why those tests passed and these did
not.
So the fixture now seeds the active row too. Nothing about the extraction was
wrong; a fixture lost a capability and the error message blamed the wrong thing.
Verified: the five affected files plus test_setup_reconcile (the other consumer
of this fixture) -- 96 passed, 0 failed, where 20 failed before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `app` fixture called create_app('testing') for every one of 3173 tests,
registering 90+ blueprints each time. Measured on a fixed 199-test sample
(pytest --splits 16 --group 5 --durations=0):
setup 144.9s 89.5%
call 8.8s 5.4%
teardown 8.2s 5.0%
95% of the suite's time was preparing to test; 5% was testing. The app is now
built once per process and shared.
Same 794-test slice, before and after — identical results, 2.66x faster:
before 4 failed, 773 passed, 17 skipped 593.30s
after 4 failed, 773 passed, 17 skipped 223.11s
(the 4 are pre-existing locally: serverkit-wordpress isn't checked out here.)
Sharing one app surfaces three things a per-test app hid, all handled in the
fixture rather than patched test by test:
1. create_app() is not a pure constructor — it SEEDS the bundled flagship
extension rows as a boot side effect (test_cloudflare_extraction asserts
they are "seeded on boot"). The per-test drop_all wipes them, and a shared
app never re-boots, so every test after the first saw no flagship and the
extension routes answered 503. seed_flagship_extensions() is now re-run
after each create_all. The session fixture also boots twice on purpose:
once to create the schema, once to seed into a schema that exists.
2. Config mutation now leaks where it used to die with the app (e.g.
test_demo_deploys sets DEMO_DEPLOYS_ENABLED=False and never restores it),
so config is snapshotted and restored around every test.
3. Structure mutation cannot be undone at all — Flask has no
unregister_blueprint, and refuses setup methods once an app has served a
request. Those tests opt out via `pytestmark = pytest.mark.fresh_app` and
get a private app. Three groups needed it: 13 modules calling
register_blueprint/add_url_rule directly, 11 that install plugins (
plugin_service hot-loads their blueprints onto the live app), and
test_audit_log which adds routes with @app.route.
Tests using the `wp_extension` fixture are detected from the fixture graph
instead of being marked, since that fixture mounts blueprints on whatever app
it is handed — miss one marker and the failure lands in an unrelated test much
later in the run.
Plan 64 Phase 1. Phase 2 (savepoint rollback instead of create_all/drop_all)
is NOT included: it currently takes the slice from 4 failures to 100 failures
and 201 errors, and needs its own round.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite pointed every run at one fixed file, %TEMP%/serverkit_test.db,
deleted once at conftest import — and the delete swallowed OSError. On Windows
that is not a rare edge: a finished pytest whose app threads outlive it still
holds the file, the delete fails, the error is discarded, and the next run
quietly inherits the previous run's rows. create_all then collides with the
already-seeded settings row, and tests error at fixture setup — in a different
place every run, never in the test that caused it. Two suites running at once
share one database and truncate each other's tables mid-test, which is the same
symptom from the other direction.
That is the "flaky tests" folklore in this repo, and it is why a full-suite run
reported four errors that all passed when re-run alone.
Each process now uses its own serverkit_test_<pid>.db and sweeps stale ones at
startup, so a lingering process can no longer poison the next run and two
suites can run side by side. Session finish disposes the engine and removes the
file; if a thread still holds it, the next run's sweep gets it.
Verified by running two suites concurrently — both green, where the shared file
previously corrupted whichever one started second.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SERVERKIT_REGISTRY_URL unset now resolves to the curated raw index at
jhd3197/serverkit-extensions (the go-live step for the registry infra);
set-but-empty explicitly disables remote fetching (air-gapped panels,
and how the test suite stays offline — pinned in conftest). URL is
resolved per fetch so env changes apply without a restart. Cache ->
bundled fallback chain unchanged; the Marketplace never blanks.
3 proving tests (default, empty-disables, override); registry/plugin
selection 76 passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The previous commit added a module-level `from sqlalchemy import event` to
register SQLite speed PRAGMAs. But the test-system-utils workflow runs
tests/test_utils_system.py with only `pip install pytest` (no SQLAlchemy), and
pytest still imports conftest.py — so it broke with
ModuleNotFoundError: No module named 'sqlalchemy'.
Wrap the import + listener in try/except ImportError. Those jobs run no
DB-backed test, so skipping the PRAGMA tuning there is harmless; the full
backend job still gets it.
Verified: tests/test_utils_system.py (41 passed) and the agent e2e roundtrip
both pass; conftest now imports with just pytest + stdlib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_command_roundtrip_via_send_command drives agent_registry.send_command
from a background thread while the main thread handles /poll and /result.
Flask-SQLAlchemy serves :memory: from a single shared connection (StaticPool),
so the two threads issue interleaved transactions on one connection and the
background commit intermittently dies with ObjectDeletedError /
PendingRollbackError (it passed some CI runs, failed others).
Point tests at a temp-file SQLite instead: each thread gets its own connection
with SQLite's file locking, which is safe and deterministic. Add
synchronous=OFF / journal_mode=MEMORY PRAGMAs so the file DB stays ~as fast as
in-memory (throwaway data needs no durability). No production code changed.
Verified: the e2e roundtrip passes 22/22 consecutive runs (was flaky); full
suite green (exit 0, 232 passed / 12 skipped locally).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Allow authenticated (non-admin) users to fetch API key usage by replacing @admin_required with @auth_required in api_analytics.py and importing auth_required. Add optional user support to privileged_cmd and run_privileged in app/utils/system.py so callers can run commands as a specific user (sudo -u). Update wordpress_service to use the new user keyword when invoking privileged commands. Fix tests/conftest.py to put the backend root (parent directory) on sys.path so tests import correctly.
Introduce team invitation support and a revamped email subsystem plus admin features. Adds Invitation model and API endpoints (create, list, revoke, resend, validate) and integrates invitation validation into user registration. Replaces old email account model with richer email models (EmailDomain, EmailAccount, EmailAlias, EmailForwardingRule, DNSProviderConfig) and updates models __init__. Refactors email API to manage domains, accounts, aliases, forwarding, DNS providers, spamassassin, Roundcube/Postfix services and queue/log endpoints. Adds several backend services (invitation, DNS provider, DKIM, Dovecot, Postfix, Roundcube, SpamAssassin, permission service) and a DB migration for permissions/invitations. Enhances admin API with permission management and activity dashboard endpoints, extends audit log actions, and adds a permission_required RBAC decorator. Includes frontend additions for settings/invitations UI and miscellaneous config/service updates and tests.
Juan Denis