ServerKit

2,886 Commits 20 Branches 91 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
Juan DenisandClaude Opus 5	5f8a3ef204	feat(recycle-bin): applications are soft-deletable, and a delete stops keeping data hostage Plan 70. Domain went through this and TEN read paths were wrong, four with effects outside the panel. Application has 211 call sites across 68 files and drives more -- container lifecycles, nginx, backups, registry pulls, the OS crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are now query_active(); the 35 left unfiltered are each commented in place. The delete / purge split is what makes the tombstone honest. delete_app used to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing that would put a Restore button next to a row whose data the delete had already destroyed -- the one promise a recycle bin cannot break. So: DELETE containers down (VOLUMES KEPT), vhost removed, firewall closed, cron suspended, row tombstoned. All reversible. RESTORE vhost rewritten, firewall reopened, cron resumed. Containers left STOPPED -- "put it back" must not silently start processes or open ports, the same reasoning that stops domain restore re-issuing certs. PURGE volumes removed, uploaded source deleted, cron detached. register() gained on_purge for that, wired into BOTH purge paths. And purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine when it holds data volumes and a source tree, so a daily builtin job now runs it. Without that, "kept until purge" would have meant kept forever and delete would have quietly stopped reclaiming disk. A side effect worth naming: the old _remove_data_flag protection for database engines is now the DEFAULT for every app, and the flag is carried into ?purge=true so an explicit skip-the-bin delete still honours it. Cron was the blocker with no mechanical answer. before_delete never fires on a soft delete, and clear_application is LOSSY -- it nulls the association without recording it, so nothing could put it back. suspend_for_application instead disables only the jobs that were CURRENTLY ENABLED and stamps each suspended_by_app; resume re-enables exactly those. That flag is the whole point: without it a restore switches back on jobs the user deliberately turned off. clear_application survives as purge-only, where lossiness is correct. The indirect reach (audit 3.1) is the class that grep cannot find. ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated on a SCHEDULE and resolve application_id afterwards, so they never appear in a search for Application.query. Left alone, a tombstoned app's sleep policy is swept forever and wake_app restarts the containers the delete tore down. Those loops test Application.deleted_ids(). Deliberately NOT scoped: reservation checks (port, name, directory) -- a tombstone must keep holding those or a new app takes them and the restore is refused; api/git.py's webhook VISIBILITY gate, which falls through to "panel-wide visible" so hiding a tombstone would WIDEN access; and cron attribution, which only labels a row. Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees its private_slug, two live rows still collide), and a full delete -> restore round trip. The 20 remaining full-suite failures are the known serverkit-wordpress-not-installed class and are identical on a clean tree. Two traps caught by testing, both recorded in the plan: a bad import made the ENTIRE purge cleanup silently no-op while purge() still reported success (_run_purge_hook swallows by design, so assert on the warning, not just ok), and private_slug had to lose unique=True at the MODEL level or a models-built database burns the slug regardless of the migration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-12 20:26:04 -04:00
Juan DenisandClaude Opus 5	5927e44487	fix(databases): run all twelve engines before claiming they work The engine templates were written from upstream docs and reviewed carefully, but only ever read. Every one has now been brought up for real, authenticated against, and torn down. CouchDB was broken. A fresh node never created its system databases: `/_all_dbs` answered `[]`, `_users` did not exist so no non-admin account could ever be created, and it error-looped on "Missing system database _users". The image reads only COUCHDB_USER/PASSWORD/SECRET/ERLANG_COOKIE — both COUCHDB_COUCHDB__SINGLE_NODE and ERL_FLAGS are rejected as unknown_config_value — so `single_node = true` ships as a local.d config file instead. Meilisearch and Valkey advertised versions they do not install (1.0 vs the 1.12 that boots; 8.0 vs 8.1.9), caught by comparing the catalog header against what actually reported for itself. The method mattered more than the count. Probing auth from INSIDE the container is not a test and would have passed two engines that were never checked: postgres initdb writes `host all all 127.0.0.1/32 trust`, so a deliberately wrong password succeeds on loopback, and the ClickHouse client reads CLICKHOUSE_PASSWORD out of the container's own environment. Re-run from a remote source address, both correctly refuse. Every auth assertion here was made from off-host. Nine tests cover what can be checked without Docker, led by the one that matters most: the generated admin password must actually REACH the image. Mongo, ClickHouse and CouchDB all start happily with authentication disabled when the password is generated but never passed — silent, and indistinguishable from working. Also: every ${VAR} resolves, no port binds all interfaces, the catalog port matches the published one, the declared version is the tag installed, and a config file can never be rendered over the data volume's mount point (which would delete persistence without a word). Verified by reverting each fix and confirming the tests fail with the right message, then restoring. 67 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 22:51:04 -04:00
Juan DenisandClaude Opus 5	3070353631	feat(databases): install any engine, because an engine is just a template You could not install a database from ServerKit. The tree's four roots were literals, only MySQL and PostgreSQL reported real state, and no install path existed anywhere in the backend. Rather than add one, this makes a database engine an ordinary app template that declares a top-level `engine:` block — family, wire protocol, admin user, which variable holds the password, what it calls a "table". Presence of that block is the only thing that lists a template in the Databases catalog, and installing it is the template pipeline that already existed: same deployment job, same streaming console. The consequence is the point. Adding CockroachDB is adding a YAML file. So is adding something we have never heard of — author it locally or sync a template repo — and the install form still works, because it is generated from the template's own `variables[]` rather than hand-written per engine. There is no engine model, no engine table, no engine install mechanism to keep in step. `test_dropping_a_new_engine_yaml_lists_it` is the design; if it ever needs code to pass, this went wrong. Twelve engines ship: postgresql, mysql, mariadb, mongodb, redis, valkey, cockroachdb, clickhouse, influxdb, neo4j, meilisearch, couchdb. Each pins its image, binds to loopback, names a data volume, and was checked against the upstream image's actual environment contract — a wrong variable name is a container that boots with authentication silently disabled. Two safety notes worth reading. Engines bind 127.0.0.1 unless explicitly exposed, and the flag is applied after caller variables so a handcrafted payload cannot quietly publish one. And CockroachDB single-node runs `--insecure` because password auth needs a certificate-backed cluster: for engines with no credentials at all the loopback bind is the only control, so public exposure is refused outright rather than warned about. A test pins the passwordless list to exactly cockroachdb, so a future template that drops its password variable fails loudly instead of opening a hole. Also surfaced at last: `ManagedDatabase.owner_application_id` has been accepted by the create endpoints for ages with no UI to set it. Creating a database now offers "Attach to application". Deferred deliberately: new-table builder, SQL dump import, engine extensions (pgvector, TimescaleDB) — each a data-mutation surface deserving its own round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 20:09:05 -04:00

Author

SHA1

Message

Date

Juan DenisandClaude Opus 5

5f8a3ef204

feat(recycle-bin): applications are soft-deletable, and a delete stops keeping data hostage

Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.

**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:

  DELETE   containers down (VOLUMES KEPT), vhost removed, firewall closed,
           cron suspended, row tombstoned. All reversible.
  RESTORE  vhost rewritten, firewall reopened, cron resumed. Containers left
           STOPPED -- "put it back" must not silently start processes or open
           ports, the same reasoning that stops domain restore re-issuing certs.
  PURGE    volumes removed, uploaded source deleted, cron detached.

register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.

A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.

**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.

**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().

Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.

Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.

Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-12 20:26:04 -04:00

Juan DenisandClaude Opus 5

5927e44487

fix(databases): run all twelve engines before claiming they work

The engine templates were written from upstream docs and reviewed carefully, but
only ever read. Every one has now been brought up for real, authenticated
against, and torn down.

CouchDB was broken. A fresh node never created its system databases: `/_all_dbs`
answered `[]`, `_users` did not exist so no non-admin account could ever be
created, and it error-looped on "Missing system database _users". The image
reads only COUCHDB_USER/PASSWORD/SECRET/ERLANG_COOKIE — both
COUCHDB_COUCHDB__SINGLE_NODE and ERL_FLAGS are rejected as unknown_config_value
— so `single_node = true` ships as a local.d config file instead.

Meilisearch and Valkey advertised versions they do not install (1.0 vs the 1.12
that boots; 8.0 vs 8.1.9), caught by comparing the catalog header against what
actually reported for itself.

The method mattered more than the count. Probing auth from INSIDE the container
is not a test and would have passed two engines that were never checked:
postgres initdb writes `host all all 127.0.0.1/32 trust`, so a deliberately
wrong password succeeds on loopback, and the ClickHouse client reads
CLICKHOUSE_PASSWORD out of the container's own environment. Re-run from a remote
source address, both correctly refuse. Every auth assertion here was made from
off-host.

Nine tests cover what can be checked without Docker, led by the one that matters
most: the generated admin password must actually REACH the image. Mongo,
ClickHouse and CouchDB all start happily with authentication disabled when the
password is generated but never passed — silent, and indistinguishable from
working. Also: every ${VAR} resolves, no port binds all interfaces, the catalog
port matches the published one, the declared version is the tag installed, and a
config file can never be rendered over the data volume's mount point (which
would delete persistence without a word).

Verified by reverting each fix and confirming the tests fail with the right
message, then restoring. 67 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-25 22:51:04 -04:00

Juan DenisandClaude Opus 5

3070353631

feat(databases): install any engine, because an engine is just a template

You could not install a database from ServerKit. The tree's four roots were
literals, only MySQL and PostgreSQL reported real state, and no install path
existed anywhere in the backend.

Rather than add one, this makes a database engine an ordinary app template that
declares a top-level `engine:` block — family, wire protocol, admin user, which
variable holds the password, what it calls a "table". Presence of that block is
the only thing that lists a template in the Databases catalog, and installing it
is the template pipeline that already existed: same deployment job, same
streaming console.

The consequence is the point. Adding CockroachDB is adding a YAML file. So is
adding something we have never heard of — author it locally or sync a template
repo — and the install form still works, because it is generated from the
template's own `variables[]` rather than hand-written per engine. There is no
engine model, no engine table, no engine install mechanism to keep in step.
`test_dropping_a_new_engine_yaml_lists_it` is the design; if it ever needs code
to pass, this went wrong.

Twelve engines ship: postgresql, mysql, mariadb, mongodb, redis, valkey,
cockroachdb, clickhouse, influxdb, neo4j, meilisearch, couchdb. Each pins its
image, binds to loopback, names a data volume, and was checked against the
upstream image's actual environment contract — a wrong variable name is a
container that boots with authentication silently disabled.

Two safety notes worth reading. Engines bind 127.0.0.1 unless explicitly
exposed, and the flag is applied after caller variables so a handcrafted payload
cannot quietly publish one. And CockroachDB single-node runs `--insecure`
because password auth needs a certificate-backed cluster: for engines with no
credentials at all the loopback bind is the only control, so public exposure is
refused outright rather than warned about. A test pins the passwordless list to
exactly cockroachdb, so a future template that drops its password variable
fails loudly instead of opening a hole.

Also surfaced at last: `ManagedDatabase.owner_application_id` has been accepted
by the create endpoints for ages with no UI to set it. Creating a database now
offers "Attach to application".

Deferred deliberately: new-table builder, SQL dump import, engine extensions
(pgvector, TimescaleDB) — each a data-mutation surface deserving its own round.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-25 20:09:05 -04:00