Four foundational enhancements toward the deploy-code developer experience: - Container status aggregator (C7): new ContainerStatusService applies a deterministic priority hierarchy (degraded > restarting > running:unhealthy > starting > running:healthy > exited > unknown) over an app's containers, cached with a short TTL. Exposed at /api/v1/status and pushed over a new container_status Socket.IO channel; Applications page now renders an aggregated ContainerStatusPill with sub-status tooltip. - API token scopes + sensitive data masking (C10): require_scope() decorator enforces fine-grained scopes for API-key requests (pass-through for JWT), a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains a grouped ApiKeyScopesModal picker and per-key scope pills. - Server onboarding state machine (C8): new onboarding lifecycle (pending -> validating -> installing_prerequisites -> installing_docker -> pairing_agent -> ready/failed) with ServerOnboardingService, a server_onboarding_logs table, servers.onboarding_* columns, REST endpoints, and an OnboardingWizard surfaced on the server detail overview. - Declarative template catalog (C5): documented catalog schema and magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*) layered onto the existing YAML template loader without changing behavior for templates that don't use them; GET /api/v1/templates/catalog/schema. Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend builds clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
11 KiB
Template Catalog Schema
This document describes the declarative service-template format ServerKit uses for one-click application deployment, and the magic-variable convention that lets template authors use industry-standard placeholders without declaring an explicit variable for every generated secret.
Templates are plain YAML files. The bundled catalog lives in
backend/templates/*.yaml (≈105 templates), indexed by backend/templates/index.json.
A remote/community repository serves the same shape at <repo>/index.json plus
<repo>/templates/<id>.yaml, fetched by the panel's repository sync.
- Parser / validator / installer:
backend/app/services/template_service.py(TemplateService) - HTTP surface:
backend/app/api/templates.py(/api/v1/templates) - Live, machine-readable description of variable types + magic tokens:
GET /api/v1/templates/catalog/schema
1. Template document
A template is a single YAML mapping. Fields:
| Field | Required | Type | Notes |
|---|---|---|---|
id |
recommended | string (slug) | Lowercase a-z 0-9 -. Should match the filename stem (ghost.yaml → ghost). |
name |
yes | string | Human-readable display name. |
version |
yes | string | Free-form version (quote it so YAML keeps it a string, e.g. "5.75"). |
description |
yes | string | Short one-line summary. |
icon |
recommended | string | URL or inline data:image/svg+xml;base64,... (offline-safe). |
categories |
no | list[string] | Used for browse/filter (e.g. cms, database). |
website |
no | string | Project homepage. |
documentation |
no | string | Docs URL. |
variables |
no | list or dict | See §2. |
compose |
yes* | mapping | A Docker Compose document (services: required). |
dockerfile |
yes* | string | Alternative to compose. |
files |
no | list | Config files to materialize + bind-mount. See §4. |
scripts |
no | mapping | pre_install / post_install / pre_update / post_update shell scripts. |
ports |
no | list | Documentation of exposed ports (display only). |
requirements |
no | mapping | e.g. memory: 512MB, storage: 2GB (display only). |
auto_domain |
no | bool | When true, the installer publishes the app at <slug>.<base_domain> if a managed-sites base domain is configured. Best-effort, never forces SSL. |
* A template must have either compose or dockerfile. Validation
lives in TemplateService.validate_template.
Minimal example
id: my-service
name: My Service
version: "1.0"
description: Example service
icon: https://example.com/icon.png
categories:
- utility
variables:
- name: HTTP_PORT
type: port
default: "8080"
description: Port to expose the service
compose:
services:
app:
image: example/app:latest
container_name: ${APP_NAME}
restart: unless-stopped
ports:
- "${HTTP_PORT}:8080"
2. Variables
variables: may be written in either form; the installer normalizes both:
# List form (preferred for new templates)
variables:
- name: DB_PASSWORD
type: password
length: 24
description: Database password
hidden: true
# Dict form (legacy, still supported)
variables:
DB_PASSWORD:
type: password
length: 24
Variable fields
| Field | Notes |
|---|---|
name |
Required. Referenced in the template as ${NAME}. Convention: UPPER_SNAKE_CASE. |
type |
One of the types below. Defaults to string. |
default |
Default value (used as the seed for port, fallback for string). |
description |
Shown in the install UI. |
required |
If true and not auto-generated, the user must supply it. |
hidden |
Hide from the install form (auto-generated secrets / ports). |
length |
For password / random. |
special_chars |
For password: include !@#$%^&* when true. |
options |
Suggested choices (display hint). |
Variable types
Resolved by TemplateService.generate_value:
| Type | Auto-generated | Behavior |
|---|---|---|
string |
no | Uses default; if required and unset, the user must provide it. |
password |
yes | Strong secret of length chars (default 32); adds special chars when special_chars: true. |
port |
yes | Always auto-assigned to a free host port (DB + Docker + bind-test checked). Never taken from user input. A global managed_app_base_port setting overrides the per-template seed. |
uuid |
yes | UUIDv4. |
random |
yes | Random hex token of length chars. |
The API marks
port,password,random, anduuidasauto_generated(andportashidden) when describing a template to the UI.
Substitution
Anywhere in compose, files[].content, and scripts, the token ${NAME}
is replaced with the resolved variable value (TemplateService.substitute_variables,
pattern \$\{([A-Z_][A-Z0-9_]*)\}). ${APP_NAME} is always available (the
chosen app name). All resolved variables are written to the app's .env.
3. Magic variables
Magic variables are auto-resolved placeholders that need no variables:
entry. They let authors use industry-standard tokens for generated secrets,
service users, and the app's public address. They are resolved at install time
and merged into the install variables (so they render via the normal ${...}
substitution, land in .env, and can be surfaced post-install).
Resolution is implemented in TemplateService.resolve_magic_variables /
collect_magic_variables and is pure and unit-testable — no Docker, no
network. The only contextual input is an optional context
(app_name / fqdn / scheme).
Supported tokens
Written as ${SERVICE_<KIND>_<NAME>}, where <NAME> is an author-chosen
identifier (UPPER_SNAKE_CASE) that groups related tokens — the same
<NAME> resolves to the same value throughout one install.
| Token | Resolves to |
|---|---|
${SERVICE_PASSWORD_<NAME>} |
A generated strong password (32 alphanumeric chars; compose/shell-safe). |
${SERVICE_USER_<NAME>} |
A generated service username: svc_<name>_<rand> (lowercased, [a-z0-9_]). |
${SERVICE_FQDN_<NAME>} |
The app's auto-assigned hostname <slug>.<base_domain> when the template sets auto_domain: true and a base domain is configured; otherwise a localhost placeholder the finalizer can fill in later. |
${SERVICE_URL_<NAME>} |
Full URL derived from the FQDN and scheme (https when wildcard HTTPS covers the host, else http). |
${SERVICE_BASE64_<NAME>} |
Base64 of a freshly generated secret. |
Stability: within a single install, each unique token is generated once. Re-using
${SERVICE_PASSWORD_DB}in five places yields five copies of the same password.${SERVICE_PASSWORD_DB}and${SERVICE_PASSWORD_CACHE}get different values.
Best-effort & non-fatal: FQDN/URL resolution degrades gracefully. If site routing isn't configured (or the template doesn't opt into
auto_domain), these fall back to a documentedlocalhost-style placeholder instead of failing the install.
Example
compose:
services:
app:
image: example/app:latest
container_name: ${APP_NAME}
environment:
APP_URL: "${SERVICE_URL_WEB}"
DB_USER: "${SERVICE_USER_DB}"
DB_PASSWORD: "${SERVICE_PASSWORD_DB}"
SESSION_SECRET: "${SERVICE_BASE64_SESSION}"
db:
image: postgres:16
environment:
POSTGRES_USER: "${SERVICE_USER_DB}" # same value as above
POSTGRES_PASSWORD: "${SERVICE_PASSWORD_DB}"
Equivalence with declared variables
Magic variables are sugar over the existing generated-variable machinery. The
classic, fully-declared form — as used today by, for example,
backend/templates/ghost.yaml:
variables:
- name: DB_PASSWORD
type: password
length: 24
hidden: true
compose:
services:
ghost-db:
environment:
MYSQL_ROOT_PASSWORD: "${DB_PASSWORD}"
MYSQL_PASSWORD: "${DB_PASSWORD}"
…is equivalent to the magic-variable form (no variables: entry needed for the
secret):
compose:
services:
ghost-db:
environment:
MYSQL_ROOT_PASSWORD: "${SERVICE_PASSWORD_DB}"
MYSQL_PASSWORD: "${SERVICE_PASSWORD_DB}" # same generated value
Both end up writing the generated secret to .env. The declared form lets you
tune length/special_chars; the magic form is terser. (This is shown for
illustration only — templates are not mass-converted; existing templates
keep working unchanged.)
4. Files
files: materializes config files next to the compose project and bind-mounts
them into the container. ${VAR} and ${SERVICE_*} tokens in content are
substituted.
files:
- path: /etc/app/config.yaml # path inside the container
mode: 0o644
content: |
secret: ${SERVICE_BASE64_CONFIG}
user: ${SERVICE_USER_APP}
The installer writes the file locally (basename of path) and replaces the
matching named volume / directory mount with a bind mount. Files are re-rendered
on update so config survives a compose regeneration.
5. Validation
Two validators are available:
TemplateService.validate_template(template)— the loader's check: required fields,compose/dockerfilepresence, well-formedvariables. Returns{'valid': bool, 'errors': [...]}.TemplateService.validate_catalog_entry(entry)— a lightweight catalog-level check that wraps the above and additionally verifies theidslug, flags unknown variabletypevalues, and flags malformed magic tokens. Returns{'valid': bool, 'errors': [...], 'warnings': [...]}. Warnings are non-fatal so the loader stays permissive.
6. API
| Method | Path | Purpose |
|---|---|---|
GET |
/api/v1/templates/ |
List templates (local + remote). |
GET |
/api/v1/templates/categories |
List categories. |
GET |
/api/v1/templates/<id> |
Template detail (processed variables). |
GET |
/api/v1/templates/catalog/schema |
Machine-readable description of variable types + magic tokens (this document, in JSON). |
POST |
/api/v1/templates/<id>/install |
Install as a new app (admin). |
POST |
/api/v1/templates/validate-install |
Pre-flight validation. |
POST |
/api/v1/templates/sync |
Fetch templates from configured repos (admin). |
GET/POST/DELETE |
/api/v1/templates/repos |
Manage repositories. |
GET |
/api/v1/templates/repos/index |
The publishable index.json for this instance. |
GET /catalog/schema response (shape)
{
"schema_version": "1.0",
"variable_types": [
{"type": "string", "auto_generated": false, "description": "..."},
{"type": "password", "auto_generated": true, "description": "..."},
{"type": "port", "auto_generated": true, "description": "..."},
{"type": "uuid", "auto_generated": true, "description": "..."},
{"type": "random", "auto_generated": true, "description": "..."}
],
"magic_variables": [
{"token": "${SERVICE_PASSWORD_<NAME>}", "description": "..."},
{"token": "${SERVICE_USER_<NAME>}", "description": "..."},
{"token": "${SERVICE_FQDN_<NAME>}", "description": "..."},
{"token": "${SERVICE_URL_<NAME>}", "description": "..."},
{"token": "${SERVICE_BASE64_<NAME>}", "description": "..."}
],
"notes": ["..."]
}