Files
ServerKit/docs/TEMPLATE_CATALOG_SCHEMA.md
Juan DenisandClaude Opus 4.8 f47d0e1428 feat: Phase 1 foundation — status aggregator, API scopes, onboarding, template magic vars
Four foundational enhancements toward the deploy-code developer experience:

- Container status aggregator (C7): new ContainerStatusService applies a
  deterministic priority hierarchy (degraded > restarting > running:unhealthy >
  starting > running:healthy > exited > unknown) over an app's containers,
  cached with a short TTL. Exposed at /api/v1/status and pushed over a new
  container_status Socket.IO channel; Applications page now renders an
  aggregated ContainerStatusPill with sub-status tooltip.

- API token scopes + sensitive data masking (C10): require_scope() decorator
  enforces fine-grained scopes for API-key requests (pass-through for JWT),
  a canonical SCOPES catalog at /api/v1/api-keys/scopes, and a reusable
  sensitive_data_filter (mask_sensitive) honoring secrets:read. Frontend gains
  a grouped ApiKeyScopesModal picker and per-key scope pills.

- Server onboarding state machine (C8): new onboarding lifecycle
  (pending -> validating -> installing_prerequisites -> installing_docker ->
  pairing_agent -> ready/failed) with ServerOnboardingService, a
  server_onboarding_logs table, servers.onboarding_* columns, REST endpoints,
  and an OnboardingWizard surfaced on the server detail overview.

- Declarative template catalog (C5): documented catalog schema and
  magic-variable resolution (SERVICE_PASSWORD_/USER_/FQDN_/URL_/BASE64_*)
  layered onto the existing YAML template loader without changing behavior for
  templates that don't use them; GET /api/v1/templates/catalog/schema.

Migration 029 adds the onboarding table + columns. 60 new tests pass; frontend
builds clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 18:23:47 -04:00

11 KiB

Template Catalog Schema

This document describes the declarative service-template format ServerKit uses for one-click application deployment, and the magic-variable convention that lets template authors use industry-standard placeholders without declaring an explicit variable for every generated secret.

Templates are plain YAML files. The bundled catalog lives in backend/templates/*.yaml (≈105 templates), indexed by backend/templates/index.json. A remote/community repository serves the same shape at <repo>/index.json plus <repo>/templates/<id>.yaml, fetched by the panel's repository sync.

  • Parser / validator / installer: backend/app/services/template_service.py (TemplateService)
  • HTTP surface: backend/app/api/templates.py (/api/v1/templates)
  • Live, machine-readable description of variable types + magic tokens: GET /api/v1/templates/catalog/schema

1. Template document

A template is a single YAML mapping. Fields:

Field Required Type Notes
id recommended string (slug) Lowercase a-z 0-9 -. Should match the filename stem (ghost.yaml → ghost).
name yes string Human-readable display name.
version yes string Free-form version (quote it so YAML keeps it a string, e.g. "5.75").
description yes string Short one-line summary.
icon recommended string URL or inline data:image/svg+xml;base64,... (offline-safe).
categories no list[string] Used for browse/filter (e.g. cms, database).
website no string Project homepage.
documentation no string Docs URL.
variables no list or dict See §2.
compose yes* mapping A Docker Compose document (services: required).
dockerfile yes* string Alternative to compose.
files no list Config files to materialize + bind-mount. See §4.
scripts no mapping pre_install / post_install / pre_update / post_update shell scripts.
ports no list Documentation of exposed ports (display only).
requirements no mapping e.g. memory: 512MB, storage: 2GB (display only).
auto_domain no bool When true, the installer publishes the app at <slug>.<base_domain> if a managed-sites base domain is configured. Best-effort, never forces SSL.

* A template must have either compose or dockerfile. Validation lives in TemplateService.validate_template.

Minimal example

id: my-service
name: My Service
version: "1.0"
description: Example service
icon: https://example.com/icon.png
categories:
  - utility

variables:
  - name: HTTP_PORT
    type: port
    default: "8080"
    description: Port to expose the service

compose:
  services:
    app:
      image: example/app:latest
      container_name: ${APP_NAME}
      restart: unless-stopped
      ports:
        - "${HTTP_PORT}:8080"

2. Variables

variables: may be written in either form; the installer normalizes both:

# List form (preferred for new templates)
variables:
  - name: DB_PASSWORD
    type: password
    length: 24
    description: Database password
    hidden: true

# Dict form (legacy, still supported)
variables:
  DB_PASSWORD:
    type: password
    length: 24

Variable fields

Field Notes
name Required. Referenced in the template as ${NAME}. Convention: UPPER_SNAKE_CASE.
type One of the types below. Defaults to string.
default Default value (used as the seed for port, fallback for string).
description Shown in the install UI.
required If true and not auto-generated, the user must supply it.
hidden Hide from the install form (auto-generated secrets / ports).
length For password / random.
special_chars For password: include !@#$%^&* when true.
options Suggested choices (display hint).

Variable types

Resolved by TemplateService.generate_value:

Type Auto-generated Behavior
string no Uses default; if required and unset, the user must provide it.
password yes Strong secret of length chars (default 32); adds special chars when special_chars: true.
port yes Always auto-assigned to a free host port (DB + Docker + bind-test checked). Never taken from user input. A global managed_app_base_port setting overrides the per-template seed.
uuid yes UUIDv4.
random yes Random hex token of length chars.

The API marks port, password, random, and uuid as auto_generated (and port as hidden) when describing a template to the UI.

Substitution

Anywhere in compose, files[].content, and scripts, the token ${NAME} is replaced with the resolved variable value (TemplateService.substitute_variables, pattern \$\{([A-Z_][A-Z0-9_]*)\}). ${APP_NAME} is always available (the chosen app name). All resolved variables are written to the app's .env.


3. Magic variables

Magic variables are auto-resolved placeholders that need no variables: entry. They let authors use industry-standard tokens for generated secrets, service users, and the app's public address. They are resolved at install time and merged into the install variables (so they render via the normal ${...} substitution, land in .env, and can be surfaced post-install).

Resolution is implemented in TemplateService.resolve_magic_variables / collect_magic_variables and is pure and unit-testable — no Docker, no network. The only contextual input is an optional context (app_name / fqdn / scheme).

Supported tokens

Written as ${SERVICE_<KIND>_<NAME>}, where <NAME> is an author-chosen identifier (UPPER_SNAKE_CASE) that groups related tokens — the same <NAME> resolves to the same value throughout one install.

Token Resolves to
${SERVICE_PASSWORD_<NAME>} A generated strong password (32 alphanumeric chars; compose/shell-safe).
${SERVICE_USER_<NAME>} A generated service username: svc_<name>_<rand> (lowercased, [a-z0-9_]).
${SERVICE_FQDN_<NAME>} The app's auto-assigned hostname <slug>.<base_domain> when the template sets auto_domain: true and a base domain is configured; otherwise a localhost placeholder the finalizer can fill in later.
${SERVICE_URL_<NAME>} Full URL derived from the FQDN and scheme (https when wildcard HTTPS covers the host, else http).
${SERVICE_BASE64_<NAME>} Base64 of a freshly generated secret.

Stability: within a single install, each unique token is generated once. Re-using ${SERVICE_PASSWORD_DB} in five places yields five copies of the same password. ${SERVICE_PASSWORD_DB} and ${SERVICE_PASSWORD_CACHE} get different values.

Best-effort & non-fatal: FQDN/URL resolution degrades gracefully. If site routing isn't configured (or the template doesn't opt into auto_domain), these fall back to a documented localhost-style placeholder instead of failing the install.

Example

compose:
  services:
    app:
      image: example/app:latest
      container_name: ${APP_NAME}
      environment:
        APP_URL: "${SERVICE_URL_WEB}"
        DB_USER: "${SERVICE_USER_DB}"
        DB_PASSWORD: "${SERVICE_PASSWORD_DB}"
        SESSION_SECRET: "${SERVICE_BASE64_SESSION}"
    db:
      image: postgres:16
      environment:
        POSTGRES_USER: "${SERVICE_USER_DB}"     # same value as above
        POSTGRES_PASSWORD: "${SERVICE_PASSWORD_DB}"

Equivalence with declared variables

Magic variables are sugar over the existing generated-variable machinery. The classic, fully-declared form — as used today by, for example, backend/templates/ghost.yaml:

variables:
  - name: DB_PASSWORD
    type: password
    length: 24
    hidden: true

compose:
  services:
    ghost-db:
      environment:
        MYSQL_ROOT_PASSWORD: "${DB_PASSWORD}"
        MYSQL_PASSWORD: "${DB_PASSWORD}"

…is equivalent to the magic-variable form (no variables: entry needed for the secret):

compose:
  services:
    ghost-db:
      environment:
        MYSQL_ROOT_PASSWORD: "${SERVICE_PASSWORD_DB}"
        MYSQL_PASSWORD: "${SERVICE_PASSWORD_DB}"   # same generated value

Both end up writing the generated secret to .env. The declared form lets you tune length/special_chars; the magic form is terser. (This is shown for illustration only — templates are not mass-converted; existing templates keep working unchanged.)


4. Files

files: materializes config files next to the compose project and bind-mounts them into the container. ${VAR} and ${SERVICE_*} tokens in content are substituted.

files:
  - path: /etc/app/config.yaml      # path inside the container
    mode: 0o644
    content: |
      secret: ${SERVICE_BASE64_CONFIG}
      user: ${SERVICE_USER_APP}

The installer writes the file locally (basename of path) and replaces the matching named volume / directory mount with a bind mount. Files are re-rendered on update so config survives a compose regeneration.


5. Validation

Two validators are available:

  • TemplateService.validate_template(template) — the loader's check: required fields, compose/dockerfile presence, well-formed variables. Returns {'valid': bool, 'errors': [...]}.
  • TemplateService.validate_catalog_entry(entry) — a lightweight catalog-level check that wraps the above and additionally verifies the id slug, flags unknown variable type values, and flags malformed magic tokens. Returns {'valid': bool, 'errors': [...], 'warnings': [...]}. Warnings are non-fatal so the loader stays permissive.

6. API

Method Path Purpose
GET /api/v1/templates/ List templates (local + remote).
GET /api/v1/templates/categories List categories.
GET /api/v1/templates/<id> Template detail (processed variables).
GET /api/v1/templates/catalog/schema Machine-readable description of variable types + magic tokens (this document, in JSON).
POST /api/v1/templates/<id>/install Install as a new app (admin).
POST /api/v1/templates/validate-install Pre-flight validation.
POST /api/v1/templates/sync Fetch templates from configured repos (admin).
GET/POST/DELETE /api/v1/templates/repos Manage repositories.
GET /api/v1/templates/repos/index The publishable index.json for this instance.

GET /catalog/schema response (shape)

{
  "schema_version": "1.0",
  "variable_types": [
    {"type": "string",   "auto_generated": false, "description": "..."},
    {"type": "password", "auto_generated": true,  "description": "..."},
    {"type": "port",     "auto_generated": true,  "description": "..."},
    {"type": "uuid",     "auto_generated": true,  "description": "..."},
    {"type": "random",   "auto_generated": true,  "description": "..."}
  ],
  "magic_variables": [
    {"token": "${SERVICE_PASSWORD_<NAME>}", "description": "..."},
    {"token": "${SERVICE_USER_<NAME>}",     "description": "..."},
    {"token": "${SERVICE_FQDN_<NAME>}",     "description": "..."},
    {"token": "${SERVICE_URL_<NAME>}",      "description": "..."},
    {"token": "${SERVICE_BASE64_<NAME>}",   "description": "..."}
  ],
  "notes": ["..."]
}