Files
ServerKit/backend/app/api/manifests.py
T
Juan DenisandClaude Opus 5 5f8a3ef204 feat(recycle-bin): applications are soft-deletable, and a delete stops keeping data hostage
Plan 70. Domain went through this and TEN read paths were wrong, four with
effects outside the panel. Application has 211 call sites across 68 files and
drives more -- container lifecycles, nginx, backups, registry pulls, the OS
crontab -- so it was audited in full first (docs/plans/70_...md). 180 sites are
now query_active(); the 35 left unfiltered are each commented in place.

**The delete / purge split is what makes the tombstone honest.** delete_app used
to run compose_down(volumes=True) and rmtree(). Keeping a tombstone AND doing
that would put a Restore button next to a row whose data the delete had already
destroyed -- the one promise a recycle bin cannot break. So:

  DELETE   containers down (VOLUMES KEPT), vhost removed, firewall closed,
           cron suspended, row tombstoned. All reversible.
  RESTORE  vhost rewritten, firewall reopened, cron resumed. Containers left
           STOPPED -- "put it back" must not silently start processes or open
           ports, the same reasoning that stops domain restore re-issuing certs.
  PURGE    volumes removed, uploaded source deleted, cron detached.

register() gained on_purge for that, wired into BOTH purge paths. And
purge_expired had NO scheduler -- fine when a tombstone was just a row, not fine
when it holds data volumes and a source tree, so a daily builtin job now runs
it. Without that, "kept until purge" would have meant kept forever and delete
would have quietly stopped reclaiming disk.

A side effect worth naming: the old _remove_data_flag protection for database
engines is now the DEFAULT for every app, and the flag is carried into
?purge=true so an explicit skip-the-bin delete still honours it.

**Cron was the blocker with no mechanical answer.** before_delete never fires on
a soft delete, and clear_application is LOSSY -- it nulls the association
without recording it, so nothing could put it back. suspend_for_application
instead disables only the jobs that were CURRENTLY ENABLED and stamps each
suspended_by_app; resume re-enables exactly those. That flag is the whole point:
without it a restore switches back on jobs the user deliberately turned off.
clear_application survives as purge-only, where lossiness is correct.

**The indirect reach (audit 3.1) is the class that grep cannot find.**
ContainerSleepPolicy, BackupPolicy, DeploymentJob and WafPolicy are enumerated
on a SCHEDULE and resolve application_id afterwards, so they never appear in a
search for Application.query. Left alone, a tombstoned app's sleep policy is
swept forever and wake_app restarts the containers the delete tore down. Those
loops test Application.deleted_ids().

Deliberately NOT scoped: reservation checks (port, name, directory) -- a
tombstone must keep holding those or a new app takes them and the restore is
refused; api/git.py's webhook VISIBILITY gate, which falls through to
"panel-wide visible" so hiding a tombstone would WIDEN access; and cron
attribution, which only labels a row.

Verified: 15 new tests, the migration replayed on a fresh DB (a tombstone frees
its private_slug, two live rows still collide), and a full delete -> restore
round trip. The 20 remaining full-suite failures are the known
serverkit-wordpress-not-installed class and are identical on a clean tree.

Two traps caught by testing, both recorded in the plan: a bad import made the
ENTIRE purge cleanup silently no-op while purge() still reported success
(_run_purge_hook swallows by design, so assert on the warning, not just ok), and
private_slug had to lose unique=True at the MODEL level or a models-built
database burns the slug regardless of the migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 20:26:04 -04:00

203 lines
7.5 KiB
Python

"""Declarative serverkit.yaml manifest API.
Phase 0: `scaffold` (render a v1 manifest from a live app).
Later phases add `get`, `plan` and `apply`. Admin-gated; applies are
audit-logged.
"""
from flask import Blueprint, jsonify, request, Response
from flask_jwt_extended import jwt_required, get_jwt_identity
from app import db
from app.models.application import Application
from app.models.application_manifest import ApplicationManifest, STATUS_PENDING
from app.models.project import Project
from app.models.user import User
from app.services.manifest_scaffold_service import ManifestScaffoldService
from app.services.manifest_spec_service import ManifestSpecService, ManifestError
from app.services.manifest_apply_service import ManifestApplyService
from app.services.manifest_persistence_service import ManifestPersistenceService
manifests_bp = Blueprint('manifests', __name__)
def _load_normalized(data):
"""Normalize from an inline body, or fall back to the project's stored row.
Returns (normalized, raw_text, error_response_or_None)."""
if 'content' in data:
try:
return ManifestSpecService.normalize_text(data['content']), data['content'], None
except ManifestError as exc:
return None, None, (jsonify({'error': 'Invalid manifest', 'errors': exc.errors}), 400)
if 'manifest' in data:
try:
return ManifestSpecService.normalize(data['manifest']), None, None
except ManifestError as exc:
return None, None, (jsonify({'error': 'Invalid manifest', 'errors': exc.errors}), 400)
# fall back to the stored manifest for the project
project_id = data.get('project_id')
row = ApplicationManifest.query.filter_by(project_id=project_id).first() if project_id else None
if row and row.get_normalized():
return row.get_normalized(), row.raw_text, None
return None, None, (jsonify({'error': 'Provide `content`/`manifest`, or store one first'}), 400)
def _require_admin():
user = User.query.get(get_jwt_identity())
if not user or user.role != 'admin':
return None, (jsonify({'error': 'Admin access required'}), 403)
return user, None
@manifests_bp.route('/scaffold', methods=['GET'])
@jwt_required()
def scaffold_manifest():
"""GET /api/v1/manifests/scaffold?app_id=&format=json|yaml"""
user, err = _require_admin()
if err:
return err
app_id = request.args.get('app_id', type=int)
if not app_id:
return jsonify({'error': 'app_id is required'}), 400
app = Application.query_active().filter_by(id=app_id).first()
if not app:
return jsonify({'error': 'Application not found'}), 404
fmt = (request.args.get('format') or 'json').lower()
if fmt == 'yaml':
yaml_text = ManifestScaffoldService.scaffold_yaml(app)
return Response(yaml_text, mimetype='text/yaml')
manifest = ManifestScaffoldService.scaffold_for_app(app)
return jsonify({'manifest': manifest,
'yaml': ManifestScaffoldService.scaffold_yaml(app)}), 200
@manifests_bp.route('/validate', methods=['POST'])
@jwt_required()
def validate_manifest():
"""POST /api/v1/manifests/validate { content | manifest } -> normalized summary."""
user, err = _require_admin()
if err:
return err
data = request.get_json(silent=True) or {}
try:
if 'content' in data:
normalized = ManifestSpecService.normalize_text(data['content'])
elif 'manifest' in data:
normalized = ManifestSpecService.normalize(data['manifest'])
else:
return jsonify({'error': 'Provide `content` or `manifest`'}), 400
except ManifestError as exc:
return jsonify({'valid': False, 'errors': exc.errors}), 200
return jsonify({
'valid': True,
'summary': ManifestSpecService.summarize(normalized),
'normalized': normalized,
}), 200
@manifests_bp.route('', methods=['GET'])
@jwt_required()
def get_manifest():
"""GET /api/v1/manifests?project_id= -> the stored manifest for a project."""
user, err = _require_admin()
if err:
return err
project_id = request.args.get('project_id', type=int)
if not project_id:
return jsonify({'error': 'project_id is required'}), 400
row = ApplicationManifest.query.filter_by(project_id=project_id).first()
if not row:
return jsonify({'manifest': None}), 200
return jsonify({'manifest': row.to_dict(include_raw=True)}), 200
@manifests_bp.route('/plan', methods=['POST'])
@jwt_required()
def plan_manifest():
"""POST /api/v1/manifests/plan {project_id, content|manifest?} -> dry-run plan."""
user, err = _require_admin()
if err:
return err
data = request.get_json(silent=True) or {}
project = Project.query.get(data.get('project_id')) if data.get('project_id') else None
if not project:
return jsonify({'error': 'A valid project_id is required'}), 400
normalized, _raw, err_resp = _load_normalized(data)
if err_resp:
return err_resp
plan = ManifestApplyService.plan(project, normalized)
return jsonify({'plan': plan}), 200
@manifests_bp.route('/apply', methods=['POST'])
@jwt_required()
def apply_manifest():
"""POST /api/v1/manifests/apply {project_id, content|manifest?} -> apply."""
user, err = _require_admin()
if err:
return err
data = request.get_json(silent=True) or {}
project = Project.query.get(data.get('project_id')) if data.get('project_id') else None
if not project:
return jsonify({'error': 'A valid project_id is required'}), 400
normalized, raw, err_resp = _load_normalized(data)
if err_resp:
return err_resp
# persist/refresh the manifest row (pending) before applying
row = ManifestPersistenceService.store_manifest(
project_id=project.id, normalized=normalized, raw_text=raw, status=STATUS_PENDING)
db.session.commit()
result = ManifestApplyService.apply(project, normalized, user_id=user.id,
manifest_row=row)
# Appliance-tier blockers (plan 35): apply refused, nothing executed.
if result.get('refused'):
return jsonify(result), 400
try:
from app.services.audit_service import AuditService
AuditService.log('manifest.apply', user_id=user.id, target_type='project',
target_id=project.id,
details={'success': result['success'], 'applied': result.get('applied'),
'job_id': result.get('job_id')})
except Exception:
pass
status = 200 if result['success'] else 207
return jsonify(result), status
@manifests_bp.route('/bootstrap/reset', methods=['POST'])
@jwt_required()
def reset_bootstrap():
"""POST /api/v1/manifests/bootstrap/reset {app_id, confirm}
Re-arm a one-shot appliance bootstrap so the next apply runs it again.
Destructive-ish (the bootstrap may regenerate config), so it requires
typing the app name in `confirm`.
"""
user, err = _require_admin()
if err:
return err
data = request.get_json(silent=True) or {}
app = (Application.query_active().filter_by(id=data.get('app_id')).first()
if data.get('app_id') else None)
if not app:
return jsonify({'error': 'Application not found'}), 404
if (data.get('confirm') or '') != app.name:
return jsonify({'error': f'Type the app name "{app.name}" to confirm'}), 400
app.bootstrap_done = False
db.session.commit()
return jsonify({'success': True, 'app_id': app.id, 'bootstrap_done': False}), 200