Full VM installs and Test Sandbox full mode now record a short
post-install runtime observation (60s settle + 120s sampling) with an
explicit measured/unavailable/failed/interrupted status that never
rewrites install success. Adds the standalone scripts/measure-runtime.py
sampler, the measure-installed-runtime.py harness wrapper, E2E report
rendering, results-API summaries, CI coverage and METRICS.md accounting
boundaries.
Introduces a full-stack Test Sandbox feature to run distro matrix tests in Docker from the UI: new backend API/service, sandbox_runs model + migration, run/log endpoints, and a new frontend page with mode selection, run history, live polling, cancel, and per-distro logs.
Also adds ARM64 scripts CI coverage (ubuntu-24.04-arm) and strengthens install/test automation for minimal images and overlay rebuilds: early curl/bootstrap index refresh, stricter venv validation, release-venv glibc fallback, Node reprovisioning on source fallback, rsync --delete overlay behavior, npm bootstrap in VM install, Windows tar pinning in PowerShell harness, plus updated test docs and distro coverage notes.
- Vagrantfile: don't `dnf install rsync/curl` on RHEL boxes — it upgrades
openssl-libs and breaks the box's sshd (same ABI issue as install.sh).
rsync/curl ship preinstalled on generic/rocky9; fail loudly if a future
box lacks them rather than silently corrupting the run.
- full-stack-test.ps1: drop fedora from the default distro set (Vagrant
Cloud's generic/fedora is EOL at f39; opt in via -Only fedora).
- vm-install.sh (new): run install.sh from uploaded local source, overlay
the working tree, rebuild, and probe health — so the harness tests the
exact local code (incl. uncommitted) with sshd/firewall snapshots on every
exit path for post-mortems. Recovered from a local stash.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes a cascade of PowerShell + Multipass quirks found during the first
real run:
- Rewrote orchestrator sequentially. Start-Job's separate runspace was
hiding the real errors below behind opaque "method on null" failures.
- Removed `function Mp` helper. PowerShell ships a built-in alias `mp`
for Move-ItemProperty that silently shadows same-named functions and
turned multipass calls into Move-ItemProperty calls. Inline `& $MpExe`
everywhere instead.
- Lowered ErrorActionPreference from Stop to Continue. Multipass writes
harmless "cannot set permissions" warnings to stderr when transferring
to NTFS; Stop turned those into fatal aborts.
- Stopped piping `multipass exec` stdout through Tee-Object. The pipe
could stay open indefinitely after the remote command exited, hanging
the orchestrator for 45+ min on a finished install. Now redirects to a
file on the VM and only checks the exit code.
- Tightened tar excludes. Tarball dropped from 305 MB to 20 MB.
- Added `npm ci` in vm-install.sh after the local-code overlay. The
original installer ran `npm ci` from origin/main, then the overlay
dropped in a dev package.json that needed different deps, causing a
spurious frontend build failure.
- Added -ReuseVm flag for re-running just the harness against an
already-installed VM (skips the 20-min launch cost).
First successful end-to-end run on Ubuntu 24.04: 10/10 tests passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
One-click PowerShell entry point that spins up Ubuntu/Debian VMs in
parallel, installs ServerKit from the local working tree (not origin/main),
runs a pytest API harness against the live panel, and aggregates results
into a self-contained HTML report.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Juan Denis