mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-10-29 16:17:39 +09:00
## Problem Assets-backed Builder pages can take 6–7 seconds to become usable even when their Assets resources only request metadata. On the production-derived Blog project, a single Assets query measured 8,703 ms and an immediate repeat measured 8,110 ms. Four sibling Assets resources repeated the same authorization, development-build load/parse, compilation, canonical-asset preparation, runtime metadata load, and document work. ## Architecture The loader now treats remote resources holistically while preserving provider boundaries: - The client scheduler is resource-provider neutral. It versions logical requests, frees capacity when obsolete requests are replaced, ignores stale results, and aborts physical batches only when every member is obsolete. - Ordinary Assets requests in one server resource-loader call use a deferred Assets provider and execute as one repository batch. Independent HTTP, GraphQL, and other resources start immediately instead of waiting for Assets preparation. - Arbitrary remote responses are **not** coalesced or persistently cached. Their auth headers, cookies, upstream freshness, and `Vary` semantics remain isolated. - The Assets provider authorizes and creates its API context once per batch, then loads only `props`, `dataSources`, and `resources` from the development Build. - Queries covered by the saved Build compilation plan reuse that exact plan/artifact, preserving published byte limits, truncation, and pinned-revision behavior. Uncovered unpinned queries use a deterministic literal union and safely fall back to their existing individual path if shared preparation cannot preserve behavior. - Request-scoped document loading uses npm's byte-bounded `lru-cache` so sibling queries do not reread the same referenced author/content files. - Sibling dynamic queries with the same filter, sort, output, and content shape stitch into one document-graph resolution while keeping filter values, pagination windows, results, and failures independent. Aggregate document/byte-limit failures fall back to isolated execution. - Provider-neutral batching lives in the SDK runtime, and project asset-query preview orchestration lives in the shared project package so Builder and MCP paths can reuse the same behavior. - Cancellation flows from the client batch through the SDK/server/build/query path, with checkpoints between expensive phases. - Runtime asset metadata remains conservatively artifact-wide because URL/size filters, structured references, markdown hydration, materialized rows, and saved plans have different reachability rules. To make 1,000-asset artifacts safe, PostgREST `.in(...)` lookups are deduplicated and split into 100-ID chunks with bounded concurrency. ## Implementation checklist - [x] Trace the production Blog resource path and identify repeated work - [x] Copy the production Blog project into an ignored disposable local project - [x] Record production and local baselines - [x] Add provider-neutral client scheduling, stale-result protection, and cancellation - [x] Batch ordinary Assets queries without coupling or caching HTTP/GraphQL responses - [x] Load only Build fields required to create the saved content compilation plan - [x] Preserve saved-plan, truncation, fallback, diagnostics, and revision semantics - [x] Share Assets artifacts, runtime metadata, and revision-keyed document reads per request - [x] Stitch compatible query shapes into one graph resolution with isolated fallback - [x] Extract reusable SDK batching and project preview orchestration for MCP/runtime consumers - [x] Bound large runtime asset metadata lookups for the 1,000-asset project limit - [x] Add regression coverage and demonstrate new tests failing before their fixes - [x] Run focused and broad tests, type checks, lint, format, package-boundary, and diff checks - [x] Review documentation impact; no authoritative `docs/` content changes are required - [x] Regenerate and verify the affected asset-query runtime fixture; no production data is changed - [x] Address all actionable PR review comments - [x] Rerun the final parity-preserving implementation against the copied Blog project ## Measurements Production trace before the change: - Updates Assets query: 8,703 ms - Immediate identical repeat: 8,110 ms - Query artifact: 9,344 bytes / 7 documents - Saved source database: 42,086 bytes / 28 documents Copied-project local baseline: - Builder navigation until resource loading: 5,111 ms - Exact four-resource batches: 1,448 / 797 / 1,377 / 774 / 3,018 ms During implementation, before the final saved-plan parity tightening: - Exact four-resource batch: 174 ms first run, then 51–80 ms - Actual Builder resource-loader requests: 234–246 ms Final parity-preserving copied-project rerun: - Preflight: 10 ms - Exact four-resource batch: 435 ms cold, then 119 / 97 / 90 / 79 ms warm - Every response returned HTTP 200 with the same 16,884-byte payload The final cold batch is 70% faster than the best copied-project cold baseline, and the fastest warm batch is 90% faster than the best warm baseline. The production trace remains the authoritative production baseline; these before/after comparisons use the same disposable local project and captured request payload. ## Verification Passed: - Full package suites: Builder (212 files / 2,496 tests), asset-uploader (22 / 193), content-engine (40 / 274), project (7 / 84), SDK (29 / 508), and project-build (82 / 2,070) - Final focused regressions: Builder preview/API/resource integration (3 files / 42 tests), SDK resource loader (23 tests), and document-graph resolution (6 tests) - Builder, asset-uploader, content-engine, project, SDK, and project-build type checks - Fixture regeneration is deterministic; the generated vendor exactly matches the source runtime bundle - Fixture typecheck and production build - Changed-file Oxfmt, repository Oxlint, and package-boundary checks - `git diff --check` - `git diff --name-only HEAD --exit-code` New behavioral tests were demonstrated to fail before implementation and pass afterward, including Assets batch coordination, saved-plan wiring, truncation and pinned revisions, shared document reads, cancellation checkpoints, stale/replaced client requests, and large PostgREST ID filters. Agent evaluations were not run because they require separate explicit approval. ## Compatibility and remaining work The public resource result/error shape and on-demand diagnostics contract stay compatible. Generic HTTP/GraphQL caching semantics are unchanged. Compilation already in progress is not force-cancelled because safely interrupting shared compilation requires ownership/reference tracking; cancellation prevents later phases from starting. Closes #6003
59 lines
1.6 KiB
TypeScript
59 lines
1.6 KiB
TypeScript
import { spawnSync } from "node:child_process";
|
|
import { existsSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
|
import { tmpdir } from "node:os";
|
|
import { join } from "node:path";
|
|
import { expect, test } from "vitest";
|
|
|
|
const runStepScript = join(process.cwd(), "e2e/run-step.sh");
|
|
|
|
test.runIf(process.platform !== "win32")(
|
|
"stops descendant processes when a step times out",
|
|
() => {
|
|
const temporaryDirectory = mkdtempSync(join(tmpdir(), "run-step-"));
|
|
const survivedFile = join(temporaryDirectory, "survived");
|
|
const descendantScript = join(temporaryDirectory, "descendant.sh");
|
|
|
|
try {
|
|
writeFileSync(
|
|
descendantScript,
|
|
`#!/usr/bin/env bash
|
|
trap '' TERM
|
|
sleep 2
|
|
touch "$SURVIVED_FILE"
|
|
`
|
|
);
|
|
|
|
const result = spawnSync(
|
|
"bash",
|
|
[
|
|
"-c",
|
|
`
|
|
source "$RUN_STEP_SCRIPT"
|
|
spawn_descendant() {
|
|
bash "$DESCENDANT_SCRIPT" &
|
|
wait
|
|
}
|
|
run_step "test descendant cleanup" 1 spawn_descendant
|
|
`,
|
|
],
|
|
{
|
|
encoding: "utf8",
|
|
env: {
|
|
...process.env,
|
|
DESCENDANT_SCRIPT: descendantScript,
|
|
RUN_STEP_SCRIPT: runStepScript,
|
|
RUN_STEP_TERMINATION_GRACE_SECONDS: "0.1",
|
|
SURVIVED_FILE: survivedFile,
|
|
},
|
|
timeout: 15_000,
|
|
}
|
|
);
|
|
|
|
expect(result.status).toBe(124);
|
|
expect(existsSync(survivedFile)).toBe(false);
|
|
} finally {
|
|
rmSync(temporaryDirectory, { recursive: true, force: true });
|
|
}
|
|
}
|
|
);
|