mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-11-03 18:48:06 +09:00
Oleg Isonen
56a22fde8a
## Summary Introduce a document graph architecture for Assets resources so structured documents can reference one another while keeping full document payloads on CDN-backed storage. The content database will retain the searchable index and dependency graph. At runtime, the server will select a root document, determine its complete dependency set before performing I/O, fetch each unique document concurrently, and assemble the final result server-side. ## Implementation todos ### Document graph foundation - [x] Add normalized internal reference contracts for whole documents, JSON paths, Markdown body, and Markdown frontmatter. - [x] Add an immutable graph abstraction with deterministic ordering, dependency traversal, revision validation, and cycle detection. - [x] Add a storage-independent resolver with dependency closure, bounded parallel loading, shared-content deduplication, representation selection, assembly hooks, cancellation, and structured errors. - [x] Add focused unit coverage for the isolated document-graph abstraction. - [x] Define and validate the external remote-document reference syntax. - [x] Add a pure graph compiler that resolves canonical document URLs into IDs and revisions and validates missing, duplicate, and cyclic dependencies. ### Build-time graph - [x] Add a storage-independent source-catalog compiler for bounded parallel reference discovery and deterministic graph compilation. - [x] Add a versioned deterministic document graph artifact with canonical serialization and integrity verification. - [x] Discover and normalize statically declared references while Assets resources are processed. - [x] Resolve document identities and revisions and report missing, stale, or cyclic dependencies with document context. - [x] Persist document nodes and dependency edges alongside searchable resource metadata. - [x] Include persisted graph data in artifact integrity and size validation. ### Document representations - [x] Add a standalone JSON document adapter for exact reference markers, compiler occurrences, JSON path selection, and immutable assembly. - [x] Add a standalone Markdown document adapter for bounded decoding, frontmatter references, immutable assembly, and source, body, and frontmatter selection. - [x] Parse fetched JSON and Markdown documents through format-specific adapters. - [x] Select complete documents, JSON fragments, Markdown body, and Markdown frontmatter through the shared reference model. - [x] Assemble nested and shared references into the final server-side value. ### Runtime resolution - [x] Enforce reachable-document, per-document byte, and aggregate-byte limits before parsing and assembly. - [x] Add a revision-keyed bounded source-cache wrapper with replayable bytes, stale-entry fallback, and in-flight deduplication. - [x] Add a storage-neutral loaded-source contract that rejects revision mismatches before parsing. - [x] Resolve graph-backed property paths before filtering, sorting, pagination, and projection; skip graph loading for unrelated queries. - [x] Add an injected HTTP/CDN document loader with revision metadata validation. - [x] Apply runtime payload limits, bounded concurrency, caching, and structured failure mapping. - [x] Ensure shared documents are fetched and parsed once per resolution. ### Product integration - [x] Integrate equivalent behavior in local preview, SSR, SSG, and published runtime paths. - [x] Preserve existing embedded Assets resource behavior without requiring migration. - [x] Add observability for graph loading, document fetching, caching, and resolution failures. - [x] Document the public reference and resolution behavior. ### Review hardening - [x] Enforce graph, artifact, cache, and published-storage identity consistency. - [x] Compile only graph sources reachable from selected build entries. - [x] Resolve graph-backed fields before query evaluation and avoid unnecessary graph loading. - [x] Consolidate duplicated graph schemas, utilities, and document content-type detection. - [x] Correct runtime documentation and use structural evaluation validation. ### Verification - [x] Keep a deterministic high-impact agent evaluation for Markdown-to-JSON references across the dynamic blog overview and article routes. - [x] Add a separate discoverability evaluation that must learn the reference workflow from MCP guidance without prompt-supplied syntax, queries, bindings, or call order. - [ ] Re-run the complete local agent evaluation suite after the discoverability fixture change and compare every fixture with its accepted baseline. - [x] Add integration coverage for CDN-backed hydration, parallel fetching, caching, and shared dependencies. - [x] Add integration coverage for missing references, cycles, revision changes, limits, and cancellation. - [x] Verify equivalent behavior across preview, SSR, SSG, and published deployments. - [x] Verify that existing embedded Assets resources retain their current behavior. ## Intended outcome - Storage-backed documents do not require their complete payloads to be embedded in the content database. - The server knows the complete statically discoverable dependency set before fetching document content. - Shared dependencies are fetched once per resolution and independent resources can be fetched in parallel. - JSON fragments and Markdown body/frontmatter representations use the same document-resolution system. - Reference and payload revisions cannot be mixed silently. - Existing Assets resources continue to work without migration. ## Verification results - `@webstudio-is/content-engine`: 37 files / 244 tests passed; typecheck passed. - `@webstudio-is/asset-uploader`: 22 files / 182 tests passed; typecheck passed. - CLI: 57 files / 925 tests passed, including generated SSR and SSG build/prerender coverage; typecheck passed. - Repository-wide `pnpm oxfmt --check` passed across 2,757 files. - Package-boundary validation passed. - Previous `pnpm evaluations` run: all 4 original high-impact fixtures passed. The complete 5-fixture rerun is pending explicit approval after adding the discoverability fixture.
119 lines
4.3 KiB
JSON
119 lines
4.3 KiB
JSON
{
|
|
"name": "webstudio",
|
|
"version": "0.0.0-webstudio-version",
|
|
"description": "Webstudio CLI",
|
|
"author": "Webstudio <github@webstudio.is>",
|
|
"homepage": "https://webstudio.is",
|
|
"type": "module",
|
|
"bin": {
|
|
"webstudio-cli": "./bin.js",
|
|
"webstudio": "./bin.js"
|
|
},
|
|
"files": [
|
|
"lib/*",
|
|
"templates/*",
|
|
"bin.js",
|
|
"!*.{test,stories}.*"
|
|
],
|
|
"scripts": {
|
|
"typecheck": "tsc --noEmit",
|
|
"generate-docs": "tsx scripts/generate-docs.ts && oxfmt src/docs.generated.ts",
|
|
"build:content-runtime": "esbuild @webstudio-is/content-engine/runtime --bundle --format=esm --platform=browser --target=es2022 --conditions=webstudio --minify --outfile=lib/content-runtime.js",
|
|
"build": "pnpm generate-docs && rm -rf lib && vite build && pnpm build:content-runtime",
|
|
"pretest": "mkdir -p lib && pnpm build:content-runtime",
|
|
"evaluations": "tsx --import ../../scripts/register-react-global.ts --conditions=webstudio evaluations/high-impact/run-local-agent.ts",
|
|
"test": "vitest run",
|
|
"test:release-smoke": "pnpm build && tsx --conditions=webstudio scripts/release-smoke.ts",
|
|
"test:release-smoke:model": "WEBSTUDIO_RELEASE_SMOKE_MODEL_EVAL_REQUIRED=1 pnpm test:release-smoke",
|
|
"test:release-smoke:registry": "tsx --conditions=webstudio scripts/release-smoke.ts"
|
|
},
|
|
"license": "AGPL-3.0-or-later",
|
|
"engines": {
|
|
"node": ">=22"
|
|
},
|
|
"dependencies": {
|
|
"@clack/prompts": "^0.10.0",
|
|
"@emotion/hash": "^0.9.2",
|
|
"@trpc/client": "^10.45.2",
|
|
"@webstudio-is/http-client": "workspace:*",
|
|
"@webstudio-is/project-migrations": "workspace:*",
|
|
"@webstudio-is/protocol": "workspace:*",
|
|
"@webstudio-is/sync-client": "workspace:*",
|
|
"acorn": "^8.14.1",
|
|
"acorn-jsx": "^5.3.2",
|
|
"acorn-walk": "^8.3.4",
|
|
"change-case": "^5.4.4",
|
|
"chrome-launcher": "^1.2.1",
|
|
"deepmerge": "^4.3.1",
|
|
"decode-named-character-reference": "^1.0.2",
|
|
"env-paths": "^3.0.0",
|
|
"esbuild": "^0.25.3",
|
|
"fast-deep-equal": "^3.1.3",
|
|
"immer": "^10.1.1",
|
|
"nanoid": "^5.1.5",
|
|
"p-limit": "^6.2.0",
|
|
"parse5": "7.3.0",
|
|
"picocolors": "^1.1.1",
|
|
"reserved-identifiers": "^1.0.0",
|
|
"tinyexec": "^0.3.2",
|
|
"tus-js-client": "^4.3.1",
|
|
"warn-once": "^0.1.1",
|
|
"yargs": "^17.7.2",
|
|
"zod": "^4.4.3"
|
|
},
|
|
"devDependencies": {
|
|
"@cloudflare/vite-plugin": "^1.1.0",
|
|
"@netlify/vite-plugin-react-router": "^1.0.1",
|
|
"@react-router/dev": "^7.5.3",
|
|
"@react-router/fs-routes": "^7.5.3",
|
|
"@react-router/node": "^7.5.3",
|
|
"@react-router/serve": "^7.5.3",
|
|
"@remix-run/cloudflare": "^2.16.5",
|
|
"@remix-run/cloudflare-pages": "^2.16.5",
|
|
"@remix-run/dev": "^2.16.5",
|
|
"@remix-run/node": "^2.16.5",
|
|
"@remix-run/react": "^2.16.5",
|
|
"@remix-run/server-runtime": "^2.16.5",
|
|
"@types/mdast": "^4.0.4",
|
|
"@types/react": "^18.2.70",
|
|
"@types/react-dom": "^18.2.25",
|
|
"@types/yargs": "^17.0.33",
|
|
"@vercel/react-router": "^1.1.0",
|
|
"@vitejs/plugin-react": "^4.4.1",
|
|
"@webstudio-is/css-engine": "workspace:*",
|
|
"@webstudio-is/content-engine": "workspace:*",
|
|
"@webstudio-is/expression": "workspace:*",
|
|
"@webstudio-is/fonts": "workspace:*",
|
|
"@webstudio-is/image": "workspace:*",
|
|
"@webstudio-is/project-build": "workspace:*",
|
|
"@webstudio-is/query-builder": "workspace:*",
|
|
"@webstudio-is/react-sdk": "workspace:*",
|
|
"@webstudio-is/sdk": "workspace:*",
|
|
"@webstudio-is/sdk-components-animation": "workspace:*",
|
|
"@webstudio-is/sdk-components-react": "workspace:*",
|
|
"@webstudio-is/sdk-components-react-radix": "workspace:*",
|
|
"@webstudio-is/sdk-components-react-remix": "workspace:*",
|
|
"@webstudio-is/sdk-components-react-router": "workspace:*",
|
|
"@webstudio-is/sdk-components-registry": "workspace:*",
|
|
"@webstudio-is/tsconfig": "workspace:*",
|
|
"@webstudio-is/wsauth": "workspace:*",
|
|
"h3": "^1.15.1",
|
|
"ipx": "^3.0.3",
|
|
"isbot": "^5.1.25",
|
|
"mdast-util-directive": "^3.1.0",
|
|
"mdast-util-from-markdown": "^2.0.3",
|
|
"mdast-util-to-string": "^4.0.0",
|
|
"micromark-extension-directive": "^4.0.0",
|
|
"oxfmt": "0.58.0",
|
|
"react": "18.3.0-canary-14898b6a9-20240318",
|
|
"react-dom": "18.3.0-canary-14898b6a9-20240318",
|
|
"react-router": "^7.5.3",
|
|
"ts-expect": "^1.3.0",
|
|
"typescript": "7.0.2",
|
|
"vike": "^0.4.229",
|
|
"vite": "^6.3.4",
|
|
"vitest": "^3.1.2",
|
|
"wrangler": "^3.63.2"
|
|
}
|
|
}
|