Oleg Isonen
56a22fde8a
## Summary Introduce a document graph architecture for Assets resources so structured documents can reference one another while keeping full document payloads on CDN-backed storage. The content database will retain the searchable index and dependency graph. At runtime, the server will select a root document, determine its complete dependency set before performing I/O, fetch each unique document concurrently, and assemble the final result server-side. ## Implementation todos ### Document graph foundation - [x] Add normalized internal reference contracts for whole documents, JSON paths, Markdown body, and Markdown frontmatter. - [x] Add an immutable graph abstraction with deterministic ordering, dependency traversal, revision validation, and cycle detection. - [x] Add a storage-independent resolver with dependency closure, bounded parallel loading, shared-content deduplication, representation selection, assembly hooks, cancellation, and structured errors. - [x] Add focused unit coverage for the isolated document-graph abstraction. - [x] Define and validate the external remote-document reference syntax. - [x] Add a pure graph compiler that resolves canonical document URLs into IDs and revisions and validates missing, duplicate, and cyclic dependencies. ### Build-time graph - [x] Add a storage-independent source-catalog compiler for bounded parallel reference discovery and deterministic graph compilation. - [x] Add a versioned deterministic document graph artifact with canonical serialization and integrity verification. - [x] Discover and normalize statically declared references while Assets resources are processed. - [x] Resolve document identities and revisions and report missing, stale, or cyclic dependencies with document context. - [x] Persist document nodes and dependency edges alongside searchable resource metadata. - [x] Include persisted graph data in artifact integrity and size validation. ### Document representations - [x] Add a standalone JSON document adapter for exact reference markers, compiler occurrences, JSON path selection, and immutable assembly. - [x] Add a standalone Markdown document adapter for bounded decoding, frontmatter references, immutable assembly, and source, body, and frontmatter selection. - [x] Parse fetched JSON and Markdown documents through format-specific adapters. - [x] Select complete documents, JSON fragments, Markdown body, and Markdown frontmatter through the shared reference model. - [x] Assemble nested and shared references into the final server-side value. ### Runtime resolution - [x] Enforce reachable-document, per-document byte, and aggregate-byte limits before parsing and assembly. - [x] Add a revision-keyed bounded source-cache wrapper with replayable bytes, stale-entry fallback, and in-flight deduplication. - [x] Add a storage-neutral loaded-source contract that rejects revision mismatches before parsing. - [x] Resolve graph-backed property paths before filtering, sorting, pagination, and projection; skip graph loading for unrelated queries. - [x] Add an injected HTTP/CDN document loader with revision metadata validation. - [x] Apply runtime payload limits, bounded concurrency, caching, and structured failure mapping. - [x] Ensure shared documents are fetched and parsed once per resolution. ### Product integration - [x] Integrate equivalent behavior in local preview, SSR, SSG, and published runtime paths. - [x] Preserve existing embedded Assets resource behavior without requiring migration. - [x] Add observability for graph loading, document fetching, caching, and resolution failures. - [x] Document the public reference and resolution behavior. ### Review hardening - [x] Enforce graph, artifact, cache, and published-storage identity consistency. - [x] Compile only graph sources reachable from selected build entries. - [x] Resolve graph-backed fields before query evaluation and avoid unnecessary graph loading. - [x] Consolidate duplicated graph schemas, utilities, and document content-type detection. - [x] Correct runtime documentation and use structural evaluation validation. ### Verification - [x] Keep a deterministic high-impact agent evaluation for Markdown-to-JSON references across the dynamic blog overview and article routes. - [x] Add a separate discoverability evaluation that must learn the reference workflow from MCP guidance without prompt-supplied syntax, queries, bindings, or call order. - [ ] Re-run the complete local agent evaluation suite after the discoverability fixture change and compare every fixture with its accepted baseline. - [x] Add integration coverage for CDN-backed hydration, parallel fetching, caching, and shared dependencies. - [x] Add integration coverage for missing references, cycles, revision changes, limits, and cancellation. - [x] Verify equivalent behavior across preview, SSR, SSG, and published deployments. - [x] Verify that existing embedded Assets resources retain their current behavior. ## Intended outcome - Storage-backed documents do not require their complete payloads to be embedded in the content database. - The server knows the complete statically discoverable dependency set before fetching document content. - Shared dependencies are fetched once per resolution and independent resources can be fetched in parallel. - JSON fragments and Markdown body/frontmatter representations use the same document-resolution system. - Reference and payload revisions cannot be mixed silently. - Existing Assets resources continue to work without migration. ## Verification results - `@webstudio-is/content-engine`: 37 files / 244 tests passed; typecheck passed. - `@webstudio-is/asset-uploader`: 22 files / 182 tests passed; typecheck passed. - CLI: 57 files / 925 tests passed, including generated SSR and SSG build/prerender coverage; typecheck passed. - Repository-wide `pnpm oxfmt --check` passed across 2,757 files. - Package-boundary validation passed. - Previous `pnpm evaluations` run: all 4 original high-impact fixtures passed. The complete 5-fixture rerun is pending explicit approval after adding the discoverability fixture.
Webstudio CLI
The Webstudio CLI helps you to link, sync and build projects from Webstudio Cloud. This README will guide you through the process of setting up a Webstudio project on your local machine and continually sync it with the cloud.
Prerequisites: Installing Node.js
You need Node.js to use the Webstudio CLI. If Node.js is already installed in your system, you can skip ahead to the section on installing the Webstudio CLI.
To install Node.js using NVM, first install NVM by running:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash
Once NVM is installed, you can install Node.js version 22 by running:
nvm install 22
Verify your Node.js installation by checking its version:
node --version
Installing the Webstudio CLI
To get started with the Webstudio CLI:
-
Download and install the CLI using the following command:
npm install -g webstudio@latest -
Confirm the installation by checking the CLI version:
webstudio --version -
To keep your CLI updated, use the same command used for installation whenever a new release is available.
Initiating a Webstudio Project
Now, you can run a Webstudio project on your local machine using this command:
webstudio
This will initiate the flow to connect your Webstudio Cloud project and build it in your local computer. The default flow will guide you through the steps. You can also perform all the operations individually using independent commands.
Commands
Here is the list of independent commands:
- version
- help
- link
- sync
- build
link
The link command stores access to a Webstudio Cloud project. MCP clients use that connection to read and edit the latest Builder build directly; the project does not need to be published or synchronized first.
Shell-capable agents can use local CLI commands such as webstudio meta.index immediately after linking. webstudio connect <client> is optional persistent native MCP registration and is unnecessary for local CLI usage.
You can link a project from Webstudio Cloud with the following command:
webstudio link
This command will prompt you to paste a link which you can create using the Share option in your project.
Make sure to provide Build access when generating the link in Webstudio Cloud.
sync
Once the project is linked, use the sync command to sync it with the cloud:
webstudio sync
Make sure to publish the project in Webstudio Cloud before running the sync command in your local Webstudio project.
sync is only for downloading a published project export. It is not part of the MCP editing workflow.
build
Now, you can build your project with the build command:
webstudio build
During this phase, the CLI establishes the necessary routes and pages, scaffolding the entire application using the default Remix template. Additionally, all assets, such as images and fonts are downloaded to the assets folder inside the public directory.
Once the project is scaffolded, you can run npm install and then npm run dev to run your app in development mode.
If you want to build a production version of the app, you can run npm run build.
Deployment
Vercel
Once you've built the project locally, you can use the Vercel CLI to deploy your project directly to Vercel:
vercel deploy
Follow the instructions here to install the vercel CLI. We plan to add more deployment targets in future.
Netlify
If you want to deploy to netlify, you can use Netlify CLI to deploy your project directly to Netlify:
netlify deploy
You can configure the project to support netlify serverless/edge-functions respectively, as deployment target at the time of initially setting up your project. Please check the initiating-a-webstudio-project section.
You can manually change it using the build command. For serverless functions:
webstudio build --template netlify
Important Notes
If you use vercel build before vercel deploy, make sure to clean your app folder in the project afterward.
Vercel injects a few files to support and deploy Remix using their CLI, but these files are not necessary for your project when you use it locally.