mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-10-28 23:57:17 +09:00
codex/headers-settings-docs
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleg Isonen
|
e69a3d2f8e |
feat: Create structured MDX entries from collection folders (#6317)
## Paragraph editing follow-up — |
||
|
Oleg Isonen
|
05706b5425 |
feat: Make Content Block templates the MDX authoring interface (#6307)
Closes #6306 ## Outcome Content Block templates are now the public component model for MDX authoring. - Standard Markdown resolves through matching Content Block templates. - PascalCase JSX resolves to a named template first, then to a registered component. - Adding or changing a template immediately affects matching MDX, including content authored before that template existed. - Every supported standard MDX element is present in new Content Blocks by default. - Template `name` is a stable PascalCase JSX identifier. `label` remains optional display text only. - Lowercase JSX represents native HTML and SVG without exposing `ws.element` in authored syntax. - Unknown, ambiguous, and unsafe JSX stays in source, produces a source-ranged Builder diagnostic, and is omitted from published output. - Frontmatter images preserve exact `$ref` values and use the Asset description for `alt` text. - Keyboard and mouse insertion from the slash menu remove the trigger slash and preserve the selected template. ## Public MDX contract - Markdown is canonical for standard prose. - Lowercase JSX creates native elements: `<section><p>Text</p></section>`. - PascalCase JSX uses templates and registered components: `<PromotionCard />`, `<Heading tag="h1">Title</Heading>`, and `<YouTube src="https://youtu.be/KI5JHpzBK1s" />`. - Component-name collisions use stable qualified identifiers such as `Checkbox` and `RadixCheckbox`. - A self-closing template keeps its designed default subtree. An explicitly paired empty template clears it. - Static quoted values and bare booleans are supported. - Imports, exports, expressions, spreads, functions, executable JavaScript, unsafe attributes, `<$.*>`, and authored `<ws.element>` are rejected. - `ws` remains only for compiler primitives with no React component. ## Implementation - [x] Add stable, unique template names and keep them separate from labels. - [x] Parse and validate Markdown and JSX through their ASTs with precise source ranges. - [x] Share component identity between template JSX, Content MDX, Builder materialization, reload, copy, and publication. - [x] Resolve templates live after add, edit, rename, reorder, removal, undo/redo, collaboration updates, and reload. - [x] Preserve authored props, children, Asset references, template defaults, and the exact slash-menu template choice. - [x] Make one typed MDX semantic inventory drive default templates and slash-menu eligibility. - [x] Move component-dependent template assembly into the component registry. - [x] Replace production `$`, `animation`, and `radix` JSX namespaces with direct component imports. - [x] Keep only explicit compiler primitives in `ws` template JSX. - [x] Migrate repository templates, fixtures, stories, examples, MCP guidance, CLI guidance, and generated outputs. - [x] Add Builder unit and Playwright coverage for authoring, diagnostics, insertion, synchronization, reload, and publication. - [x] Review and update the authoritative Content Block and generated MCP documentation. ## Compatibility - Persisted project data continues to use namespaced component IDs and `ws:element` for generic HTML elements. No project-data migration is required. - Legacy `<ws.element ws:tag="…">`, `<ws.element ws:name="…">`, `<$.*>`, `<animation.*>`, and `<radix.*>` input remains readable during the compatibility window. - Current serializers and repository-owned sources emit only canonical Markdown, lowercase intrinsic JSX, or PascalCase identifiers. - Untouched user MDX is not rewritten automatically. Legacy syntax is canonicalized when its owning source is edited. - Deferred follow-up: remove legacy namespace readers and proxy support only after the canonical writer is deployed and existing projects are backfilled. This is intentionally outside this PR. ## Verification - [x] CI package tests, typechecks, lint, package boundaries, generated assets, migrations, and size checks pass. - [x] All six Builder E2E shards pass. - [x] Fixture checks pass on Ubuntu and Windows. - [x] Targeted local Content Engine, Template, SDK, registry, Project Build, migration, Builder, component, CLI, and Playwright tests pass. - [x] Runtime contracts, CLI docs, and MCP docs were regenerated from their owning sources. - [x] Local fixture generation completed without an unexpected fixture diff. - [x] Documentation impact reviewed and included in this PR. - [x] Review and approve the intentional List visual snapshot change. - [x] Run the complete agent evaluation suite after explicit approval. - [x] Run the network fixture link/sync pipeline against the matching PR deployment after explicit permission. ## Known visual change The generated List story now targets its actual default `ul` element instead of `ol`, so the default list styling appears in the visual snapshot. The visual change was reviewed and the approved visual check passes. |
||
|
Oleg Isonen
|
bc8630f413 |
fix: Gate Builder releases on the published CLI contract (#6282)
Closes #6280 ## Summary - Expose the deployed Builder's CLI-facing bundle contract through a non-cached, read-only endpoint. - Resolve and execute the exact `webstudio@latest` registry artifact, observe the bundle version sent by its real `sync` request, and require an exact match before assigning the release staging alias. - Include expected and received bundle versions in runtime and release-gate errors. ## Technical choices - The existing generated `bundleVersion` remains the single compatibility boundary. Builder-only changes outside that project-bundle contract do not require a CLI release. - The CLI sync probe runs against a local fixture API and temporary directory. It reads the deployed runtime contract but never reads or modifies a Cloud project. - Verification runs after Vercel creates the exact production-built deployment and before the release staging alias is assigned. A mismatch therefore leaves production domains untouched. - Registry resolution requires version and integrity metadata, then executes the resolved immutable version rather than source from the checkout. - The gate requires the deployed Builder origin and cannot fall back to the checkout's contract. ## Todo 1. [x] Add the deployed Builder contract endpoint. 2. [x] Probe the exact published CLI through its real sync request. 3. [x] Gate the release staging alias on an exact contract match. 4. [x] Report expected and received versions for compatibility failures. 5. [x] Add regression tests and verify the production Builder build. 6. [x] Review documentation and fixture impact. ## Verification - `pnpm --filter @webstudio-is/builder exec vitest run app/services/build-router.server.test.ts app/services/cli-compatibility-route.node.test.ts` — 12 passed. - `pnpm --filter webstudio exec vitest run scripts/builder-contract-gate.test.ts scripts/release-smoke-registry.test.ts` — 7 passed. - Rebased onto `main` at `9ec4c820e1`. - Builder and CLI typechecks passed. - `pnpm lint` and changed-file formatting passed. - `pnpm --filter @webstudio-is/builder build` passed and emitted `/rest/cli-compatibility`. - The changed GitHub Action parses as valid YAML. - Real registry probe passed for `webstudio@0.293.0` against `bundle-kzb45v`. - Real registry probe correctly blocked current Builder contract `bundle-1mym82p` and reported expected `bundle-1mym82p`, received `bundle-kzb45v`. - Missing staged Builder origin fails closed before registry execution. - GitHub CI passed all four workflows, including all six Builder E2E shards. - The formerly failing empty-MDX insertion workflow passed 8 repeated runs with 4 workers locally, then passed in Builder E2E shard 4 in CI. - Full browser-based release smoke was not run, per the request not to use Chrome. Agent evaluations were not run because no evaluation approval was requested or received. - Documentation: no user-facing documentation changes are required. - Fixtures/generated outputs: no fixture inputs or generated outputs are affected. ## Operational boundary The check needs the exact deployed runtime, so Vercel creates an unaliased `--skip-domain` deployment before verification. A failed check cannot affect production domains or the normal release alias. Repository automation cannot prevent a Vercel administrator from manually promoting an arbitrary deployment outside this release path. |
||
|
Oleg Isonen
|
58eaf8c7c9 |
fix: Publish docs independently of release smoke (#6120)
## Summary - finish the `publish` job after npm publication and exact registry-artifact verification - run packaged CLI verification in a separate `release-smoke` job - keep `publish-docs` dependent on successful release creation and package publication, but independent of post-publish smoke results - verify the Codex registration command through `webstudio connect codex --print` in CI instead of requiring an installed Codex client ## Root cause The release workflow ran post-publish CLI smoke verification inside the `publish` job. GitHub therefore marked package publication as failed whenever the smoke failed, even though npm publication had already completed. Because `publish-docs` declared `needs: [release, publish]`, GitHub skipped documentation publication. The recurring failure was the smoke invoking `webstudio connect codex` on a runner without the `codex` executable. ## Technical choices and tradeoffs Package publication remains strict: npm publication and exact registry visibility must both succeed before downstream jobs run. Post-publish smoke verification remains a red, visible release job and still uploads its diagnostic artifact, but it no longer changes the truthful outcome of publication or blocks docs. The Codex package smoke now validates the generated registration command without mutating a real Codex installation. The command subprocess boundary remains covered by the existing focused connect-command tests. No migrations or compatibility changes are involved. ## Todo - [x] Separate post-publish CLI smoke from package publication. - [x] Keep docs gated on successful release and registry publication. - [x] Make Codex connection verification deterministic in CI. - [x] Manually advance the `docs` branch to release 0.290.0. - [x] Review documentation impact: no product documentation update is required because this changes release automation only. - [x] Verify formatting, workflow syntax and topology, CLI tests, lint, and typechecking. - [x] Run the non-model release smoke and record the remaining unrelated failure. ## Verification - `pnpm exec oxfmt --check .github/workflows/release.yml packages/cli/scripts/release-smoke.ts` - parsed `.github/workflows/release.yml` with Ruby Psych - asserted the parsed workflow job graph with the repository's `yaml` dependency - `pnpm --filter webstudio exec vitest run scripts/release-smoke-registry.test.ts src/commands/connect.test.ts` — 22 tests passed - `pnpm --filter webstudio typecheck` - `pnpm exec oxlint packages/cli/scripts/release-smoke.ts` - `pnpm --filter webstudio test:release-smoke` — the corrected Codex path passed; the smoke later reached `preview.start` and failed on an unrelated current-main fixture error: `Project session is missing preview data: assetFolders` Fixture review: this change does not modify fixture inputs or generated fixture outputs. Closes #6119 |
||
|
Oleg Isonen
|
93fa799cb8 |
fix: Preserve SSG exports with Assets queries (#6102)
## What changed - Preserve the SSG asset-resource template as a package-relative file URL in the bundled CLI. - Ignore Assets queries that do not bind any parameters from the current dynamic SSG route, while retaining the error for partial or split multi-parameter bindings. - Recognize the persisted `$ws$system.params.*` expression form when enumerating dynamic SSG paths. - Add a required package smoke test that deploys the release-shaped SSG package, runs the packaged CLI, runs Vite and Vike, and verifies rendered Assets data. ## Why Two gaps caused the staging export failures: 1. Vite inlined the template URL as a `data:` URL, which Node could not pass to `readFile`. 2. Dynamic-route validation treated an unrelated Assets query as an incomplete route enumerator. The initial smoke test only checked that an HTML file existed and could pass while resource loading logged a `400`. The replacement test verifies the rendered query result and fails on resource-loading errors. ## Technical choices - The package smoke uses `createPublishedProjectBundleFixture`, so fixture data stays checked against the published bundle contract. - Persisted expressions use `encodeDataSourceVariable`; the generated query now receives the real route parameter instead of compiling to `undefined`. - Temporary deployment, cleanup, subprocess execution, and assertions live in `test:ssg-package`. The workflow invokes one repository command. - Asset downloads are disabled to keep the smoke deterministic. It still covers CLI packaging, the embedded Assets index, dynamic path enumeration, generated resource execution, Vite compilation, and Vike prerender. - The test asserts the queried title in `/posts/hello-world`, rejects any resource-load error, and verifies that an unrelated query does not invent a dynamic path. ## Todo - [x] Reproduce the invalid URL scheme through a bundled template reader. - [x] Preserve the package-relative template URL in the production CLI bundle. - [x] Ignore Assets queries unrelated to the current dynamic route. - [x] Preserve rejection of incomplete multi-parameter route queries. - [x] Support persisted system-variable expressions during route enumeration. - [x] Add a production-shaped packaged CLI and Vike smoke test. - [x] Make the packaged SSG smoke test required on `main`. - [x] Run CLI tests, typecheck, lint, build, and the packaged smoke test. - [x] Rerun the previously failing `WEBSTUDIO-LANDING` SaaS SSG workflow. - [x] Review documentation impact; no user-facing documentation changes are needed. - [x] Review fixture impact; synced fixture inputs and generated outputs are unaffected because smoke data exists only in a temporary deployment. ## Verification - `pnpm --filter webstudio test` — 61 files, 977 tests passed before the final test addition - `pnpm --filter webstudio test -- prebuild.test.ts ssg-asset-resource-fetch-template.test.ts` — 57 tests passed - `pnpm --filter webstudio test:ssg-package` - `pnpm --filter webstudio typecheck` - `pnpm lint` - `pnpm --filter webstudio build` - Package regression proof: the smoke failed on the original unrelated-query implementation with `Dynamic SSG route parameters must be completely enumerated by one Assets query`, then passed with the fix restored. - Persisted-expression regression proof: the focused test returned no prerender path before encoded-system support, then passed with the support restored. - Template regression proof: the focused bundle test failed with `ERR_INVALID_URL_SCHEME` before the fix and passed after it. - Exact SaaS E2E rerun passed: project sync, CLI build, Vike build, archive upload, and database update ([run 31519515809](https://github.com/webstudio-is/webstudio-saas/actions/runs/31519515809)). Closes #6100 Supersedes #6101 after moving the change to the staging branch. |
||
|
Oleg Isonen
|
e4983b0d1d |
fix: Report unsupported Node versions (#6058)
Closes #6057 ## Outcome Generated projects now reject Node.js versions older than 22.12 before React Router, Remix, or Vite evaluates SDK components. Users see the package-manager engine error with the current and required versions instead of the indirect `sanitize-html` / `htmlparser2` `ERR_REQUIRE_ESM` stack trace. ## Root cause `sanitize-html@2.17.6` synchronously loads the ESM-only `htmlparser2@12`. That dependency combination relies on synchronous `require(ESM)` support, which Node.js enabled by default in 22.12. Generated projects incorrectly declared the broader `>=22` range, allowing an unsupported runtime to reach SSR evaluation. ## Implementation - Set the CLI and all generated base templates to `engines.node: >=22.12.0`. - Add npm `devEngines.runtime` with `onFail: error`, which runs before install, CI, and run commands. - Enable `engine-strict` in generated npm configuration so pnpm also rejects an unsupported runtime. - Regenerate every tracked CLI fixture package manifest. - Update CLI documentation and the release-smoke manifest contract. - Remove the earlier sanitizer-bundling workaround from the branch; sanitizer code and packaging are unchanged from `main`. ## Checklist - [x] Declare the exact Node.js compatibility floor - [x] Fail before generated development, build, and start commands - [x] Cover the generated engine contract with a fail/pass regression - [x] Regenerate and inspect all affected fixtures - [x] Review documentation impact and update the CLI guide - [x] Keep sanitizer behavior and SDK packaging unchanged ## Verification - Regression demonstrated failing before the template metadata change and passing afterward - `pnpm --filter webstudio test` — 946 passed - `pnpm --filter webstudio typecheck` - `pnpm --filter webstudio build` - `pnpm fixtures` - `pnpm lint` - `pnpm exec oxfmt --check` on changed source, template, fixture, and documentation files - `git diff --check` Agent evaluations were not run because they require separate approval. |
||
|
Oleg Isonen
|
57d9ad14cd |
fix: Retry published CLI registry lookup (#6013)
## What changed - Retry npm version and integrity resolution inside the CLI registry smoke test. - Keep retry timing injectable so the propagation race is covered by a fast unit test. - Preserve the final npm error when all registry attempts are exhausted. ## Why CLI 0.286.0 was published successfully, but npm briefly exposed integrity metadata while the exact version lookup still returned 404. The post-publish smoke failed and the dependent docs publication job was skipped. The 0.286.0 documentation snapshot has been recovered directly on the `docs` branch. Ref #6007 ## Verification - [x] Regression test demonstrated failing before the implementation and passing afterward - [x] CLI registry/versioning tests: 3 passed - [x] CLI typecheck - [x] Formatting and diff checks - [x] Documentation impact reviewed: no product-doc source change is needed in this PR; the missed 0.286.0 snapshot was published separately to `docs` |
||
|
Oleg Isonen
|
3ccb46cec7 |
fix: restore Marketplace settings and optimize Assets publishing (#5964)
## Outcome Restore the Marketplace section for projects without existing marketplace metadata, make the complete review-submission workflow available over MCP, clarify shared Assets database diagnostics, and reduce published database size by materializing fully static Assets queries. ## Implementation ### Marketplace and MCP - Initialize the Marketplace form from a safe partial default instead of rendering nothing when metadata is unconfigured. - Keep review submission disabled until the complete metadata schema is valid. - Allow `get-marketplace-product` to return `null` and `update-marketplace-product` to create the first marketplace product through ProjectSession/MCP. - Add a confirmed `submit-marketplace-product` server operation that validates configured metadata before setting review status to `PENDING`. - Document the MCP workflow: upload the thumbnail, update metadata, publish production, then submit. - Correct the Marketplace thumbnail upload example to use the current MCP input field. ### Assets database - Show query size and merged published-database size separately in resource diagnostics. - Clarify that only the merged database counts toward the publish limit. - Allow the Assets query editor to scroll long query source. - Materialize fully static metadata-only Assets queries at publish time. - Store materialized results as compact field columns and rows keyed by a deterministic normalized-query hash. - Serve materialized rows through the existing Assets response contract without candidate documents or a runtime field catalog. - Keep route- or variable-dependent queries on the existing document-backed runtime path. - Exclude overview-only fields from dynamic detail documents so the merged database does not duplicate them. - Preserve byte-limit selection, truncation diagnostics, integrity verification, caching, and backward compatibility with existing artifacts. ### Release - Keep the release lockfile stable when preparing the CLI package. - Regenerate runtime contracts, server-only API metadata, and CLI documentation where required. ## Todo - [x] Restore the empty Marketplace settings state - [x] Prevent empty review submissions - [x] Support first-time Marketplace metadata creation over MCP - [x] Add MCP review submission - [x] Document the end-to-end MCP workflow - [x] Correct query and merged-database diagnostics - [x] Make long Assets query source scrollable - [x] Materialize static overview queries - [x] Preserve dynamic detail-query behavior - [x] Add regression coverage with fail/pass proof - [x] Run relevant package and integration checks ## Verification - [x] Content engine tests: 158 passed - [x] Asset uploader tests: 179 passed - [x] SDK tests: 505 passed - [x] CLI tests: 921 passed - [x] CLI prebuild integration tests: 46 passed - [x] Builder Assets resource and diagnostics tests: 14 passed - [x] Project runtime and ProjectSession tests: 133 passed - [x] Builder API router tests: 30 passed - [x] Protocol operation tests: 18 passed - [x] Content engine build and declaration generation - [x] Content engine, asset uploader, SDK, Builder, and CLI typechecks - [x] Repository lint and formatting checks - [x] Generated API and documentation checks - [x] New materialization tests demonstrated failing with the implementation disabled and passing after restoration ## Notes - The representative five-post overview fixture shrinks from 2,122 bytes to 1,197 bytes while returning the same sorted and paginated result. - Materialization is intentionally limited to static, metadata-only field queries without runtime asset fields. Unsupported or data-incompatible queries safely use the existing runtime database. - The complete `pnpm evaluations` agent suite was not run because repository policy requires separate explicit approval. - No database schema or migration changes are required. |
||
|
Oleg Isonen
|
efbc3098c4 |
feat: add queryable Assets resources (#5946)
Closes #5919 Ref #5920 ## Outcome Extend the existing Assets system resource with optional queries for Markdown, JSON, and standard file metadata. Unconfigured Assets resources retain the existing fetch-all behavior and activate none of the content-query infrastructure. Files are the authoring model. A standalone content engine converts an immutable snapshot of only the files required by configured queries into one bounded, deterministic runtime database. Builder preview, SSR, and SSG use the same query semantics without requiring a database service for a published site. ## Final architecture ```text Markdown / JSON files │ ▼ ContentSource adapter (FS or Webstudio Assets) │ immutable snapshot ▼ ContentCompilationPlan (union of configured query requirements) │ ▼ ContentCompiler (parse, normalize, discover fields, enforce budget) │ ▼ ContentArtifact (bounded, deterministic, immutable JSON) │ ▼ ContentDatabase (filter, OR groups, sort, paginate, project, read content) │ ├── Builder preview ├── SSR runtime └── SSG build only ``` The authoring/storage layer owns mutable files. The content engine consumes immutable snapshots and produces immutable, disposable artifacts. Query authoring is a separate OpenAPI-driven path: ```text Assets REST OpenAPI operation + standard request JSON Schema + dynamic field paths │ ▼ Generic OpenAPI adapter │ resolved schemas, controls, paths, and transport mapping ▼ Generic query-builder core and React renderer │ arbitrary provider-defined value ▼ Assets REST request adapter │ ▼ { query: value } ``` The selected standard OpenAPI operation is the only source of query UI semantics. Request parameters and JSON Schema describe the editable data; the generic renderer derives controls and owns layout. ### Activation and compilation - The compiler does not index every asset. It combines all reachable configured Assets-resource queries into one `ContentCompilationPlan`. - Static constraints such as folder, path, and file type select candidate files. Dynamic values such as a route-bound slug cannot reduce the build-time candidate set. - The plan contains the union of fields used by filtering, sorting, routing, projection, excerpts, and requested full content. - Documents shared by multiple resources are stored once in one deployment-wide runtime database. - Markdown/frontmatter, JSON properties, excerpts, and full bodies are prepared only when required. - No configured content query means no parsing, artifact, or shipped runtime dependency. Fresh generated apps need no content-runtime cleanup; incremental generation removes stale integration if a previously configured query is deleted. ### Runtime database limit - `CONTENT_DATABASE_MAX_BYTES` limits the complete serialized runtime database, not source files, images, or unrelated assets. - The default is `500 * 1024` bytes and is presented as 500 KiB. - Counted data includes indexed metadata, frontmatter/JSON properties, excerpts, the field catalog, serialization overhead, and full Markdown bodies only when requested. - Prepublish validation builds and serializes the exact database that will be attached to the immutable build, so it knows the unbounded size, bounded size, included documents, omitted documents, and affected resources. - Overflow never stores partial documents. Candidates are ordered by `createdAt` descending, canonical path ascending, and stable document ID ascending. Missing dates follow dated documents. A document that does not fit is skipped while smaller later documents are still considered. - Exceeding the limit warns but does not block publishing. - Builder preview can report exact bounded versus unbounded results when dynamic expressions have concrete preview values. Prepublish warnings remain conservative for future route values. ### Snapshot consistency The source contract guarantees a consistent compilation, while adapters provide the mechanism: - Webstudio Assets captures asset metadata and immutable content references. - Filesystem sources capture file identity and verify files did not change before accepting the artifact. - A changed source retries once, then fails clearly instead of producing a mixed-revision database. - This requires no universal snapshot service, source-tree copy, or new versioned-asset infrastructure. ### Package ownership - `@webstudio-is/content-engine`: runtime artifact schema, query execution, field catalog, compact runtime statistics. - `@webstudio-is/content-engine/compiler`: source snapshots, compilation plans, Markdown/JSON parsing, deterministic compilation, byte accounting, and compiler diagnostics. This entry exists to keep Remark/YAML/future MDX dependencies out of runtime bundles. - `@webstudio-is/query-builder`: resolves standard OpenAPI operation schemas into an internal provider-neutral editor model, validation, mutations, and source serialization. It owns no Assets, provider-authored UI contract, or fixed REST-query semantics. - `@webstudio-is/query-builder/react`: container-neutral schema renderer whose layout is renderer policy. It remains separate from the content engine. - SDK: Webstudio Assets/Resource domain models and mapping Webstudio expressions into generic content requirements. - Protocol: external Assets REST/OpenAPI transport contracts. - Asset uploader/repository: asset and folder mutations, generic metadata, immutable byte references, and storage access. Its hosted adapter supplies Webstudio Assets snapshots and parsed-metadata cache persistence to the content engine; parsing, artifact creation, and query semantics remain engine-owned. - Builder and generated-app integrations decide when to compile, cache, embed, load, or discard a database. ### Runtime lifecycle - Builder caches immutable `ContentDatabase` instances by source revision plus compilation-plan hash and coalesces concurrent compilation. Matching operations reuse the same instance; field discovery uses a dedicated minimal all-properties plan that excludes excerpts and bodies. - SSR attaches the validated bounded database to the build and includes the small runtime only when required. - SSG uses the database only while generating pages and omits it and the runtime from final output unless a future runtime-query feature requires them. - Local development compiles from the filesystem and uses identical query semantics. - Published pages query their local artifact; they do not call Builder REST endpoints. - Published artifacts need no external index store, invalidation, garbage collection, SaaS content proxy, new bucket, or new environment variable. ## Implemented foundation These existing parts remain useful and should be moved behind the final ownership boundaries rather than reimplemented. - [x] Upgrade the existing Assets system resource in place and preserve unconfigured fetch-all behavior. - [x] Implement typed filtering, nested boolean groups, deterministic sorting, pagination, projections, dynamic values, and content hydration. - [x] Add Markdown frontmatter, excerpt, JSON-property, and field-catalog extraction. - [x] Add reusable query-control implementations and a separately imported React UI as a foundation for the final OpenAPI-driven contract. - [x] Add Assets query authoring and dynamic field discovery as existing behavior to migrate into the Assets OpenAPI operation. - [x] Add synchronized visual and editable expression authoring. - [x] Unify Assets REST operations under `/rest/assets/*`. - [x] Define authenticated collection, item, content, upload, folder, query, field-catalog, and OpenAPI operations. - [x] Support Builder cookie authentication and public API token authentication. - [x] Add Builder preview, MCP/CLI operations, SSR execution, SSG path generation, and conditional activation. - [x] Add revision-keyed parsed metadata persistence for the hosted Builder. - [x] Add focused query, API, authorization, Builder, CLI, SSR, SSG, migration, and fixture coverage. ## Remaining implementation plan ### 1. Establish the content engine - [x] Rename the generic `@webstudio-is/asset-resource` implementation to `@webstudio-is/content-engine`. - [x] Move existing behavior instead of creating a parallel implementation. - [x] Keep the primary entry runtime-only and add only the compiler entry needed for bundle separation. - [x] Remove the old package after all consumers migrate. - [x] Ensure the engine has no Builder, SDK, asset-uploader, PostgreSQL, R2, or deployment-target dependency. ### 2. Define source snapshots and compilation plans - [x] Introduce `ContentSource.openSnapshot()` and a revision-bearing `ContentSourceSnapshot`. - [x] Include canonical path, stable document ID, content type, immutable content reference, size, and optional `createdAt`. - [x] Implement Webstudio Assets and filesystem snapshot guarantees using their adapter-specific mechanisms. - [x] Introduce `ContentCompilationPlan` for the union of reachable configured resource requirements. - [x] Separate statically selectable candidates from dynamic runtime predicates. - [x] Add mutation-during-compilation tests for replacement, deletion, folder movement, and filesystem changes. ### 3. Separate storage from content processing - [x] Keep upload, replacement, folders, byte storage, revisions, and generic metadata in the Assets repository/uploader. - [x] Move parsing, excerpts, field discovery, query execution, and artifact creation into the content engine. - [x] Remove content-index coordination from asset mutation business logic. - [x] Preserve original source `createdAt` across content replacement and expose it as optional queryable metadata. - [x] Keep hosted parsed-document persistence behind the content compiler's optional cache interface. ### 4. Build deterministic compilation - [x] Move the existing Markdown, frontmatter, JSON, canonicalization, hashing, document-building, and tests into the compiler. - [x] Keep frontmatter behind a dedicated module and continue using the existing Remark/Micromark pipeline. - [x] Parse only candidate files and only the tiers required by the compilation plan. - [x] Deduplicate documents and fields required by multiple resources. - [x] Define a versioned, deterministic `ContentArtifact` with integrity information. - [x] Keep verbose parse and omission diagnostics outside the runtime artifact. - [x] Reject incompatible artifact versions cleanly. ### 5. Enforce the deployment-wide database budget - [x] Replace the old asset-index limit with `CONTENT_DATABASE_MAX_BYTES`, defaulting to 500 KiB. - [x] Have applications read configuration and pass the byte limit into the compiler. - [x] Measure the exact serialized database, including selected bodies and overhead. - [x] Implement deterministic skip-and-continue packing of complete documents. - [x] Return compiler diagnostics with unbounded/bounded bytes, included/omitted counts, omitted identities, largest contributors, and affected resources. - [x] Expose only compact artifact statistics at runtime through `ContentDatabase.getStats()`. - [x] Add exact boundary, deterministic ordering, deduplication, and oversized-document tests. ### 6. Finalize the runtime database - [x] Define `ContentDatabase` over an immutable artifact. - [x] Preserve filtering, nested AND/OR groups, sorting, offset/limit, projection, field references, and content retrieval. - [x] Expose `getFieldCatalog()` and compact `getStats()`. - [x] Keep Markdown/YAML/compiler dependencies out of the runtime entry. - [x] Add execution guards and maximum-size runtime benchmarks. ### 7. Make query authoring OpenAPI-driven The selected standard OpenAPI operation is the complete query-authoring contract. The query builder resolves parameters and request-body JSON Schema, derives its internal editor model, and applies renderer-owned layout policy. It must not embed or fetch a second capabilities/UI language through `x-webstudio-query` or another side channel. - [x] Remove `x-webstudio-query` and all vendor query-definition metadata from the Assets OpenAPI operation. - [x] Resolve local OpenAPI references and derive the editable request value, defaults, validation, labels, choices, arrays, objects, and unions solely from standard operation parameters and request-body JSON Schema. - [x] Represent dynamic Assets field paths with standard JSON Schema vocabulary in the `queryAssets` request schema. - [x] Keep the normalized editor representation internal to query-builder; do not expose another provider-authored capabilities contract. - [x] Remove provider-controlled layout metadata such as `group`; make compact scalar packing and all other layout renderer policy. - [x] Keep Webstudio expression binding as a consumer/editor concern layered over schema values, not part of the OpenAPI contract. - [x] Preserve the existing Assets filter, sort, output selection, full-content/range selection, limit/offset, editable source, and configured-field behavior. - [x] Keep the Assets resource adapter responsible for serializing the edited query into its request body. - [x] Remove the obsolete Assets query-capabilities endpoint, URL, and UI-side capability parsing. - [x] Add a non-Assets OpenAPI proof with differently named parameters and no vendor metadata. - [x] Verify the generic packages contain no Assets imports, fixed Assets property contract, or provider-authored layout model. - [x] Update focused core, React, SDK, protocol/OpenAPI, Builder, and source round-trip tests. ### 8. Manage ContentDatabase lifecycle in Builder - [x] Cache databases by source revision plus normalized compilation-plan hash. - [x] Coalesce concurrent compilation requests. - [x] Reuse cached database instances across field discovery, autocomplete, query preview, and response preview when their normalized compilation plans match. - [x] Replace it when source revision or query requirements change. - [x] Preserve the optional revision-keyed hosted parsed-document cache without making it a source of truth. - [x] Show exact preview warnings when concrete values reveal missing bounded results. ### 9. Materialize and verify immutable builds - [x] Compile and serialize the exact deployment-wide database during prepublish validation. - [x] Attach that validated bounded database to the immutable build without rebuilding it from later asset state. - [x] Show non-blocking prepublish warnings with database bytes, omissions, and potentially affected resources. - [x] Avoid claiming exact future route matches when dynamic values are unknown. - [x] Include the database/runtime only when a reachable configured query requires them. - [x] Ensure detail resources requesting Markdown content include all statically eligible bodies needed by dynamic routes. ### 10. Integrate all targets - [x] SSR: load the build's bounded database and conditionally include the small runtime. - [x] SSG: query the same database during generation, then omit database/runtime output when no runtime query is needed. - [x] Local/CLI: compile from the filesystem source with the same semantics. - [x] Preserve overview pagination, alternative route fields, draft-as-user-data behavior, and full article rendering. - [x] Verify ordinary Assets and fresh query-free sites perform no parser, artifact, content-runtime dependency install, or cleanup work. ### 11. Correct API and package ownership - [x] Move external Assets REST/OpenAPI contracts to `@webstudio-is/protocol`. - [x] Keep Webstudio domain models and Assets system-resource configuration in the SDK. - [x] Keep both cookie and token authentication behavior on the unified Assets REST surface. - [x] Remove pass-through route wrappers/reexports that add no transport behavior. - [x] Keep published runtime queries local to the artifact rather than routing them through Builder REST. ### 12. Remove superseded deployment infrastructure - [x] Remove external deployment-index upload, fetch, invalidation, cleanup, and garbage-collection code. - [x] Remove the SaaS asset-origin adapter requirement; close webstudio-is/webstudio-saas#652 as superseded because it has no independent purpose. - [x] Remove index-bucket compatibility configuration and avoid adding Infisical variables. - [x] Remove obsolete external-index paths, package dependencies, routes, scripts, and tests. - [x] Consolidate unreleased intermediate migrations where safe and retain only schema required by the final hosted cache/API design. ### 13. Verification and release readiness - [x] Move and preserve existing tests while changing ownership; add new tests only for changed contracts and behavior. - [x] Run content-engine compiler/runtime tests, query-builder tests, SDK/protocol/API tests, Builder tests, CLI tests, and affected typechecks. - [x] Run formatting, generated-file idempotence, fixtures, clean Supabase-compatible migrations, E2E shards, and complete agent evaluations. - [x] Verify SSR and SSG conditional inclusion and query-free generated package manifests. - [x] Run the five-article MCP blog workflow: authoring, field discovery, overview, dynamic detail routes, full Markdown rendering, publishing, and published-site verification. - [x] Run a maximum-budget database test and a mutation-during-compilation test. - [x] Perform a final architecture, security, API compatibility, migration, bundle, correctness, and DRY review. - [x] Reconcile every checkbox and replace the verification section with results from the final architecture before release. ## Future MDX work MDX is intentionally separate from this release. - The compiler contract should allow a future build-only MDX parser. - MDX compilation must not enter the published runtime. - Embedded JSX must use an explicit compiled representation rather than runtime source evaluation. - Webstudio components, tokens, styles, and component allowlists remain Webstudio integration concerns. - The future MDX work should reuse `ContentArtifact` and `ContentDatabase` where practical. ## Superseded design The following earlier design choices are no longer the target architecture: - GROQ or GraphQL as the Assets query language. - One external or long-lived index object per resource/query. - A deployment index fetched by the Worker from SaaS/CDN infrastructure. - Lazy production reads of source Markdown bodies. - A separate index bucket, asset origin, invalidation service, or garbage collector. - A 4 MiB published index limit. - Rebuilding an old build's database from the current mutable asset set. - Bundling query infrastructure into projects without a configured content query. ## Verification baseline The embedded runtime-database architecture previously passed the checks below. The OpenAPI-driven query-authoring change additionally passed: - Query-builder core: 4 files / 19 tests. - Query-builder React: 1 file / 7 tests. - SDK Assets source/config: 1 file / 7 tests. - Protocol Assets OpenAPI: 1 file / 9 tests, including independent OpenAPI validation. - Builder Assets OpenAPI description: 1 file / 5 tests. - Query-builder, React, SDK, protocol, and Builder typechecks. - Package-boundary validation and formatting of every affected file. Local verification baseline: - Content engine: 15 files / 104 tests. - Asset uploader: 24 files / 175 tests. - Builder: 204 files / 2,423 tests. - CLI: 54 files / 866 tests. - Complete workspace typecheck, formatting, lint, package boundaries, generated fixtures, and fixture builds. - The complete migration history applied successfully to a fresh Supabase-compatible database, including every migration after the consolidated Assets migration. - Builder production build and full local E2E suite passed. - Complete agent evaluations passed: authenticated page (170,859 ms), design input (207,101 ms), and font assets (67,050 ms). - Final review additionally verified correct REST edit/view/build permits, cookie and public-token read paths, immutable snapshot retries, exact database budgeting, query-free dependency omission, and SSR/SSG conditional runtime inclusion. Final staging verification passed against the embedded runtime-database architecture: MCP uploaded and updated five Markdown articles, discovered dynamic frontmatter fields, previewed a five-item sorted overview and one full-body slug query, updated both Assets resources, found zero binding issues, rendered `/blog` and `/blog/dynamic-routes` at HTTP 200, published build version 266, and verified both published routes at HTTP 200 with the expected article metadata and Markdown body. The workflow also found and fixed offset-form asset timestamps failing canonical datetime validation and made MCP `preview.start({ port: 0 })` correctly select an available local port. ## Release constraints - No new SaaS content service, bucket, or environment variable. - No published-site database dependency. - No parser/compiler dependency in query-free sites or runtime-only bundles. - The validated content database is immutable build output. - Publishing warnings do not block deployment. - The selected standard OpenAPI operation and its request JSON Schema are the only source of query UI semantics. |
||
|
Oleg Isonen
|
e80ca3b54c |
fix: Improve MCP authoring and screenshot reliability (#5957)
Closes #5954 Closes #5951 Closes #5950 Closes #5953 ## Summary - route external screenshot `url` and `baseUrl` targets through the self-closing browser path - keep reusable Chromium sessions limited to Webstudio-managed session previews - validate typed style values at the public runtime boundary while retaining CSS strings for design tokens - build MCP and shell validation messages from compact structured issues - advertise typed style objects with a concrete keyword example - persist `update-asset` metadata patches to the related file record and verify the saved value - store multi-sentence page titles and metadata as fixed text without misclassifying sentence punctuation as member access - capture privacy-safe aggregate token usage and MCP workflow metrics in every high-impact evaluation result - compare each evaluation with a compatible checked-in baseline and fail on correctness, token, tool-use, error/retry, broad-read, or MCP-latency regressions ## Root causes ### External screenshot lifecycle The fast iterative preview change made `session` the default preview source. External captures were then treated as managed session-preview captures and opened a reusable Chromium session. One-shot MCP processes do not call `preview.stop`, so the browser kept the process alive and could be orphaned when the caller killed the hung CLI. ### Validation errors and schemas `update-styles` declared `value` as unconstrained input, and design-token objects were checked only for a string `type`. Invalid values therefore passed the advertised/input schema and failed later in the full `StyleValue` parser, exposing the raw Zod union message. Error serialization already carried flattened issues but did not use them for `error.message`. ### Asset metadata persistence `update-asset` correctly produced an `assets/[id]/meta` patch, but asset persistence compared only Asset-table fields. Metadata lives on the related File row, so the patch was treated as unchanged and the successful mutation never wrote the new value. ### Page text expression disambiguation Page text normalization treated a period followed by whitespace and a word as JavaScript member access. Normal prose such as `Marques. Toutes nos marques` was therefore validated as an expression and rejected instead of being stored as a string literal. ### Evaluation observability The evaluation result only stored duration and two call counts, so token cost, failures, retries, mutation flow, verification coverage, and MCP latency could regress without a durable comparison. Results now use a bounded schema-v2 aggregate and never persist prompts, transcripts, stdout/stderr, credentials, arguments, or response payloads. ## Impact - external screenshots complete without retaining browser processes - repeated managed-preview screenshots retain browser reuse - invalid style inputs fail at the public boundary with paths such as `updates.0.value` - `error.message` is concise and actionable instead of a repeated raw union dump - generated schemas no longer advertise primitive or array values for `update-styles` - supported design-token CSS strings remain accepted - `update-asset` font metadata remains changed after commit, refresh, and reload - `create-page` and `update-page` accept ordinary multi-sentence titles and SEO descriptions while preserving computed expressions - every high-impact run reports input, cached-input, cache-write, output, reasoning-output, total-token, and cache-rate aggregates - MCP results report calls, success/failure, retries, focused/broad reads, dry runs, planned/committed mutations, verifications, total/p50/p95 latency, and first-mutation/verification timing - compatible results are compared with accepted baselines using a 15% relative regression threshold; correctness failures and zero-baseline errors, retries, or broad reads remain zero-tolerance - stochastic wall-clock and milestone timings remain visible as deltas without making a single noisy run fail ## Todo - [x] Reproduce external URL browser-session routing - [x] Use self-closing capture for external screenshots - [x] Preserve browser reuse for managed session previews - [x] Reproduce primitive style-value schema acceptance - [x] Validate typed style values at the public boundary - [x] Preserve CSS-string design-token inputs - [x] Format stdio and shell validation messages from structured issues - [x] Regenerate and verify runtime operation contracts - [x] Reproduce dropped asset metadata at the persistence boundary - [x] Persist and verify related File metadata updates - [x] Reproduce multi-sentence page text failures on create and update - [x] Distinguish sentence punctuation from computed member access - [x] Add privacy-safe token and MCP workflow metrics - [x] Add per-fixture baseline comparisons and regression gates - [x] Refresh accepted schema-v2 baselines for all fixtures - [x] Review the complete diff - [x] Run repository checks and the complete agent evaluation suite ## Verification - `pnpm checks` — passed, including workspace tests, typechecks, lint, package-boundary checks, generated API/docs checks, fixture sync, and fixture builds - builder tests — 189 files, 2,356 tests passed - CLI tests — 53 files, 852 tests passed - project-build tests — 81 files, 2,023 tests passed - asset-uploader tests — 11 files, 98 tests passed - targeted evaluation metrics/regression/trace/runner tests — 5 files, 17 tests passed - all four issue regression tests and the new evaluation harness tests were demonstrated failing without their implementations and passing with them - `pnpm evaluations -- --update-baselines` — 3/3 fixtures passed and schema-v2 baselines were updated - a subsequent real `font-assets-v1` result compared successfully after the regression policy was finalized |
||
|
Oleg Isonen
|
c3c0e32204 |
fix: make MCP font assets editable and deterministic (#5944)
Closes #5943 Closes #5942 ## User impact MCP-managed font assets now keep their identity, metadata, and generated CSS behavior predictable across upload, editing, synchronization, and rendering. ## Changes - Preserve explicit font metadata overrides while validating supported fields and rejecting unsupported metadata instead of silently dropping it. - Expose font metadata through `update-asset`, with partial updates merged into the existing metadata and validated before the mutation is committed. - Prefer the typographic family name when reading font files, so Fontsource and Google font files resolve to the same logical family where the file provides that information. - Deduplicate generated font sources by URL and format, choose a deterministic source when legacy assets collide, and derive the CSS format hint from the filename when historical metadata is stale. - Add MCP and CLI examples explaining font metadata upload and correction. - Add regression coverage for metadata persistence, invalid overrides, family-name extraction, duplicate source handling, stale format metadata, and generated runtime contracts. ## Verification - `pnpm checks` - 46 workspace packages tested, including 2,355 Builder tests, 809 CLI tests, 2,019 project-build tests, and 97 asset-uploader tests. - All workspace typechecks, Oxlint, package-boundary checks, generated API/docs checks, and fixture link/sync/build checks passed. |
||
|
Oleg Isonen
|
12ad74584a |
feat: Improve MCP authoring, auditing, and verification (#5886)
## Summary This improves how coding agents discover, author, verify, and audit Webstudio projects through the CLI and MCP. The work focuses on making agents more reliable with less context, while keeping edits semantic and Builder-editable. ## What changes for users - Agents can verify dynamic bindings after authoring and receive actionable diagnostics when expressions, resources, scopes, or component props do not line up. - Agents can create complete resource-backed runtime UI through a semantic operation instead of coordinating fragile low-level edits. - MCP discovery and command output are compact by default, with pagination and focused verbose detail when an agent needs more context. - Asset, style, token, font, page, and component inspection provides stronger usage context so agents can preserve the existing design system. - Rendered audits provide better evidence for responsive behavior, geometry, contrast, generated-build size, and large-project runs. - Cross-project MCP workflows can run from a manifest with explicit mutation approval, isolated failures, resumable progress, and persistent checkpoints. - Project-specific agent onboarding is clearer and does not add telemetry. - Packaged CLI releases gain stronger smoke verification for install, MCP startup, authoring, preview, screenshots, and audits. ## Reliability and architecture - Runtime operation output schemas are generated from the runtime registry and remain client-compatible. - Project-build exports are consolidated around architectural boundaries instead of feature-by-feature convenience entry points. - HTTP, preview, screenshot, and project-session paths are hardened for longer autonomous workflows. - Craft compatibility work is read-only and provenance-aware; it does not silently opt ordinary projects into Craft behavior. - A minimal-context evaluation harness exercises key workflows with a weaker model and stores bounded, privacy-safe evidence. ## Validation - Rebased on the latest `origin/main`. - `pnpm lint` - Repository-wide `oxfmt --check` - CLI typecheck and full test suite: 773 tests passed. - Project-build typecheck and full test suite: 1,866 tests passed. - Protocol typecheck and full test suite: 83 tests passed. - HTTP client tests: 23 passed. - CSS data tests: 3,620 passed, 1 skipped. - Focused Builder tests: 67 passed. - Package-boundary, generated API, and generated documentation checks passed. - Minimal-context agent evaluations passed for authenticated resource-backed UI and design-system-aware responsive authoring. |
||
|
Oleg Isonen
|
dde30ffb54 |
feat: Improve MCP authoring, compact output, and agent onboarding (#5883)
## Summary This improves how agents discover, author, inspect, and verify Webstudio projects while reducing unnecessary MCP context usage. ## What changed - Adds an **Agents** section to Project Settings with ready-to-run setup commands and client configuration for Claude Code, Codex, Cursor, and VS Code. - Makes large list and schema responses compact and paginated by default, with **verbose** available when complete records or diagnostics are needed. - Applies consistent compact output to assets, instances, styles, design tokens, CLI schemas, and project-session metadata. - Returns actionable expression warnings for props, resources, and Collection bindings, including unknown variables and risky render-time mutation resources. - Improves Collection expression guidance so agents can bind array items with the correct scoped variables. - Reports rendered audit outcomes explicitly as complete, partial, confirmation-required, or failed. - Expands runtime output schemas so MCP clients can validate responses reliably. - Strengthens the packaged CLI smoke workflow with a realistic seven-page stdio MCP scenario. ## Architecture - Consolidates `project-build` from 65 file-oriented entrypoints to 11 architecture-owned surfaces. - Separates the core model, contracts, runtime authoring, transfer, state, project sessions, MCP, visual tooling, persistence codecs, and authenticated server services. - Isolates Node-only JSX evaluation from browser-safe transfer utilities and authenticated server modules. - Keeps persistence codecs independent from authenticated database services, preventing unrelated module initialization. - Defines shared instance-path types once and reuses them across runtime modules. - Preserves Immer initialization through explicit package side-effect metadata. - Adds package tests that prevent a return to source-file entrypoints or generic exposure of Node-only surfaces. ## Validation - Full repository checks, tests, typechecks, generated-file checks, and fixtures - All 163 Builder test files and 2,141 tests - All 71 project-build test files and 1,794 tests - Lint and package-boundary validation - Full packed CLI release smoke test, including install, preview, agent setup, seven-page stdio MCP workflow, rendered audit, and structured commands |
||
|
Oleg Isonen
|
d186d24501 |
feat: Improve MCP project audits, visual verification, and dynamic content authoring (#5874)
This update helps AI agents understand, audit, and visually verify Webstudio projects more reliably. It also adds clearer guidance for expressions, Collections, screenshots, and responsive rendered audits. ### Audit complete projects - Run one `audit` command across accessibility, SEO, security, assets, styles, and performance. - Audit the whole project or focus on a specific page, scope, or severity. - Receive stable findings with locations, explanations, and recommended fixes. - Continue large result sets through pagination without repeating general guidance. ### Audit the rendered website - Build and inspect the generated website, rather than relying only on stored project data. - Check every static page at representative mobile, tablet, and desktop widths. - Use one suitable viewport inside each configured breakpoint range. - Reuse the same browser and resize pages between captures for substantially faster audits. - Process multiple pages concurrently while keeping resource use bounded. - Supply concrete examples for dynamic routes such as blog posts. - Review and confirm unusually large screenshot plans before they run. ### Find visual and performance problems Rendered audits can detect: - Horizontal page overflow. - Broken images. - Images loaded eagerly below the fold. - Images that are much larger than their rendered size. - Render-blocking resources. - Legacy font formats. - Pages that fail to load or redirect somewhere unexpected. - Pages captured before their layout becomes stable. Detailed screenshots, measurements, failures, and timing evidence are saved locally for further inspection. ### Capture generated pages directly - Capture a project route with `webstudio screenshot --path /route`. - Automatically generate and start a temporary production-style preview. - Stop the temporary preview after the screenshot is captured. - Return navigation, responsive layout, and page-size evidence with JSON output. - Allow external image domains needed by the generated preview. ### Reuse generated previews - Cache generated preview builds when the project has not changed. - Reuse preview dependencies across regenerations. - Keep preview dependencies isolated from the user’s project dependencies. - Surface preview build and startup failures with useful diagnostic output. - Include local project assets in generated previews. ### Author Collections correctly - Explain when Collection is required for arrays and objects. - Guide agents to bind the complete iterable instead of one item or the response wrapper. - Explain how Collection repeats its children for every entry. - Explain current-item and current-key scope. - Preserve Webstudio’s generated Collection parameters. - Guide agents to use stable values for repeated interactive components. ### Write expressions with fewer mistakes - Add a dedicated `webstudio://project/expressions` reference. - Document expression scope, supported syntax, and supported methods. - Explain how resources, variables, Collections, actions, and `system` values are accessed. - Distinguish fixed values from runtime expressions. - Provide valid examples for text, props, resource URLs, headers, query parameters, and bodies. - Warn agents that valid syntax does not guarantee that runtime data has the expected shape. ### Improve agent discovery - Make Collection and expression workflows discoverable through `meta.guide`. - Return Collection-specific instructions through `components.get`. - Point relevant command schemas to the expression reference. - Keep detailed guidance available without placing it in every tool response. ### Improve audit reliability - Report which rendered captures passed, found issues, failed, or were skipped. - Distinguish planning, preview, capture, and cleanup failures. - Include actionable retry instructions with rendered failures. - Preserve completed evidence when part of an audit fails. - Detect stale or conflicting preview servers more clearly. - Support cancellation and bounded capture timeouts. ### Improve packaged CLI verification - Add a release-shaped CLI smoke test. - Install the packed CLI in an isolated temporary project. - Fail on peer-dependency warning floods. - Verify preview, screenshots, MCP calls, rendered audits, and generated assets. - Confirm that generic React Router previews do not include Docker-only image dependencies. - Keep CLI and MCP stdout machine-readable while progress remains on stderr. ## User Impact Agents can now audit a real rendered Webstudio site, inspect responsive behavior, find concrete visual and performance problems, and provide evidence for their conclusions. They also receive much clearer instructions when turning API data into repeated content or writing dynamic expressions. |
||
|
Oleg IsonenandClaude Fable 5
|
de23e2a888 |
chore: migrate toolchain to oxc (oxfmt, oxlint, rolldown-vite) and typeScript 7 (#5868)
## prettier -> oxfmt (0.58.0)
- .oxfmtrc.json generated with `oxfmt --migrate=prettier`: trailingComma
es5, printWidth 80 (oxfmt defaults to 100), sortPackageJson disabled,
.prettierignore patterns folded in (file deleted).
- ignorePatterns additionally exclude yml/yaml/toml/jsonc and
packages/cli/templates/** so oxfmt covers exactly the file set prettier
used to format (ts,tsx,js,json,css,md) and never touches user-facing
scaffolding artifacts.
- root `format` script, nano-staged pre-commit hook and the CI check
(`pnpm oxfmt --check`, replacing `prettier --cache --check` and its
actions/cache step — oxfmt checks the whole repo in ~0.7s) now run
oxfmt.
- every generator script that ran `prettier --write` on __generated__
output now runs `oxfmt` (css-data, icons, html-data, postgrest, sdk,
sdk-components-*, apps/builder tailwind preflight).
- the four scripts that used prettier's JS `format()` API write raw
source instead and their package.json scripts pipe the generated file
through the oxfmt CLI: project-build generate-docs +
generate-runtime-operation-contracts, cli generate-docs, apps/builder
generate-server-only-api-metadata. oxfmt rejects paths containing "..",
so cross-package generated files are formatted via a subshell cd into
the owning package.
- one-time reformat: oxfmt's canonical output differs from prettier on 8
repo source files (binary-operator continuation indent, import() call
wrapping, sub-pixel spacing of empty for-clauses); oxfmt honors existing
`// prettier-ignore` comments so intentional layouts are kept.
- prettier remains ONLY as a cli devDependency for the
prettier/standalone HTML normalizer inside html-to-jsx.test.ts, and
inside fixtures/ + packages/cli/templates/ (ships to scaffolded user
projects; out of scope here).
- editor config switched from esbenp.prettier-vscode to oxc.oxc-vscode
(devcontainer extension list, .vscode editor.defaultFormatter).
## oxlint 1.56 -> 1.73
- oxlint 1.73 rules-of-hooks special-cases hooks NAMED useEffectEvent:
their results may only be called from effects. Our
app/shared/hook-utils/effect-event.ts is a stable-callback helper
(useEvent pattern), intentionally called from event handlers and passed
as props. The rule is disabled via .oxlintrc.json overrides for the five
files doing this (product code untouched); note those files lose
hook-order linting entirely until the rule becomes configurable.
## TypeScript 5.8.2 -> 7.0.2 (native compiler)
- `tsc` is now the Go-based native binary; all `tsgo` script usages
(workspace packages and fixtures) renamed back to `tsc` and
@typescript/native-preview removed (the package is being phased out).
- fixes for TS7 changed defaults and stricter checks:
- `types` now defaults to [] -> base tsconfig sets "types": ["node"]
(restores the implicit-@types/node behavior of TS5).
- `rootDir` now defaults to ./ -> every tsconfig.dts.json sets
"rootDir": "./src" (TS5011 otherwise).
- import assertions removed -> cli/src/cli.ts uses `with { type: "json"
}` (TS2880).
- stricter isolatedDeclarations inference -> explicit annotations in
css-data bin/property-filter.ts (TS9010) and image image-dev.stories.tsx
(TS9013).
- declaration serialization limit -> project-build builder-runtime.ts
runtimeOperationContracts gets an explicit type (TS7056); the annotation
preserves the narrow RuntimeOperationId union.
- TS7 ships no stable programmatic API:
- react-sdk (component-generator tests drive the compiler API) aliases
its typescript devDependency to @typescript/typescript6@^6.0.2; its bin
is `tsc6`, so the package's typecheck still runs native TS7 tsc from the
workspace root.
- generate-arg-types keeps typescript 5.8.2 for react-docgen-typescript.
- fixtures/ and packages/cli/templates/ keep typescript 5.8.2
(user-facing); fixture scripts still work because plain `tsc` resolves
to their local dependency.
## vite -> rolldown-vite 7.3.1 (oxc bundler)
- global pnpm override `"vite": "npm:rolldown-vite@^7.3.1"`: the builder
app (remix v2 vite plugin works unmodified), every package vite build
and the storybook build now bundle on rolldown/oxc.
- vitest is scoped BACK to vanilla vite via `vitest>vite`,
`vite-node>vite` and `@vitest/mocker>vite` overrides because vitest
3.1's runner does not understand rolldown's SSR transform
(__vite_ssr_exportName__ ReferenceError in every suite).
- those overrides pin vite EXACTLY to 6.3.4: letting ^6.3.4 float to
6.4.3 breaks cli prebuild tests (dynamic import of a generated
app/routes/$.tsx fails to resolve under 6.4.3's loader).
## unrelated cleanup requested during review
- apps/builder commands.ts imports duplicate-tokens-group statically
instead of via dynamic import. The circular dependency the dynamic
import guarded against only exists through the command-panel barrel
(command-panel.tsx > groups/index.ts > commands-group.tsx >
commands.ts); importing the group module directly is cycle-free
(verified with madge), matching the neighboring wrap-group and
convert-group imports.
## verification
- pnpm lint, oxfmt --check: clean.
- pnpm -r typecheck (TS7 native) and pnpm dts: clean.
- pnpm -r test: all suites pass (builder 2093 tests, cli 630, react-sdk
45, project-build 11, ...).
- builder app build, full workspace build, cli build and storybook build
all succeed on rolldown-vite.
## Description
1. What is this PR about (link the issue and add a short description)
## Steps for reproduction
1. click button
2. expect xyz
## Code Review
- [ ] hi @kof, I need you to do
- conceptual review (architecture, feature-correctness)
- detailed review (read every line)
- test it on preview
## Before requesting a review
- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")
## Before merging
- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oleg Isonen
|
d291e4b671 |
feat: Builder api runtime contracts guardrails for MCP (#5857)
This PR expands Webstudio MCP and CLI into a more complete agent authoring interface. Agents can discover project capabilities, edit real Webstudio content, use data/resources, verify visual output, and report useful issues when workflows fail. ## MCP User Capabilities - Create, update, duplicate, reorder, and delete pages. - Create pages from templates and manage reusable page templates. - Inspect project pages, instances, props, styles, assets, variables, resources, and metadata. - Insert Webstudio components and authored JSX fragments into pages. - Edit text, props, styles, breakpoints, design tokens, and CSS variables. - Update page metadata, SEO fields, redirects, marketplace metadata, and project settings. - Create and use static data variables. - Create and use dynamic resources fetched over HTTP. - Use REST, GraphQL, and built-in system resources in authored pages. - Bind data variables and resources to visible page content. - Create resource-backed pages such as sitemap XML, structured SEO data, and API-driven content sections. - Preview current project state without requiring a publish step. - Capture screenshots for visual verification. - Verify responsive designs across desktop, tablet, and mobile breakpoints. - Use screenshot diff and OCR-style checks to inspect rendered output. - Reset and refresh local MCP project state when needed. - Discover available CLI/MCP commands from generated manuals. - Use clearer examples for resources, data variables, JSX authoring, previewing, and visual checks. - Read marketplace product metadata without requiring a full project snapshot. - Get safer guidance when using low-level patch tools. - Get a ready-to-copy Discord report template when MCP/CLI behavior is confusing or broken. |
||
|
Oleg Isonen
|
b1a3eed1f8 | feat: guide agents through webstudio cli (#5847) | ||
|
Oleg Isonen
|
1ddf6b02c9 |
feat: API and CLI for everything (#5838)
Closes #5287 |