Commit Graph
167 Commits
Author SHA1 Message Date
Oleg Isonen efbc3098c4 feat: add queryable Assets resources (#5946)
Closes #5919
Ref #5920

## Outcome

Extend the existing Assets system resource with optional queries for
Markdown, JSON, and standard file metadata. Unconfigured Assets
resources retain the existing fetch-all behavior and activate none of
the content-query infrastructure.

Files are the authoring model. A standalone content engine converts an
immutable snapshot of only the files required by configured queries into
one bounded, deterministic runtime database. Builder preview, SSR, and
SSG use the same query semantics without requiring a database service
for a published site.

## Final architecture

```text
Markdown / JSON files
        │
        ▼
ContentSource adapter
(FS or Webstudio Assets)
        │ immutable snapshot
        ▼
ContentCompilationPlan
(union of configured query requirements)
        │
        ▼
ContentCompiler
(parse, normalize, discover fields, enforce budget)
        │
        ▼
ContentArtifact
(bounded, deterministic, immutable JSON)
        │
        ▼
ContentDatabase
(filter, OR groups, sort, paginate, project, read content)
        │
        ├── Builder preview
        ├── SSR runtime
        └── SSG build only
```

The authoring/storage layer owns mutable files. The content engine
consumes immutable snapshots and produces immutable, disposable
artifacts.


Query authoring is a separate OpenAPI-driven path:

```text
Assets REST OpenAPI operation
+ standard request JSON Schema
+ dynamic field paths
        │
        ▼
Generic OpenAPI adapter
        │ resolved schemas, controls, paths, and transport mapping
        ▼
Generic query-builder core and React renderer
        │ arbitrary provider-defined value
        ▼
Assets REST request adapter
        │
        ▼
{ query: value }
```

The selected standard OpenAPI operation is the only source of query UI
semantics. Request parameters and JSON Schema describe the editable
data; the generic renderer derives controls and owns layout.

### Activation and compilation

- The compiler does not index every asset. It combines all reachable
configured Assets-resource queries into one `ContentCompilationPlan`.
- Static constraints such as folder, path, and file type select
candidate files. Dynamic values such as a route-bound slug cannot reduce
the build-time candidate set.
- The plan contains the union of fields used by filtering, sorting,
routing, projection, excerpts, and requested full content.
- Documents shared by multiple resources are stored once in one
deployment-wide runtime database.
- Markdown/frontmatter, JSON properties, excerpts, and full bodies are
prepared only when required.
- No configured content query means no parsing, artifact, or shipped
runtime dependency. Fresh generated apps need no content-runtime
cleanup; incremental generation removes stale integration if a
previously configured query is deleted.

### Runtime database limit

- `CONTENT_DATABASE_MAX_BYTES` limits the complete serialized runtime
database, not source files, images, or unrelated assets.
- The default is `500 * 1024` bytes and is presented as 500 KiB.
- Counted data includes indexed metadata, frontmatter/JSON properties,
excerpts, the field catalog, serialization overhead, and full Markdown
bodies only when requested.
- Prepublish validation builds and serializes the exact database that
will be attached to the immutable build, so it knows the unbounded size,
bounded size, included documents, omitted documents, and affected
resources.
- Overflow never stores partial documents. Candidates are ordered by
`createdAt` descending, canonical path ascending, and stable document ID
ascending. Missing dates follow dated documents. A document that does
not fit is skipped while smaller later documents are still considered.
- Exceeding the limit warns but does not block publishing.
- Builder preview can report exact bounded versus unbounded results when
dynamic expressions have concrete preview values. Prepublish warnings
remain conservative for future route values.

### Snapshot consistency

The source contract guarantees a consistent compilation, while adapters
provide the mechanism:

- Webstudio Assets captures asset metadata and immutable content
references.
- Filesystem sources capture file identity and verify files did not
change before accepting the artifact.
- A changed source retries once, then fails clearly instead of producing
a mixed-revision database.
- This requires no universal snapshot service, source-tree copy, or new
versioned-asset infrastructure.

### Package ownership

- `@webstudio-is/content-engine`: runtime artifact schema, query
execution, field catalog, compact runtime statistics.
- `@webstudio-is/content-engine/compiler`: source snapshots, compilation
plans, Markdown/JSON parsing, deterministic compilation, byte
accounting, and compiler diagnostics. This entry exists to keep
Remark/YAML/future MDX dependencies out of runtime bundles.
- `@webstudio-is/query-builder`: resolves standard OpenAPI operation
schemas into an internal provider-neutral editor model, validation,
mutations, and source serialization. It owns no Assets,
provider-authored UI contract, or fixed REST-query semantics.
- `@webstudio-is/query-builder/react`: container-neutral schema renderer
whose layout is renderer policy. It remains separate from the content
engine.
- SDK: Webstudio Assets/Resource domain models and mapping Webstudio
expressions into generic content requirements.
- Protocol: external Assets REST/OpenAPI transport contracts.
- Asset uploader/repository: asset and folder mutations, generic
metadata, immutable byte references, and storage access. Its hosted
adapter supplies Webstudio Assets snapshots and parsed-metadata cache
persistence to the content engine; parsing, artifact creation, and query
semantics remain engine-owned.
- Builder and generated-app integrations decide when to compile, cache,
embed, load, or discard a database.

### Runtime lifecycle

- Builder caches immutable `ContentDatabase` instances by source
revision plus compilation-plan hash and coalesces concurrent
compilation. Matching operations reuse the same instance; field
discovery uses a dedicated minimal all-properties plan that excludes
excerpts and bodies.
- SSR attaches the validated bounded database to the build and includes
the small runtime only when required.
- SSG uses the database only while generating pages and omits it and the
runtime from final output unless a future runtime-query feature requires
them.
- Local development compiles from the filesystem and uses identical
query semantics.
- Published pages query their local artifact; they do not call Builder
REST endpoints.
- Published artifacts need no external index store, invalidation,
garbage collection, SaaS content proxy, new bucket, or new environment
variable.

## Implemented foundation

These existing parts remain useful and should be moved behind the final
ownership boundaries rather than reimplemented.

- [x] Upgrade the existing Assets system resource in place and preserve
unconfigured fetch-all behavior.
- [x] Implement typed filtering, nested boolean groups, deterministic
sorting, pagination, projections, dynamic values, and content hydration.
- [x] Add Markdown frontmatter, excerpt, JSON-property, and
field-catalog extraction.
- [x] Add reusable query-control implementations and a separately
imported React UI as a foundation for the final OpenAPI-driven contract.
- [x] Add Assets query authoring and dynamic field discovery as existing
behavior to migrate into the Assets OpenAPI operation.
- [x] Add synchronized visual and editable expression authoring.
- [x] Unify Assets REST operations under `/rest/assets/*`.
- [x] Define authenticated collection, item, content, upload, folder,
query, field-catalog, and OpenAPI operations.
- [x] Support Builder cookie authentication and public API token
authentication.
- [x] Add Builder preview, MCP/CLI operations, SSR execution, SSG path
generation, and conditional activation.
- [x] Add revision-keyed parsed metadata persistence for the hosted
Builder.
- [x] Add focused query, API, authorization, Builder, CLI, SSR, SSG,
migration, and fixture coverage.

## Remaining implementation plan

### 1. Establish the content engine

- [x] Rename the generic `@webstudio-is/asset-resource` implementation
to `@webstudio-is/content-engine`.
- [x] Move existing behavior instead of creating a parallel
implementation.
- [x] Keep the primary entry runtime-only and add only the compiler
entry needed for bundle separation.
- [x] Remove the old package after all consumers migrate.
- [x] Ensure the engine has no Builder, SDK, asset-uploader, PostgreSQL,
R2, or deployment-target dependency.

### 2. Define source snapshots and compilation plans

- [x] Introduce `ContentSource.openSnapshot()` and a revision-bearing
`ContentSourceSnapshot`.
- [x] Include canonical path, stable document ID, content type,
immutable content reference, size, and optional `createdAt`.
- [x] Implement Webstudio Assets and filesystem snapshot guarantees
using their adapter-specific mechanisms.
- [x] Introduce `ContentCompilationPlan` for the union of reachable
configured resource requirements.
- [x] Separate statically selectable candidates from dynamic runtime
predicates.
- [x] Add mutation-during-compilation tests for replacement, deletion,
folder movement, and filesystem changes.

### 3. Separate storage from content processing

- [x] Keep upload, replacement, folders, byte storage, revisions, and
generic metadata in the Assets repository/uploader.
- [x] Move parsing, excerpts, field discovery, query execution, and
artifact creation into the content engine.
- [x] Remove content-index coordination from asset mutation business
logic.
- [x] Preserve original source `createdAt` across content replacement
and expose it as optional queryable metadata.
- [x] Keep hosted parsed-document persistence behind the content
compiler's optional cache interface.

### 4. Build deterministic compilation

- [x] Move the existing Markdown, frontmatter, JSON, canonicalization,
hashing, document-building, and tests into the compiler.
- [x] Keep frontmatter behind a dedicated module and continue using the
existing Remark/Micromark pipeline.
- [x] Parse only candidate files and only the tiers required by the
compilation plan.
- [x] Deduplicate documents and fields required by multiple resources.
- [x] Define a versioned, deterministic `ContentArtifact` with integrity
information.
- [x] Keep verbose parse and omission diagnostics outside the runtime
artifact.
- [x] Reject incompatible artifact versions cleanly.

### 5. Enforce the deployment-wide database budget

- [x] Replace the old asset-index limit with
`CONTENT_DATABASE_MAX_BYTES`, defaulting to 500 KiB.
- [x] Have applications read configuration and pass the byte limit into
the compiler.
- [x] Measure the exact serialized database, including selected bodies
and overhead.
- [x] Implement deterministic skip-and-continue packing of complete
documents.
- [x] Return compiler diagnostics with unbounded/bounded bytes,
included/omitted counts, omitted identities, largest contributors, and
affected resources.
- [x] Expose only compact artifact statistics at runtime through
`ContentDatabase.getStats()`.
- [x] Add exact boundary, deterministic ordering, deduplication, and
oversized-document tests.

### 6. Finalize the runtime database

- [x] Define `ContentDatabase` over an immutable artifact.
- [x] Preserve filtering, nested AND/OR groups, sorting, offset/limit,
projection, field references, and content retrieval.
- [x] Expose `getFieldCatalog()` and compact `getStats()`.
- [x] Keep Markdown/YAML/compiler dependencies out of the runtime entry.
- [x] Add execution guards and maximum-size runtime benchmarks.

### 7. Make query authoring OpenAPI-driven

The selected standard OpenAPI operation is the complete query-authoring
contract. The query builder resolves parameters and request-body JSON
Schema, derives its internal editor model, and applies renderer-owned
layout policy. It must not embed or fetch a second capabilities/UI
language through `x-webstudio-query` or another side channel.

- [x] Remove `x-webstudio-query` and all vendor query-definition
metadata from the Assets OpenAPI operation.
- [x] Resolve local OpenAPI references and derive the editable request
value, defaults, validation, labels, choices, arrays, objects, and
unions solely from standard operation parameters and request-body JSON
Schema.
- [x] Represent dynamic Assets field paths with standard JSON Schema
vocabulary in the `queryAssets` request schema.
- [x] Keep the normalized editor representation internal to
query-builder; do not expose another provider-authored capabilities
contract.
- [x] Remove provider-controlled layout metadata such as `group`; make
compact scalar packing and all other layout renderer policy.
- [x] Keep Webstudio expression binding as a consumer/editor concern
layered over schema values, not part of the OpenAPI contract.
- [x] Preserve the existing Assets filter, sort, output selection,
full-content/range selection, limit/offset, editable source, and
configured-field behavior.
- [x] Keep the Assets resource adapter responsible for serializing the
edited query into its request body.
- [x] Remove the obsolete Assets query-capabilities endpoint, URL, and
UI-side capability parsing.
- [x] Add a non-Assets OpenAPI proof with differently named parameters
and no vendor metadata.
- [x] Verify the generic packages contain no Assets imports, fixed
Assets property contract, or provider-authored layout model.
- [x] Update focused core, React, SDK, protocol/OpenAPI, Builder, and
source round-trip tests.

### 8. Manage ContentDatabase lifecycle in Builder

- [x] Cache databases by source revision plus normalized
compilation-plan hash.
- [x] Coalesce concurrent compilation requests.
- [x] Reuse cached database instances across field discovery,
autocomplete, query preview, and response preview when their normalized
compilation plans match.
- [x] Replace it when source revision or query requirements change.
- [x] Preserve the optional revision-keyed hosted parsed-document cache
without making it a source of truth.
- [x] Show exact preview warnings when concrete values reveal missing
bounded results.

### 9. Materialize and verify immutable builds

- [x] Compile and serialize the exact deployment-wide database during
prepublish validation.
- [x] Attach that validated bounded database to the immutable build
without rebuilding it from later asset state.
- [x] Show non-blocking prepublish warnings with database bytes,
omissions, and potentially affected resources.
- [x] Avoid claiming exact future route matches when dynamic values are
unknown.
- [x] Include the database/runtime only when a reachable configured
query requires them.
- [x] Ensure detail resources requesting Markdown content include all
statically eligible bodies needed by dynamic routes.

### 10. Integrate all targets

- [x] SSR: load the build's bounded database and conditionally include
the small runtime.
- [x] SSG: query the same database during generation, then omit
database/runtime output when no runtime query is needed.
- [x] Local/CLI: compile from the filesystem source with the same
semantics.
- [x] Preserve overview pagination, alternative route fields,
draft-as-user-data behavior, and full article rendering.
- [x] Verify ordinary Assets and fresh query-free sites perform no
parser, artifact, content-runtime dependency install, or cleanup work.

### 11. Correct API and package ownership

- [x] Move external Assets REST/OpenAPI contracts to
`@webstudio-is/protocol`.
- [x] Keep Webstudio domain models and Assets system-resource
configuration in the SDK.
- [x] Keep both cookie and token authentication behavior on the unified
Assets REST surface.
- [x] Remove pass-through route wrappers/reexports that add no transport
behavior.
- [x] Keep published runtime queries local to the artifact rather than
routing them through Builder REST.

### 12. Remove superseded deployment infrastructure

- [x] Remove external deployment-index upload, fetch, invalidation,
cleanup, and garbage-collection code.
- [x] Remove the SaaS asset-origin adapter requirement; close
webstudio-is/webstudio-saas#652 as superseded because it has no
independent purpose.
- [x] Remove index-bucket compatibility configuration and avoid adding
Infisical variables.
- [x] Remove obsolete external-index paths, package dependencies,
routes, scripts, and tests.
- [x] Consolidate unreleased intermediate migrations where safe and
retain only schema required by the final hosted cache/API design.

### 13. Verification and release readiness

- [x] Move and preserve existing tests while changing ownership; add new
tests only for changed contracts and behavior.
- [x] Run content-engine compiler/runtime tests, query-builder tests,
SDK/protocol/API tests, Builder tests, CLI tests, and affected
typechecks.
- [x] Run formatting, generated-file idempotence, fixtures, clean
Supabase-compatible migrations, E2E shards, and complete agent
evaluations.
- [x] Verify SSR and SSG conditional inclusion and query-free generated
package manifests.
- [x] Run the five-article MCP blog workflow: authoring, field
discovery, overview, dynamic detail routes, full Markdown rendering,
publishing, and published-site verification.
- [x] Run a maximum-budget database test and a
mutation-during-compilation test.
- [x] Perform a final architecture, security, API compatibility,
migration, bundle, correctness, and DRY review.
- [x] Reconcile every checkbox and replace the verification section with
results from the final architecture before release.

## Future MDX work

MDX is intentionally separate from this release.

- The compiler contract should allow a future build-only MDX parser.
- MDX compilation must not enter the published runtime.
- Embedded JSX must use an explicit compiled representation rather than
runtime source evaluation.
- Webstudio components, tokens, styles, and component allowlists remain
Webstudio integration concerns.
- The future MDX work should reuse `ContentArtifact` and
`ContentDatabase` where practical.

## Superseded design

The following earlier design choices are no longer the target
architecture:

- GROQ or GraphQL as the Assets query language.
- One external or long-lived index object per resource/query.
- A deployment index fetched by the Worker from SaaS/CDN infrastructure.
- Lazy production reads of source Markdown bodies.
- A separate index bucket, asset origin, invalidation service, or
garbage collector.
- A 4 MiB published index limit.
- Rebuilding an old build's database from the current mutable asset set.
- Bundling query infrastructure into projects without a configured
content query.

## Verification baseline

The embedded runtime-database architecture previously passed the checks
below. The OpenAPI-driven query-authoring change additionally passed:

- Query-builder core: 4 files / 19 tests.
- Query-builder React: 1 file / 7 tests.
- SDK Assets source/config: 1 file / 7 tests.
- Protocol Assets OpenAPI: 1 file / 9 tests, including independent
OpenAPI validation.
- Builder Assets OpenAPI description: 1 file / 5 tests.
- Query-builder, React, SDK, protocol, and Builder typechecks.
- Package-boundary validation and formatting of every affected file.

Local verification baseline:

- Content engine: 15 files / 104 tests.
- Asset uploader: 24 files / 175 tests.
- Builder: 204 files / 2,423 tests.
- CLI: 54 files / 866 tests.
- Complete workspace typecheck, formatting, lint, package boundaries,
generated fixtures, and fixture builds.
- The complete migration history applied successfully to a fresh
Supabase-compatible database, including every migration after the
consolidated Assets migration.
- Builder production build and full local E2E suite passed.
- Complete agent evaluations passed: authenticated page (170,859 ms),
design input (207,101 ms), and font assets (67,050 ms).
- Final review additionally verified correct REST edit/view/build
permits, cookie and public-token read paths, immutable snapshot retries,
exact database budgeting, query-free dependency omission, and SSR/SSG
conditional runtime inclusion.

Final staging verification passed against the embedded runtime-database
architecture: MCP uploaded and updated five Markdown articles,
discovered dynamic frontmatter fields, previewed a five-item sorted
overview and one full-body slug query, updated both Assets resources,
found zero binding issues, rendered `/blog` and `/blog/dynamic-routes`
at HTTP 200, published build version 266, and verified both published
routes at HTTP 200 with the expected article metadata and Markdown body.
The workflow also found and fixed offset-form asset timestamps failing
canonical datetime validation and made MCP `preview.start({ port: 0 })`
correctly select an available local port.

## Release constraints

- No new SaaS content service, bucket, or environment variable.
- No published-site database dependency.
- No parser/compiler dependency in query-free sites or runtime-only
bundles.
- The validated content database is immutable build output.
- Publishing warnings do not block deployment.
- The selected standard OpenAPI operation and its request JSON Schema
are the only source of query UI semantics.
2026-07-31 00:17:46 +02:00
Oleg IsonenandClaude Fable 5 de23e2a888 chore: migrate toolchain to oxc (oxfmt, oxlint, rolldown-vite) and typeScript 7 (#5868)
## prettier -> oxfmt (0.58.0)

- .oxfmtrc.json generated with `oxfmt --migrate=prettier`: trailingComma
es5, printWidth 80 (oxfmt defaults to 100), sortPackageJson disabled,
.prettierignore patterns folded in (file deleted).
- ignorePatterns additionally exclude yml/yaml/toml/jsonc and
packages/cli/templates/** so oxfmt covers exactly the file set prettier
used to format (ts,tsx,js,json,css,md) and never touches user-facing
scaffolding artifacts.
- root `format` script, nano-staged pre-commit hook and the CI check
(`pnpm oxfmt --check`, replacing `prettier --cache --check` and its
actions/cache step — oxfmt checks the whole repo in ~0.7s) now run
oxfmt.
- every generator script that ran `prettier --write` on __generated__
output now runs `oxfmt` (css-data, icons, html-data, postgrest, sdk,
sdk-components-*, apps/builder tailwind preflight).
- the four scripts that used prettier's JS `format()` API write raw
source instead and their package.json scripts pipe the generated file
through the oxfmt CLI: project-build generate-docs +
generate-runtime-operation-contracts, cli generate-docs, apps/builder
generate-server-only-api-metadata. oxfmt rejects paths containing "..",
so cross-package generated files are formatted via a subshell cd into
the owning package.
- one-time reformat: oxfmt's canonical output differs from prettier on 8
repo source files (binary-operator continuation indent, import() call
wrapping, sub-pixel spacing of empty for-clauses); oxfmt honors existing
`// prettier-ignore` comments so intentional layouts are kept.
- prettier remains ONLY as a cli devDependency for the
prettier/standalone HTML normalizer inside html-to-jsx.test.ts, and
inside fixtures/ + packages/cli/templates/ (ships to scaffolded user
projects; out of scope here).
- editor config switched from esbenp.prettier-vscode to oxc.oxc-vscode
(devcontainer extension list, .vscode editor.defaultFormatter).

## oxlint 1.56 -> 1.73

- oxlint 1.73 rules-of-hooks special-cases hooks NAMED useEffectEvent:
their results may only be called from effects. Our
app/shared/hook-utils/effect-event.ts is a stable-callback helper
(useEvent pattern), intentionally called from event handlers and passed
as props. The rule is disabled via .oxlintrc.json overrides for the five
files doing this (product code untouched); note those files lose
hook-order linting entirely until the rule becomes configurable.

## TypeScript 5.8.2 -> 7.0.2 (native compiler)

- `tsc` is now the Go-based native binary; all `tsgo` script usages
(workspace packages and fixtures) renamed back to `tsc` and
@typescript/native-preview removed (the package is being phased out).
- fixes for TS7 changed defaults and stricter checks:
- `types` now defaults to [] -> base tsconfig sets "types": ["node"]
(restores the implicit-@types/node behavior of TS5).
- `rootDir` now defaults to ./ -> every tsconfig.dts.json sets
"rootDir": "./src" (TS5011 otherwise).
- import assertions removed -> cli/src/cli.ts uses `with { type: "json"
}` (TS2880).
- stricter isolatedDeclarations inference -> explicit annotations in
css-data bin/property-filter.ts (TS9010) and image image-dev.stories.tsx
(TS9013).
- declaration serialization limit -> project-build builder-runtime.ts
runtimeOperationContracts gets an explicit type (TS7056); the annotation
preserves the narrow RuntimeOperationId union.
- TS7 ships no stable programmatic API:
- react-sdk (component-generator tests drive the compiler API) aliases
its typescript devDependency to @typescript/typescript6@^6.0.2; its bin
is `tsc6`, so the package's typecheck still runs native TS7 tsc from the
workspace root.
- generate-arg-types keeps typescript 5.8.2 for react-docgen-typescript.
- fixtures/ and packages/cli/templates/ keep typescript 5.8.2
(user-facing); fixture scripts still work because plain `tsc` resolves
to their local dependency.

## vite -> rolldown-vite 7.3.1 (oxc bundler)

- global pnpm override `"vite": "npm:rolldown-vite@^7.3.1"`: the builder
app (remix v2 vite plugin works unmodified), every package vite build
and the storybook build now bundle on rolldown/oxc.
- vitest is scoped BACK to vanilla vite via `vitest>vite`,
`vite-node>vite` and `@vitest/mocker>vite` overrides because vitest
3.1's runner does not understand rolldown's SSR transform
(__vite_ssr_exportName__ ReferenceError in every suite).
- those overrides pin vite EXACTLY to 6.3.4: letting ^6.3.4 float to
6.4.3 breaks cli prebuild tests (dynamic import of a generated
app/routes/$.tsx fails to resolve under 6.4.3's loader).

## unrelated cleanup requested during review

- apps/builder commands.ts imports duplicate-tokens-group statically
instead of via dynamic import. The circular dependency the dynamic
import guarded against only exists through the command-panel barrel
(command-panel.tsx > groups/index.ts > commands-group.tsx >
commands.ts); importing the group module directly is cycle-free
(verified with madge), matching the neighboring wrap-group and
convert-group imports.

## verification

- pnpm lint, oxfmt --check: clean.
- pnpm -r typecheck (TS7 native) and pnpm dts: clean.
- pnpm -r test: all suites pass (builder 2093 tests, cli 630, react-sdk
45, project-build 11, ...).
- builder app build, full workspace build, cli build and storybook build
all succeed on rolldown-vite.

## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 01:03:36 +02:00
Oleg Isonen d291e4b671 feat: Builder api runtime contracts guardrails for MCP (#5857)
This PR expands Webstudio MCP and CLI into a more complete agent
authoring interface. Agents can discover project capabilities, edit real
Webstudio content, use data/resources, verify visual output, and report
useful issues when workflows fail.

## MCP User Capabilities

- Create, update, duplicate, reorder, and delete pages.
- Create pages from templates and manage reusable page templates.
- Inspect project pages, instances, props, styles, assets, variables,
resources, and metadata.
- Insert Webstudio components and authored JSX fragments into pages.
- Edit text, props, styles, breakpoints, design tokens, and CSS
variables.
- Update page metadata, SEO fields, redirects, marketplace metadata, and
project settings.
- Create and use static data variables.
- Create and use dynamic resources fetched over HTTP.
- Use REST, GraphQL, and built-in system resources in authored pages.
- Bind data variables and resources to visible page content.
- Create resource-backed pages such as sitemap XML, structured SEO data,
and API-driven content sections.
- Preview current project state without requiring a publish step.
- Capture screenshots for visual verification.
- Verify responsive designs across desktop, tablet, and mobile
breakpoints.
- Use screenshot diff and OCR-style checks to inspect rendered output.
- Reset and refresh local MCP project state when needed.
- Discover available CLI/MCP commands from generated manuals.
- Use clearer examples for resources, data variables, JSX authoring,
previewing, and visual checks.
- Read marketplace product metadata without requiring a full project
snapshot.
- Get safer guidance when using low-level patch tools.
- Get a ready-to-copy Discord report template when MCP/CLI behavior is
confusing or broken.
2026-07-10 02:13:22 +02:00
Oleg Isonen c3d2fa7fa3 feat: Render current page state on links in preview mode (#5810)
Closes https://github.com/webstudio-is/webstudio/issues/5207
2026-06-11 21:11:11 +02:00
Oleg Isonen 2b06b2a987 fix: Fix current link state in static export (#5809)
Closes https://github.com/webstudio-is/webstudio/issues/3909

Current page state for links doesn't work with static export
2026-06-11 00:24:46 +02:00
Oleg Isonen 03f398d9b2 fix: Fix local link matching when query params are used (#5802)
Closes https://github.com/webstudio-is/webstudio/issues/5377


## Summary

Fixes local link matching so links are marked current only when the full
URL matches: pathname, search query, and hash.

## Changes

- Replaced duplicated Remix/React Router `Link` logic with a shared
`createLocalLink` helper.
- Switched from `NavLink` route-only active matching to explicit full
URL matching.
- Preserved exact `end`-style behavior while also requiring query/hash
equality.
- Kept hash-only links as native anchors so current search params are
not rewritten.
- Preserved browser-like behavior for `href=""` by resolving it to the
full current URL.
- Kept asset/external/protocol links as plain anchors.
- Added comments explaining why the shared local-link code exists and
why specific edge cases avoid router navigation.

## Tests

- Added URL matching unit tests for pathname/search/hash, case
sensitivity, trailing slash, empty href, hash href, and router-resolved
paths.
- Added React Router integration coverage for:
  - exact active/current matching
  - query/hash mismatches
  - active class and `aria-current`
  - hash-only links
  - empty links preserving search/hash
  - external/protocol/asset links
  - relative route links
  - `preventDefault`
  - pending navigation/render behavior
  - same-url resource updates
  - HTML embed navigation behavior

## Verification

- `pnpm --filter @webstudio-is/sdk-components-react test`
- `pnpm --filter @webstudio-is/sdk-components-react-router test`
- `pnpm --filter @webstudio-is/sdk-components-react typecheck`
- `pnpm --filter @webstudio-is/sdk-components-react-router typecheck`
- `pnpm --filter @webstudio-is/sdk-components-react-remix typecheck`
- `pnpm --filter @webstudio-is/sdk-components-react dts`
- `pnpm --filter @webstudio-is/sdk-components-react-router dts`
- `pnpm --filter @webstudio-is/sdk-components-react-remix dts`
2026-06-10 00:06:33 +02:00
Oleg Isonen 91d2136921 refactor(stories): consolidate exports, add StorySection/StoryGrid everywhere, remove argTypes/args 2026-03-15 19:21:33 +00:00
Oleg Isonen fdee339de0 chore: Migrate to tsgo (#5578) 2026-01-20 17:25:12 +01:00
Bogdan Chadkin 0eea205bfc refactor: merge base and animation props with component metas (#5214)
Related https://github.com/webstudio-is/webstudio/pull/5211
2025-05-16 23:31:06 +03:00
Bogdan Chadkin 452c50a41a refactor: upgrade vite for vitest and storybook (#5165)
Tough cookie. There is a problem with resolving react-remove-scroll,
figured it by reducing allowed amount of server conditions.
2025-04-29 14:17:42 +04:00
Ivan Starkov e5369ef930 experimental: Youtube animations (#5072)
## Description

### Not finished, should be merged as has small issues fixed

https://video-viking.wstd.work/
https://video-viking.wstd.work/entry

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2025-04-06 14:47:16 +03:00
Bogdan Chadkin 945bf04331 experimental: add field label for animation section (#5068)
Added generic field label to give animation labels similar experience to
property labels.
<img width="268" alt="Screenshot 2025-03-28 at 15 30 34"
src="https://github.com/user-attachments/assets/a9d15f03-8e8b-4f9a-8d77-e4b7847de87e"
/>
2025-03-28 17:17:01 +07:00
Bogdan Chadkin 935983b543 build: upgrade checks (#4982)
- vitest
- testing-library (got much lighter)
- eslint and plugins (@types/eslint__js no longer necessary)
- storybook
- esbuild
2025-03-11 15:22:57 +07:00
Ivan StarkovandOleg Isonen 3c42f8ca4b experimental: View Animations support (#4817)
## Description

ref  #2645

- [x] - Add view-timeline-name and scroll-timeline-name support, and
prefixer (use custom property for unsupported browsers)
- [x] - Add initial story

## Steps for reproduction


https://6382151c8b47d4399fb9fc69-uljkhwtbmc.chromatic.com/iframe.html?args=&globals=&id=sdk-components-animation-view-animations--in-out&viewMode=story

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file

---------

Co-authored-by: Oleg Isonen <oleg008@gmail.com>
2025-02-04 06:02:55 +03:00
Bogdan Chadkin b72a74c825 refactor: bump ui libraries (#4802)
All maintenance or unrelated bulk bumps.

-
https://www.radix-ui.com/primitives/docs/overview/releases#january-22-2025
-
https://github.com/floating-ui/floating-ui/releases/tag/%40floating-ui%2Futils%400.2.9
- https://react-spectrum.adobe.com/releases/2025-01-15.html
2025-01-30 19:58:10 +07:00
Bogdan Chadkin 5acb458a68 build: upgrade dev dependencies (#4764)
Nothing critical, all not related to production

storybook 8.5
eslint 9.18
vitest 3
typescript 5.7.3
@types/node 22.10.7
type-fest 4.32
2025-01-21 00:14:56 +07:00
Bogdan Chadkin e27628cfd9 refactor: generate stories for base components (#4734)
Added support for using core components in content and markdown embeds.
2025-01-10 22:10:21 +07:00
Bogdan Chadkin 71f56ea7d6 refactor: bundle webstudio packages in cli (#4706)
Having all webstudio packages as dependency in CLI leads to very long
and bloated install.

Before
<img width="762" alt="Screenshot 2025-01-03 at 19 49 31"
src="https://github.com/user-attachments/assets/96624dae-c72a-4407-83ea-e56ed5d3f7d8"
/>

After
<img width="385" alt="Screenshot 2025-01-03 at 20 00 17"
src="https://github.com/user-attachments/assets/f3271c6e-07ca-47f6-9dfb-491ca68e7caf"
/>

after/before
<img width="633" alt="Screenshot 2025-01-03 at 19 58 15"
src="https://github.com/user-attachments/assets/8273072a-c71e-485c-aa39-116c9698bc8f"
/>
2025-01-04 01:02:13 +07:00
Bogdan Chadkin 2fe69ce31e refactor: upgrade ui libraries (#4693)
All has minor changes

- floating-ui
- react-error-boundary
(https://github.com/bvaughn/react-error-boundary/releases/tag/5.0.0)
- urlpattern polyfill (https://github.com/whatwg/urlpattern/issues/179)
- use-debounce
(https://github.com/xnimorz/use-debounce/releases/tag/10.0.0)
- react-aria
- react-hot-toast
(https://github.com/timolins/react-hot-toast/releases/tag/v2.5.0)
2025-01-03 02:30:58 +07:00
Bogdan Chadkin aa52e9fa35 feat: support tables in content and markdown embed (#4679)
Markdown embed now can convert tables like this into html

```
| Month    | Savings |
| -------- | ------- |
| January  | $250    |
| February | $80     |
| March    | $420    |
```

Both markdown embed and content embed have descendants to style tables.
Added table, row, header and cell.
2025-01-01 01:49:05 +07:00
Bogdan Chadkin 5ee696cbbf fix: avoid cli installing private template package (#4657)
Fixes https://github.com/webstudio-is/webstudio/issues/4654

template package is private for now so better move it into dev
dependencies.
2024-12-26 01:05:18 +07:00
Bogdan Chadkin a32d6329ed refactor: add templates meta support (#4653)
Here added one more description file for components `*.template.tsx`.

It is decoupled from component meta to fix the issue with some templates
like Sheet being treated as components though we don't have actual
components with this name.

Another reason is have it separately to not include builder specific
code and dependencies into CLI and published websites. Template package
is defined as optional in peer dependencies meta.

In this PR migrated only Button and List as examples. There is still not
support for data sources and styles.

Added support for text placeholders in children
2024-12-25 21:37:15 +07:00
Ivan Starkov b867e67a91 feat: Experimental INP fix (#4560)
## Description


https://vercel.com/blog/demystifying-inp-new-tools-and-actionable-insights

## new 
https://inp-inp-inp-inp.wstd.work/

## prev
https://no-inp-no-no.wstd.work/

## 20x slowdown new (click)
<img width="323" alt="image"
src="https://github.com/user-attachments/assets/08b063f6-93a3-4e2b-8e5d-e9521f97a40d"
/>
## 20x slowdown new (keyboard)
<img width="305" alt="image"
src="https://github.com/user-attachments/assets/3a1bb563-392f-40d9-9067-7cb085adcedd"
/>


## 20x slowdown old (click)
<img width="308" alt="image"
src="https://github.com/user-attachments/assets/41ff36cd-bd27-4e5d-b569-9040ce12b845"
/>
## 20x slowdown old (keyboard)
<img width="304" alt="image"
src="https://github.com/user-attachments/assets/4cc41ef9-f6d1-4e2a-ad41-de601d713bc9"
/>



## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2024-12-12 16:43:24 +03:00
Bogdan Chadkin c8c1268002 build: drop legacy publish action (#4525)
One less action on every commit in master.
Made version placeholder more abstract "0.0.0-webstudio-version"
2024-12-06 19:51:08 +07:00
Bogdan Chadkin 1d9157bb34 build: publish CLI 0.191.0 to npm (#4522) 2024-12-06 17:27:53 +07:00
Bogdan Chadkin 82d1bd255b build: bump dev deps (#4503)
- prettier
- vike
- vitest
- storybook
- eslint
2024-12-04 16:48:00 +07:00
Bogdan Chadkin 260c903045 build: bump typescript (#4471)
Upgraded typescript to 5.7, removed it from packages to reuse from root
instead and fixed a couple of changes.
2024-11-26 11:59:07 +07:00
Bogdan Chadkin 6123a8ca74 build: upgrade devtools (#4455)
- storybook
- vitest
- eslint
- typefest
- types/node
- vike
- tsx
- pnpm
2024-11-21 21:06:22 +07:00
Bogdan Chadkin c4eea571c9 refactor: preserve components structure for better tree shaking (#4418)
Here replaced esbuild with vite for building sdk-components packages.
Vite can be configured to output preserved structure instead of bundling
everything into single module which hurts treeshaking a lot.

sdk-components-react
./fixtures/ssg/dist/client: 284kB (-24kB)
./fixtures/webstudio-remix-netlify-functions/build/client: 368kB (-28kB)
./fixtures/webstudio-remix-vercel/build/client: 1004kB (-8kB)


sdk-components-react-radix
./fixtures/ssg/dist/client: 284kB (-4kB)
./fixtures/webstudio-remix-netlify-functions/build/client: 372kB (0kB)
./fixtures/webstudio-remix-vercel/build/client: 900kB (-108kB)
2024-11-15 16:43:12 +07:00
Bogdan Chadkin cb255932bf refactor: migrate CLI tests to vitest (#4409)
Jest is barely maintained and vitest is blazinly faster.
2024-11-13 01:38:19 +04:00
Bogdan Chadkin 50ca393e73 build: publish CLI 0.189.0 (#4395) 2024-11-09 18:43:55 +04:00
Bogdan Chadkin d61945402c build: upgrade to typescript 5.6 (#4345)
Only some apis added to types.
2024-10-28 12:17:43 +03:00
Bogdan Chadkin ac74d634ac refactor: bump ui libraries (#4344)
Here only patches with bug fixes. Shouldn't affect us. Bumping right
after release so we have time to tinker with UI.
2024-10-25 13:21:37 +03:00
Bogdan Chadkin d29065b7fe build: publish 0.185.0 (#4286) 2024-10-14 22:13:18 +03:00
Ivan Starkov 334caf4ec6 build: CLI release 0.182.0 (#4172)
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2024-09-27 20:52:28 +03:00
Bogdan Chadkin 671fdd50b2 refactor: reimplement navigator tree with new component (#4165)
Ref https://github.com/webstudio-is/webstudio/issues/3399

What to test

- hover
- select
- hiding (with show button) item should be transparent
- slot should be purple
- hide slot fragment
- render collections
- enter should trigger edit instance text
- support editing instance label with cmd+e
- should prevent dragging rich text child
- should prevent dragging non detachable instances (accordion header)
- should support reparenting after the last child
2024-09-27 19:33:00 +02:00
Bogdan Chadkin 632a9ee2e3 build: publish 0.181.0 (#4147)
Here fixed node version in publish dialog and published cli packages.
2024-09-23 13:15:42 +02:00
Ivan Starkov 1e615a18fc build: CLI release (#4079)
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2024-09-07 23:39:43 +03:00
a0407f3e1f fix: Default form control values (#4045)
## Description

- always use defaultValue and defaultChecked props when passing to react
- expose value and checked as props, remove default*
- reorder initial props to make more sense (name, value, checked - in
this order)

Affected components:

Forms:
- checkbox
- radio
- input
- select
- textarea

Radix:
- checkbox
- radio group
- select
- switch

## Steps for reproduction

1. test each component named above how bound and unbound value/checked
works
2. use in regular form and bind to url params
3. use in webhook form 

## Code Review

- [ ] hi @john, I need you to do
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file

---------

Co-authored-by: Ivan Starkov <istarkov@gmail.com>
Co-authored-by: Bogdan Chadkin <opensource@trysound.io>
2024-09-04 00:15:43 +02:00
Bogdan Chadkin ae1e72051c build: publish 0.175.0 (#4054) 2024-09-02 22:55:05 +02:00
Bogdan Chadkin 431e58dd34 refactor: rewrite normalize styles with css (#3951)
Our parser is good enough to represent any styles with css. Here
converted normalize styles into css and moved into sdk package.

Also fixed (remix) form component name
2024-08-17 15:00:31 +02:00
Bogdan Chadkin a904d2cdc7 feat: add markdown embed component (#3884)
Ref https://github.com/webstudio-is/webstudio/issues/3485

Added component with descendants support.

<img width="861" alt="Screenshot 2024-08-06 at 15 53 33"
src="https://github.com/user-attachments/assets/4c488573-f52c-4661-b626-cedc113ee10c">
2024-08-07 00:51:41 +02:00
Bogdan Chadkin 60159b894f build: publish 0.174.0 (#3829) 2024-07-30 16:01:35 +02:00
Bogdan Chadkin dca04597fb build: publish 0.173.0 (#3816) 2024-07-29 17:48:40 +02:00
Bogdan Chadkin 6505b5ad7c build: drop jest-jsdom (#3678)
Looks like we don't really need jest-jsdom for our tests all checks here
are just null checks.
2024-07-09 13:00:14 +02:00
Bogdan Chadkin a58f2eff04 build: run all tests and typechecks in CI (#3661)
Here improved our checks. No longet rely on "checks" script but run
"test" and "typescript" scripts from each package in monorepo.

Would also make each package less boilerplaty.
2024-07-02 18:21:05 +03:00
Ivan Starkov 4d0574d573 build: Update typescript (#3637)
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2024-06-28 11:46:13 +03:00
Ivan Starkov 38d49f9f69 feat: Add native select control (#3616)
## Description

rename of #3609

## New

- Native Select

## Changes

- Improved detection algorithm to include invisible elements (like
`option` or `display:none`).
- Added interceptor to handle pointer down events, preventing select
from opening.
- Enhanced element selection on canvas to support `display:none` and
option selection.
- Fixed prop types generation (previously, the value property was
broken).
- Memory props has effect on canvas only
- Recreate component on Canvas in case of defaultValue has changed


## Steps for reproduction

Add `Select` see it works

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
2024-06-26 07:07:37 +03:00
Bogdan Chadkin 936dcd3ca8 build: publish 0.168.0 (#3556) 2024-06-13 14:57:21 +03:00
Bogdan Chadkin c5a9a38838 build: publish 0.167.0 (#3498) 2024-06-05 11:34:59 +03:00