admin/webstudio
1
0 Fork 0
mirror of https://github.com/webstudio-is/webstudio.git synced 2026-10-30 08:37:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
6,651 Commits 122 Branches 310 Tags
0.294.0
Commit Graph
2497 Commits
Author SHA1 Message Date
Oleg Isonen c49c0786be fix: unify dashboard background surface (#6287) 
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
 2026-08-31 21:55:59 +02:00
Oleg Isonen f953d1be37 fix: Restore the login background gradient (#6286) 
## Summary

Restores the original Webstudio login gradient after the color-system
migration added a dark scrim over it. Fixed Webstudio identity artwork
now belongs to the Design System, while generic surfaces, content,
translucency, and overlays use the semantic color system instead of a
Builder-local brand color bag.

## Architecture

- The Design System exports only the fixed Webstudio background
gradient, progress gradient, and progress shadow as brand artwork.
- Generic black, white, translucent panel, and scrim entries were
removed rather than moved.
- Login and error surfaces use semantic light/dark pairings.
- Project-card tags derive their local 30% scrim from the semantic
overlay color.

## Todo

- [x] Restore the original login gradient without a scheme-dependent
scrim.
- [x] Move fixed identity artwork into the Design System package.
- [x] Remove the Builder-local brand color registry.
- [x] Replace generic fixed colors with semantic colors or local
derivations.
- [x] Add explicit dark-mode Storybook coverage for the login screen.
- [x] Run Design System and Builder typechecks.
- [x] Run Design System tests, scoped lint, formatting, and
package-boundary validation.
- [x] Review documentation impact and document ownership in the Design
System color README.

## Verification

- Design System tests: 222 passed.
- Design System and Builder typechecks: passed.
- Scoped Oxlint and formatting: passed.
- Package-boundary validation: passed.
- Focused visual regression: 23 affected stories compared; the newly
added dark login story is the only difference.
- Fixture impact: no fixture inputs or generated outputs are affected.

Ref #3818
 2026-08-31 21:16:26 +02:00
Oleg Isonen 4273b7001f refactor: Organize Builder development and test tooling (#6285) 
## Outcome

Builder development, backend, E2E, and CI tooling now share one
ownership model: Builder defines named commands and backend modes, while
CI only schedules those commands.

## Technical choices

- Group shared Compose files, PostgreSQL initialization, and backend
helpers under `apps/builder/backend`.
- Resolve `compose.<mode>.yaml` by convention instead of maintaining
Compose filename lists in scripts and workflows.
- Keep persistent development and disposable test Compose projects
separate by default, while allowing the standard `COMPOSE_PROJECT_NAME`
override.
- Make schema snapshots invalidate from discovered migration sources and
backend files.
- Move E2E behavior recipes into named Builder commands; retain concise
root proxies for developer entry points.
- Let Vitest allocate available browser ports instead of maintaining a
cross-package port registry.
- Keep test-suite scheduling explicit in `main.yml`, but move executable
test commands to root package scripts.
- Move fixture-size enforcement from inline GitHub JavaScript into a
locally runnable repository script.

## Todo

- [x] Consolidate Builder backend and Compose ownership.
- [x] Add named development, E2E, schema-cache, Supabase, and CI-shard
commands.
- [x] Remove Compose topology and behavior flags from CI.
- [x] Remove the static Vitest browser-port registry.
- [x] Extract fixture-size checking from the workflow.
- [x] Verify migration bootstrap, snapshot-backed E2E, tests, builds,
types, formatting, lint, generated contracts, and workflow syntax.
- [x] Review documentation impact: updated the E2E developer guide; no
user-facing `docs/` content is affected.

## Verification

- Full package, CLI, Builder core, and Builder app test suites
- Full workspace typecheck
- Builder production build
- Schema-cache generation through the complete migration history
- Supabase migration compatibility
- Snapshot-backed Playwright animation workflow
- Backend regression tests, including demonstrated fail/pass coverage
- All four Compose mode configurations
- `oxfmt`, `oxlint`, package boundaries, generated API, and generated
docs checks
- `actionlint` and Bash syntax checks

The fixture-size checker preserves the existing Linux `du` thresholds.
Its three inputs build locally; macOS allocation sizes are not directly
comparable to the Linux CI thresholds, so CI remains the authoritative
size result.
 2026-08-31 21:09:54 +02:00
Oleg Isonen bc8630f413 fix: Gate Builder releases on the published CLI contract (#6282) 
Closes #6280

## Summary

- Expose the deployed Builder's CLI-facing bundle contract through a
non-cached, read-only endpoint.
- Resolve and execute the exact `webstudio@latest` registry artifact,
observe the bundle version sent by its real `sync` request, and require
an exact match before assigning the release staging alias.
- Include expected and received bundle versions in runtime and
release-gate errors.

## Technical choices

- The existing generated `bundleVersion` remains the single
compatibility boundary. Builder-only changes outside that project-bundle
contract do not require a CLI release.
- The CLI sync probe runs against a local fixture API and temporary
directory. It reads the deployed runtime contract but never reads or
modifies a Cloud project.
- Verification runs after Vercel creates the exact production-built
deployment and before the release staging alias is assigned. A mismatch
therefore leaves production domains untouched.
- Registry resolution requires version and integrity metadata, then
executes the resolved immutable version rather than source from the
checkout.
- The gate requires the deployed Builder origin and cannot fall back to
the checkout's contract.

## Todo

1. [x] Add the deployed Builder contract endpoint.
2. [x] Probe the exact published CLI through its real sync request.
3. [x] Gate the release staging alias on an exact contract match.
4. [x] Report expected and received versions for compatibility failures.
5. [x] Add regression tests and verify the production Builder build.
6. [x] Review documentation and fixture impact.

## Verification

- `pnpm --filter @webstudio-is/builder exec vitest run
app/services/build-router.server.test.ts
app/services/cli-compatibility-route.node.test.ts` — 12 passed.
- `pnpm --filter webstudio exec vitest run
scripts/builder-contract-gate.test.ts
scripts/release-smoke-registry.test.ts` — 7 passed.
- Rebased onto `main` at `9ec4c820e1`.
- Builder and CLI typechecks passed.
- `pnpm lint` and changed-file formatting passed.
- `pnpm --filter @webstudio-is/builder build` passed and emitted
`/rest/cli-compatibility`.
- The changed GitHub Action parses as valid YAML.
- Real registry probe passed for `webstudio@0.293.0` against
`bundle-kzb45v`.
- Real registry probe correctly blocked current Builder contract
`bundle-1mym82p` and reported expected `bundle-1mym82p`, received
`bundle-kzb45v`.
- Missing staged Builder origin fails closed before registry execution.
- GitHub CI passed all four workflows, including all six Builder E2E
shards.
- The formerly failing empty-MDX insertion workflow passed 8 repeated
runs with 4 workers locally, then passed in Builder E2E shard 4 in CI.
- Full browser-based release smoke was not run, per the request not to
use Chrome. Agent evaluations were not run because no evaluation
approval was requested or received.
- Documentation: no user-facing documentation changes are required.
- Fixtures/generated outputs: no fixture inputs or generated outputs are
affected.

## Operational boundary

The check needs the exact deployed runtime, so Vercel creates an
unaliased `--skip-domain` deployment before verification. A failed check
cannot affect production domains or the normal release alias. Repository
automation cannot prevent a Vercel administrator from manually promoting
an arbitrary deployment outside this release path.
 2026-08-31 20:43:52 +02:00
Oleg Isonen 1b48716ff4 docs: Document MDX-backed Content Blocks (#6273) 
## Summary

- Document how people create, connect, edit, validate, and publish
MDX-backed Content Blocks.
- Document the complete MCP workflow, including scoped expression
sources, frontmatter updates, diagnostics, conflicts, and disconnect
behavior.
- Add two focused screenshots for the human workflow.
- Fix issues found by a low-reasoning MCP agent while executing every
documented workflow.

Ref #6182

## Implementation notes

- Connection confirmation uses the public `confirmReplacement` field
because `confirm` is reserved as a hidden transport field.
- MCP edits frontmatter with `update-content-block-frontmatter`;
materialized MDX elements are not persistent targets for generic
instance mutation tools.
- Reload now intentionally discards a conflicted local source and
accepts the Asset's latest storage revision before an edit is reapplied.
- Generated MCP documentation and runtime contracts are updated from
their source files.

## Checklist

- [x] Document the human MDX Content Block workflow.
- [x] Document MCP discovery, connection, editing, diagnostics,
frontmatter, dynamic sources, conflicts, and disconnection.
- [x] Add only screenshots that clarify otherwise hard-to-describe UI
steps.
- [x] Test every documented MDX capability through the local MCP CLI
with a low-reasoning agent.
- [x] Reproduce and fix the replacement-confirmation schema mismatch.
- [x] Reproduce and fix persistent-session conflict reload and reapply.
- [x] Add regression tests that failed before each product fix and pass
afterward.
- [x] Regenerate runtime contracts and generated MCP documentation.
- [x] Review authoritative `docs/` content and update affected pages.
- [x] Run affected tests, typechecks, formatting, and diff checks.

## Verification

- Project Build: 208 affected tests passed.
- Content Engine Asset session: 8 tests passed.
- CLI documentation tests: 14 tests passed.
- MCP docs generator: 3 tests passed.
- Content Engine, Project Build, CLI, and MCP docs typechecks passed.
- `pnpm oxfmt --check` passed.
- `git diff --check` passed.
- Low-reasoning MCP evaluation passed all documented workflows,
including a real two-client revision conflict, reload, reapply, and
final saved inspection.
 2026-08-31 18:39:05 +02:00
Oleg Isonen 8718feb465 test: simplify browser testing infrastructure (#6271) 
## Summary

Unifies frontend testing around three intentional boundaries:

- Vitest Node projects for pure and server tests
- Vitest Browser Mode with Playwright/Chromium for DOM and component
tests
- Playwright Test for Builder end-to-end workflows

The Builder no longer uses jsdom. Ordinary E2E workflows use an
automatically managed isolated `page` fixture; explicit contexts remain
only for simultaneous identities and suite-scoped setup.

## Technical choices

- Split Builder tests into one Node project and independent browser
projects so each Chromium/Vite process has a bounded workload.
- Added shared Browser Mode configuration for the affected SDK React
packages.
- Preserve the Builder's production top-window bridge ownership; Browser
Mode uses explicit test-only installation seams.
- Centralized Playwright context options and kept action/navigation
timeouts bounded.
- Validate Playwright filters before Docker, migrations, browser
installation, or Builder builds start.
- Preserve the first failed E2E attempt's report, traces, screenshots,
and videos when CI retries a shard.
- Split static checks, typechecks, CLI tests, core package tests,
browser package tests, and Builder projects into parallel CI jobs with
five-minute hard ceilings.
- Distribute Builder E2E tests across ten isolated shards with two
workers; parallel suites create unique per-worker setup data.
- Reuse color contrast DOM/canvas samples so the exhaustive theme sweep
takes about four seconds instead of timing out at 30 seconds.
- The removed cost is jsdom behavior drift, oversized serial check jobs,
and unnecessary E2E startup for empty selections. Real browser startup
and E2E environment setup remain.

## Todo

- [x] Split Builder pure/server tests from browser/component tests.
- [x] Migrate Builder DOM tests from jsdom to Vitest Browser Mode.
- [x] Migrate affected SDK React DOM tests to the shared Browser Mode
setup.
- [x] Remove jsdom annotations, dependencies, and compatibility setup.
- [x] Replace ordinary manual E2E contexts with the isolated `page`
fixture.
- [x] Keep explicit contexts only for concurrent identities and
`beforeAll` setup.
- [x] Preserve setup authentication isolation from workflow pages.
- [x] Add bounded Playwright action and navigation timeouts.
- [x] Preserve first-attempt E2E failure evidence across CI retry.
- [x] Reject empty E2E selections before expensive environment setup.
- [x] Parallelize repository checks into bounded CI shards.
- [x] Isolate parallel E2E setup data across workers.
- [x] Remove the color contrast sweep bottleneck.
- [x] Update E2E contributor documentation.
- [x] Review product documentation impact; no user-facing product docs
are affected.
- [x] Run the full repository tests and typechecks.
- [x] Run formatting, lint, package-boundary, generated-file, and
fixture checks.
- [x] Complete final correctness and simplification review.

## Verification

- `pnpm --filter @webstudio-is/builder test`
- `pnpm -r typecheck`
- `pnpm lint`
- `pnpm check:package-boundaries`
- `pnpm check:generated-api`
- `pnpm check:generated-docs`
- `pnpm fixtures`
- Design-system: 222 tests pass locally.
- Builder: all seven Vitest projects pass locally.
- The formerly slow E2E shard passes locally in 1m55s.
- Full database-backed Builder E2E execution passes in CI.
- All CI jobs pass; the slowest completed in 4m58s.
- No fixture inputs/outputs or generated files changed.

Ref #6270
 2026-08-31 17:17:28 +02:00
Oleg Isonen f6c524572d feat: add Builder dark theme and rebuild colors 
## Summary

Rebuilds Webstudio’s color system, migrates the entire Builder to it,
and introduces a complete dark theme.

A small set of theme colors and contrast controls now drives the
interface through relative CSS colors:

```text
Theme colors and contrast
    ↓
Light/dark derivation bounds
    ↓
Derived color families
    ↓
Reusable semantic colors
    ↓
Component composition
    ↓
Builder interface
```

CSS is the runtime source of truth. Light and dark modes share the same
theme inputs, so changing a theme color automatically updates its
surfaces, content, borders, states, and component presentation. The
refactor turns dark mode into a first-class Builder theme rather than a
separately maintained palette.

This is an atomic migration. Legacy color names and compatibility
aliases will not remain after all consumers have moved.

## Scope

- The color source, derivation model, semantic vocabulary, validation,
typing, and Storybook overview.
- Every design-system component.
- Builder, dashboard, authentication, shared UI, Query Builder, stories,
and tests that consume the legacy color scale.
- Removal of the legacy Stitches color scale after its final consumer is
migrated.
- A complete dark theme across application UI while keeping authored
canvas content independent.
- Complete light and dark visual verification.
- Unified browser-test infrastructure: Vitest Browser Mode for Design
System browser contracts and standard Playwright Test for all Builder
E2E workflows.

Literal colors remain only where color is the actual content or belongs
to a deliberately separate system, such as user-authored values,
external previews, brand artwork, data visualization, functional canvas
overlays, and source assets. Every remaining exception must have an
explicit owner.

## Architecture decisions

- Theme authors control six color families and three bounded contrast
controls.
- Relative CSS colors derive the rest of the system.
- Dark mode changes derivation bounds instead of maintaining a second
independent palette.
- Components consume a small reusable semantic vocabulary.
- Component states and exceptional presentation are composed inside
components.
- Builder-specific categorical meaning uses focused composite recipes,
not intent semantics or a larger global vocabulary.
- TypeScript validates public semantic CSS-variable names but does not
generate runtime colors.
- No token-manager dependency, compatibility aliases, or
component-specific public color tokens.
- A new semantic or theme variable is added only when existing semantics
and local composition cannot represent a reusable cross-feature
decision.

## Todo

### Color-system foundation

- [x] Establish direct CSS as the runtime source of truth.
- [x] Define bounded theme colors and contrast controls.
- [x] Derive light and dark modes from the same theme inputs.
- [x] Reduce the semantic inventory to reusable decisions and explicit
accessibility pairs.
- [x] Generate only TypeScript names for public semantic CSS variables.
- [x] Validate layers, references, cycles, consumption, naming,
generated types, and browser-computed contrast.
- [x] Add the Storybook color-system overview and theme controls.
- [x] Document the architecture and its relationship to the public Craft
specification.

### Design-system migration

- [x] Replace every active legacy color consumer in design-system
components and Storybook chrome.
- [x] Compose component states locally without component-specific public
tokens.
- [x] Preserve intentional preview and sample-content colors.
- [x] Add complete positive, negative, warning, informative, and accent
accessibility pairings.
- [x] Validate light and dark scheme-bound references independently.
- [x] Verify the design-system package with typecheck, lint, formatting,
Storybook, and 222 tests.

### Inventory and contract

- [x] Inventory all 319 legacy references across 107 Builder, Query
Builder, story, test, and design-system files.
- [x] Classify every reference as a direct semantic replacement, local
composite recipe, documented literal exception, or removable code.
- [x] Record the intentional authored-content, external-preview,
functional-preview, data-visualization, brand, and source-asset
exceptions.
- [x] Treat the public semantic vocabulary as closed by default and
justify any proposed addition by cross-feature reuse and accessibility
need.
- [x] Keep Builder code limited to typed public variables accepted by
`cssVar()`.

### Theme infrastructure

- [x] Add `system` (default), `light`, and `dark` application
preferences.
- [x] Persist the preference in existing client settings and resolve
`system` through `matchMedia`.
- [x] Apply the resolved scheme before paint and respond to
operating-system changes.
- [x] Expose the persisted theme control under **View → Color scheme**.
- [x] Keep authored canvas documents independent from the application
theme.
- [x] Ensure portals, overlays, embedded surfaces, and third-party
controls receive the correct interface scheme.

### Generic interface migration

- [x] Migrate shared dialogs, project settings, notifications, controls,
and Query Builder.
- [x] Migrate dashboard, authentication, publishing, and shared
application UI.
- [x] Migrate the Builder loading state, topbar, sidebars, navigator,
workspace, footer, settings, pages, alerts, and menus.
- [x] Replace fixed interface literals alongside their owning surfaces.
- [x] Remove the old scrollbar variable and hardcoded loading color.

### Style panel and style-source recipes

- [x] Migrate ordinary style-panel inputs, labels, layouts, spacing,
alignment, menus, property states, and value editors.
- [x] Model style-source provenance, kind, interaction, and error as
separate axes.
- [x] Compose categorical style-source colors from neutral and
accent-relative colors.
- [x] Use negative semantics only for actual errors, not overwritten
provenance.
- [x] Reuse one feature-owned recipe across style-source controls,
badges, menus, and values without adding global tokens.

### Canvas and editors

- [x] Migrate canvas selection, reusable outlines, grid guides, resize
handles, labels, badges, and overlays.
- [x] Keep collaborator colors as data visualization while guaranteeing
readable labels.
- [x] Prevent Builder theme values from leaking into authored content.
- [x] Migrate editor surfaces, selections, autocomplete, lint states,
and tooltips.
- [x] Derive syntax visualization from neutral and accent-relative
colors; reserve intent colors for diagnostics.

### Explicit color domains

- [x] Isolate external search and social previews from the application
scheme.
- [x] Preserve user-authored CSS values, functional color previews, and
checkerboards.
- [x] Review brand surfaces independently in both schemes.
- [x] Document every retained literal in the exception ledger.

### Stories, tests, and legacy removal

- [x] Migrate all affected Builder stories and tests.
- [x] Replace the design-system test that intentionally references the
legacy scale.
- [x] Add theme-preference and scheme-resolution behavior tests.
- [x] Remove the legacy `design-tokens.color` registry.
- [x] Remove the Stitches `colors` scale and `rawTheme.colors`.
- [x] Remove color-name conversion code that exists only for the legacy
scale.
- [x] Add a focused repository check that rejects legacy references.
- [x] Confirm no compatibility mappings or deprecated aliases remain.

### Browser-test infrastructure

- [x] Run pure Design System tests in Node and browser-dependent tests
in Vitest Browser Mode with Playwright Chromium.
- [x] Migrate all 79 Builder E2E workflows from the custom harness to
standard Playwright Test.
- [x] Replace filename-based shard discovery with standard Playwright
sharding.
- [x] Preserve clean-backend shard retries and upload Playwright failure
artifacts in CI.
- [x] Remove the custom runner, registry, and timeout infrastructure.

### Verification and documentation

- [x] Compare light-mode visual regressions directly with `main`; review
every difference rather than accepting snapshots wholesale.
- [x] Add representative dark coverage for dashboard, Builder shell,
style panel, dialogs, editors, canvas tools, and status states.
- [x] Test default and extreme custom-theme presets.
- [x] Verify semantic and categorical-recipe contrast.
- [x] Test first paint, persistence, system changes, portals, and canvas
isolation.
- [x] Run affected tests, typechecks, lint, formatting, and Storybook
checks.
- [x] Review fixture impact; no fixture inputs or generated outputs are
affected.
- [x] Review authoritative `docs/` content and update it where needed.
- [x] Review the final diff and make this checklist reflect delivered
scope.

## Current verification

- Design-system typecheck and generated-type freshness: passed.
- Design-system tests: 222 passed, including the full theme-parameter
contrast sweep and browser component coverage.
- Builder typecheck and production build: passed.
- Theme preference and scheme-resolution tests: 10 passed.
- Scoped Oxlint, formatting, and diff checks: passed.
- Design-system Storybook build and representative light/dark visual
review: passed.
- Builder E2E: 80/80 Playwright entries passed (setup plus 79
workflows).

Ref #3818
Closes #6267
 2026-08-30 03:24:56 +01:00
Oleg Isonen f9681dc30f feat: MDX-backed Content Blocks (#6234) 
## Summary

Supersede #6182 with a clean implementation of MDX-backed Content Blocks
on top of the unchanged `main` Content Block editing model. A connected
`.mdx` Asset changes where authored children are loaded and saved; it
must not change how Content mode inserts, edits, selects, moves, copies,
or deletes instances.

The previous implementation remains available in
`agent/mdx-content-block-6124` and
`agent/mdx-content-block-6124-review`. Reuse its independently tested
parsers, converters, schemas, Asset repository, template resolution,
publication, MCP, and UI work where those pieces fit the corrected
architecture. Do not port its per-operation storage routing, projected
editing stores, MDX undo/history, or atomic multi-root machinery.

Ref #6124
Ref #6182
Closes #6261

## Locked behavior

- `.md` keeps its existing Markdown behavior; `.mdx` enables MDX.
- Standard Markdown structures are stored as Markdown. JSX is used only
for structures Markdown cannot represent losslessly.
- Generic elements use `<ws.element ws:tag="…">`; template instances use
`<ws.element ws:name="…">`.
- Invalid, unsafe, or unsupported MDX never prevents editing the file or
publishing the project. Builder and publication use the same recovery
result: recoverable nodes are skipped with source-located warnings,
while an unrecoverable document renders no authored file content in
either environment.
- Connecting a source never serializes existing Content Block children
into the file. **Connect** removes ordinary persisted children,
preserves Templates, sets `src`, and uses file content.
- Disconnecting resets `src`, discards derived instances, leaves the
Content Block empty, and never changes the Asset.
- While connected, normal Content Block mutations update in-memory
instances synchronously. Asset persistence happens afterward and never
blocks rendering.
- Genuine external revision conflicts use the standard reload-required
flow. Rapid local edits are ordered/coalesced and must never conflict
with each other.
- Repeated scopes use Content Block ID plus Collection item identity.
- Multiple Content Blocks may intentionally share one MDX Asset. They
share one revisioned document session and edits propagate to every
connected occurrence.
- Missing templates show warnings/placeholders only in Builder and are
skipped on published sites.

## Reuse strategy

- [x] Audit all 62 original implementation commits and 5 later
review/fix commits
- [x] Reuse the shared Content Engine Markdown/MDX AST, parser,
serializer, diagnostics, reference discovery, and tests
- [x] Reuse SDK source schemas, instance-name utilities, unique template
naming, and source-integrity validation
- [x] Reuse template resolution/materialization and authored-tree
conversion after removing storage-operation coupling
- [x] Reuse revisioned Asset content repository, authorization, limits,
and browser/server transport
- [x] Reuse publication, document-graph, CLI, MCP, source-control UI,
frontmatter UI, and E2E work where contracts remain valid
- [x] Reuse tests as behavioral specifications; rewrite tests that
assert the rejected projection/routing architecture
- [x] Do not port per-operation MDX routing, alternate projected
mutation stores, semantic reconciliation after each command, MDX Builder
history/undo, atomic multi-file rollback, or canvas loading/saving UI


## Old-branch porting map — do not rewrite

Before implementing any checklist batch, inspect the named
implementation and tests on `agent/mdx-content-block-6124-review`. Move
the implementation and its tests first; change only the boundary that
conflicts with the rebuilt architecture. Do not create a second
implementation under a new name. When a test encodes the rejected
projected-store architecture, retain its behavioral assertion and
replace only its setup.

### Already ported

- `parseMarkdownAst`, `createMdxDocumentFromMarkdown`,
`parseMdxDocument`, `serializeMdxDocument`, `preferMarkdownSyntax`,
`discoverMdxBodyAssetReferences`, `discoverMdxAssetReferences`, and
`previewMarkdownToMdxConversion`
- Asset-reference, byte-stream, document-graph, Markdown, and MDX
helpers included in commit `76bc388ba9`
- Tests: the corresponding `packages/content-engine/src/**/*.test.ts`
files already on this branch

### Port with their existing tests

| Area | Functions to move or adapt | Tests to move with them |
| --- | --- | --- |
| Source contract and integrity | `parseContentBlockSourceProp`,
`getContentBlockSource`, `isEqualContentBlockSource`,
`formatContentBlockSourceIntegrityIssue`,
`getContentBlockSourceIntegrityIssues`,
`allocateUniqueContentBlockTemplateName` |
`packages/sdk/src/content-block.test.ts`,
`packages/sdk/src/schema/content-block.test.ts` |
| Existing instance/name utilities | `findChildReferenceIndex`,
`findParentInstanceReference`, `getInstanceName` |
`packages/sdk/src/instances-utils.test.tsx` |
| Template naming in mutations |
`assignUniqueBlockTemplateNamesMutable`,
`findBlockTemplateNameCollision`, `getSourceBackedBlockTemplateContext`,
`getBlockTemplateNameConfirmation`, `getBlockTemplateInsertionIndex` |
`packages/project-build/src/runtime/block.test.ts` |
| Content-mode property permissions | `getContentModePropEligibility`;
preserve its distinction between editable, design-only, incompatible,
and unknown props |
`packages/project-build/src/runtime/content-mode-permissions.test.ts` |
| Template resolution | `resolveMdxTemplates` |
`packages/project-build/src/runtime/mdx-template-resolution.test.ts` |
| Template materialization | `createMdxScopeIdGenerator`,
`materializeMdxTemplates` |
`packages/project-build/src/runtime/mdx-materialization.test.ts` |
| MDX → instances and instances → MDX |
`isMaterializedMdxAuthoredContentRoot`, `preserveMdxAuthoredContentIds`,
`materializeMdxAuthoredContent`, `reconcileMdxAuthoredContent`,
`omitTransientEmptyParagraphs`, `serializeMdxTemplateInsertion` |
`packages/project-build/src/runtime/mdx-authored-content.test.ts`;
retain the pure serialization cases from `mdx-storage-adapter.test.ts` |
| Revisioned Asset I/O | `createAssetContentRevision`,
`readAssetContentBytes`, the package-owned
`createHttpAssetContentRepository`, the thin Builder
`createBuilderHttpAssetContentRepository`, `createAssetContentBridge`,
`initAssetContentBridge`, `getAssetContentBridge`; keep one repository
implementation rather than copying the old Builder duplicate |
`packages/asset-uploader/src/content-repository.test.ts`,
`apps/builder/app/builder/shared/assets/mdx-content-repository.test.ts`,
`apps/builder/app/shared/asset-content-bridge.client.test.ts` |
| Source lifecycle planning | `createContentBlockBodyRemoval`,
`prepareContentBlockConnect`, `prepareContentBlockDisconnect`,
`prepareContentBlockSwitch`, `ContentBlockSourceRevisionConflictError` |
`packages/project-build/src/runtime/content-block-source-lifecycle.test.ts`
|
| Per-Asset revision/session behavior | `getContentBlockSessionSource`,
`isContentBlockSessionSourceCommitted`; reuse the loading,
authorization, revision, queue, conflict, retry, cancellation, and
source-replacement internals of `createMdxAssetEditingSession`, but key
the rebuilt session by Asset rather than Content Block/render root |
Port the applicable cases from
`packages/project-build/src/runtime/mdx-asset-session.test.ts`; replace
root-projection setup with in-memory snapshot setup |
| Template migrations | `validateMdxTemplateMigrationConfirmation`,
`planMdxTemplateMigration` |
`packages/project-build/src/runtime/mdx-template-migration.test.ts` |
| Publication | `createPublishedMdxMaterializationCache`,
`materializePublishedMdx`, `getPublishedMdxContentDatabaseMaxBytes`,
`getDynamicPublishedMdxSourceBlockIds`, `hasDynamicPublishedMdxSources`,
`createPublishedBuildContentCompilationPlan`,
`resolvePublishedMdxDependencyClosure`,
`resolvePublishedMdxAssetCandidates` |
`packages/project-build/src/runtime/published-mdx.test.ts`,
`packages/project-build/src/content-database.test.ts` |
| Build/import integrity | `assertContentBlockSourceIntegrity` and the
Content Block cases in
`getBuildIntegrityIssues`/`formatBuildIntegrityIssue` |
`packages/project-build/src/build-integrity.test.ts` |
| MDX paste/MCP insertion | `insertMdxText` and its input/result
schemas; retain parsing, confirmation, template-resolution, and
insertion-index logic, then invoke the normal mutation path |
`packages/project-build/src/runtime/mdx-paste.test.ts` |
| Builder source control | `createContentBlockSourceController`,
`createBuilderContentBlockSourceController` and the existing
source-control components | `content-block-source-controller.test.ts`,
`content-block-source.test.tsx` |
| Diagnostics | `formatContentBlockDiagnostic`,
`deduplicateContentBlockDiagnostics`, `takeNewContentBlockDiagnostics` |
Port only their focused cases from
`apps/builder/app/shared/content-block-content.test.ts` |

### Explicitly do not port

- `projectContentStorageChanges`, `createContentStorageProjection`,
`createContentStorageSelectorProjection`,
`executeContentStorageMutation`, `executeContentStorageTextReplacement`,
`executeContentStoragePropMutation`, and
`executeContentStorageStructuralMutation`
- `applyMdxContentStorageChanges` and `prepareMdxContentStorageWrites`;
replace them with one post-mutation snapshot serializer built from the
reusable pure `reconcileMdxAuthoredContent` converter
- `createContentBlockApplicationOperations` and the persistence-plan
wrappers that route individual editing commands
- Builder `$runtime*` projected stores, materialized mutation functions,
alternate selector/tree helpers, pending projected-change state, and
MDX-specific mutation sequences from `content-block-content.ts`
- Canvas loading/saving/open-file presentation state, MDX history/undo,
and atomic rollback machinery

## Implementation checklist

### 1. Preserve the `main` editing architecture

- [x] Keep normal synchronous insert, text, Enter, slash menu, property,
delete, move, wrap, copy/paste, selection, and focus mutations unchanged
- [x] Materialize connected content through the same runtime instance
interfaces used by ordinary Content Blocks
- [x] Keep connected authored children in memory and exclude them from
persisted build data
- [x] Route persistence only after a synchronous mutation has updated
the canvas
- [x] Never wait for Asset loading, conversion, or saving before
rendering an interaction
- [x] Add no MDX-specific implementation to individual editing commands

### 2. Shared Markdown and MDX conversion

- [x] Parse Markdown and the Markdown portion of MDX through one shared
AST and plugin configuration
- [x] Implement one shared MDX-to-instances converter and one shared
instances-to-MDX converter
- [x] Serialize headings, paragraphs, links, images, lists, tables, task
lists, blockquotes, thematic breaks, code, emphasis, strong,
strikethrough, and other supported GFM structures as Markdown
- [x] Serialize only non-lossless/custom structures as Webstudio JSX
- [x] Preserve nested Markdown in MDX elements, frontmatter, comments,
tight/loose list semantics, code whitespace, reference links, and Asset
references
- [x] Omit transient empty draft paragraphs instead of emitting
`<ws.element ws:tag="p" />`
- [x] Preserve unsupported source text and authored metadata whenever it
cannot be converted safely
- [x] Add deterministic round-trip and canonical-output coverage for
every supported Markdown structure

### 3. Invalid and unsafe MDX

- [x] Preserve file source exactly; never silently repair or rewrite
invalid source
- [x] Ignore imports, exports, expressions, executable JavaScript,
invalid properties, and other unsupported nodes when they can be
isolated
- [x] Continue rendering valid siblings and report every skipped subtree
once with filename, line, and column
- [x] Use the same recovery pipeline in Builder and publication so both
render the same authored result for the current file revision
- [x] When structural syntax prevents recovery, render no authored file
content in Builder or publication instead of retaining a stale
materialization
- [x] Skip an unrecoverable Content Block on published sites and
continue the build with a warning
- [x] Keep invalid and unsafe files editable and reconnectable
- [x] Enforce UTF-8, byte-size, depth, node-count, property-count, and
generated-output limits with source-located diagnostics where possible

### 4. File formats and Asset creation

- [x] Keep `.md` parsing and raw-HTML behavior unchanged
- [x] Support creating, opening, editing, deleting, and recreating
`.mdx` files through the normal text-file UI
- [x] Open `.mdx` in the same editor experience as `.md`
- [x] Always create an independent empty MDX Asset even when identical
empty files exist
- [x] Allow renaming `.md` to `.mdx` without converting or rewriting its
contents; after rename, handle invalid contents through the normal MDX
diagnostics and recovery pipeline
- [x] Update an already-open MDX editor immediately after canvas
persistence
- [x] Bound read/write buffering and verify the returned Asset identity
and revision

### 5. Source contracts and lifecycle

- [x] Support direct Asset bindings and dynamic expression bindings
- [x] Keep source mutation controls available only in Design mode; show
the connected filename and **Open** action read-only in Content mode
- [x] Reject duplicate source props, unsupported prop types, missing
Assets, non-MDX Assets, and unauthorized Assets
- [x] Analyze selected file diagnostics before confirmation without
modifying the block or file; MDX syntax diagnostics do not block
connection
- [x] For a non-empty block, show **Abort** and **Connect** and explain
that ordinary children will be replaced
- [x] On connect, remove every non-template child, preserve Templates,
set `src`, and materialize file content
- [x] Replace disconnect handoff with normal `src` reset; discard
derived content and leave the block empty
- [x] Remove disconnect confirmation and conversion; resetting a
repeated binding leaves every occurrence empty
- [x] Switching sources validates/loads the target before replacing the
usable binding and leaves both files unchanged
- [x] Revalidate Asset revision and project version when confirmation is
submitted
- [x] Make repeated connect/disconnect operations idempotent
- [x] Prevent disposed, stale, or out-of-order loads from replacing the
newest source
- [x] Keep loading, saving, conflict, empty-file, and open-file actions
off canvas

### 6. Asynchronous persistence and conflicts

- [x] Snapshot the current in-memory authored root after normal
mutations
- [x] Serialize/coalesce rapid edits per resolved Asset and always
prepare against the latest local snapshot
- [x] Allow independent Assets to save concurrently without scope
leakage
- [x] Use one revisioned document session and one ordered save queue per
Asset, even when multiple Content Blocks or render scopes reference it
- [x] After a shared Asset changes, rematerialize every connected
occurrence without persisting their generated instances to project data
- [x] Pin the resolved Asset and exact loaded revision for each write
- [x] Recheck authorization immediately before storage
- [x] Ensure local queued edits never produce stale-revision conflicts
with each other
- [x] On a genuine external revision conflict, preserve local content,
stop writes, and enter the standard reload-required state
- [x] Keep failed non-conflicting writes retryable without discarding
local content
- [x] Finish already accepted persistence writes after a Content Block
unmounts or switches source
- [x] Cancel obsolete loads, materializations, and UI updates after
unmount, reload, or source switch
- [x] Dispose project-scoped roots, queues, parse caches, failure state,
generations, and session subscriptions when the Asset session is
replaced
- [x] Invalidate Asset resources and synchronize open file editors after
successful writes
- [x] Keep multi-root persistence ordered, serial, and non-atomic;
report partial completion without rollback
- [x] Do not add Asset-history undo/redo; native text-editor undo
remains local to the open editor

### 7. Content mode parity

- [x] Empty connected blocks remain selectable and expose the normal
Content-mode `+` affordance
- [x] First insertion renders immediately
- [x] Enter creates and focuses the next paragraph immediately
- [x] Slash commands work in newly created empty paragraphs
- [x] Deleting one selected instance deletes exactly that instance
- [x] Mouse and keyboard moves use the same operation and permissions
- [x] Preserve sibling reorder, nested reparenting, and moves between
editable Content Blocks where `main` permits them
- [x] Preserve content-editable props such as Link `href`
- [x] Preserve copy, paste, cut, multi-cut, wrapping, unwrapping, grid
insertion, selection, Navigator order, and canvas order
- [x] Cover mutation routing centrally, retain the full unconnected
Builder suite, and exercise representative connected interactions in
real Builder E2E

### 8. Template names and permissions

- [x] Resolve only exact user-facing names from the direct flat
Templates list
- [x] Enforce unique names within each Templates list
- [x] Reject duplicate names during rename with inline validation
- [x] Allocate unique names during insert, paste, duplicate, move, wrap,
and unwrap
- [x] Apply valid template properties using the same permissions as
Content mode
- [x] Preserve and warn about ignored design-only, invalid, or stale
properties
- [x] Preserve template shell provenance without serializing expanded
template internals
- [x] Generate deterministic materialized IDs from source identity,
render scope, and authored usage path so focus/selection survives saves
- [x] Warn before renaming or deleting referenced templates; files
remain unchanged
- [x] Provide shared AST-based rename/remove migration operations for
MCP-assisted multi-file cleanup

### 9. Scope, frontmatter, and diagnostics

- [x] Scope materialized content by Content Block ID plus exact
Collection-item/render context
- [x] Prevent repeated instances and dynamic candidates from leaking
generated IDs, template resolution, or diagnostics across scopes while
sharing the underlying Asset source and revision where applicable
- [x] Remove frontmatter editing from Settings; edit the exact YAML
through the MDX Asset file editor
- [x] Preserve the exact frontmatter source byte-for-byte during canvas
body saves
- [x] Route canvas body changes and complete-file editor saves through
one revisioned per-Asset coordinator
- [x] Show a source warning indicator in Settings and precise
source-located diagnostics in the Asset editor
- [x] Deduplicate repeated diagnostics without losing distinct locations
- [x] Keep unresolved-template placeholders selectable and accessible
only in Builder

### 10. Asset references, transfer, and integrity

- [x] Discover and rewrite Asset references in frontmatter, Markdown
links/images, and MDX properties
- [x] Preserve Asset references while editing unrelated content
- [x] Include referenced Asset and template revisions in
dependency/cache identity
- [x] Route referenced Assets through published URL rewriting
- [x] Validate Content Block source integrity during project import,
fragment/page transfer, copy/paste, build integrity, and pre-publish
audit
- [x] Allow multiple Content Blocks and dynamic render scopes to
reference the same MDX Asset without treating the connection as an
integrity error
- [x] Preserve Templates when copying a connected Content Block
- [x] Preserve direct and dynamic Content Block source bindings when
copying or pasting within the same project
- [x] When copying only one rendered Collection item, resolve that
item’s current MDX Asset and replace the copied Content Block’s dynamic
source with a direct binding to that same Asset
- [x] When copying an entire Collection, preserve the Collection, its
instances, and its dynamic source bindings
- [x] Preserve the complete MDX source and frontmatter by reference;
copying a connected block must not rewrite the shared file
- [x] During cross-project transfer, copy every discoverable distinct
bound MDX Asset and rewrite transferred bindings to the destination
Assets
- [x] If some cross-project dynamic candidates cannot be discovered or
copied, continue with successful copies, preserve what can be
represented safely, and warn about skipped candidates
- [x] Revalidate source revisions and the destination project before
applying a cross-project transfer
- [x] If cross-project Asset creation or project insertion fails, leave
the source project unchanged and clean up any newly reserved destination
Asset
- [x] Use the same transfer operation through Builder and MCP

### 11. Dynamic sources and publication

- [x] Resolve dynamic bindings once per render/save scope
- [x] Require finite, discoverable candidate sets for publication and
load no unrelated MDX
- [x] Warn and skip unbounded, missing, unreadable, malformed, or unsafe
sources without failing the complete build
- [x] Materialize only rendered Content Blocks; ignore Templates
definitions and `src` on non-Content-Block instances
- [x] Skip unresolved template subtrees while keeping valid siblings
- [x] Include selected template styles, fonts, resources, unresolved
names, and referenced Asset revisions in generated dependency output
- [x] Keep draft-only MDX Assets out of publication retention

### 12. Builder, MCP, CLI, and shared operations

- [x] Use the same converters, source lifecycle, revision checks,
persistence service, diagnostics, and template migrations in Builder and
MCP
- [x] Keep frontmatter editing file-based; retain whole-frontmatter
runtime operations only where compatibility requires them
- [x] Avoid Builder-only or MCP-only mutation/conversion implementations
- [x] Regenerate runtime/API contracts from their source definitions
- [x] Update CLI Content Engine reference and generated MCP
documentation

### 13. Verification and delivery

- [x] Add regression tests that reproduce failures from the previous
implementation, including double deletion and blocked rendering
- [x] Cover two Content Blocks sharing one Asset: edits from either
block serialize once, update both canvases, preserve scope-specific
Templates, and cannot overwrite a newer local snapshot
- [x] Verify new tests fail against the broken implementation and pass
with the corrected behavior
- [x] Run SDK, Content Engine, Asset Uploader, Project Build, Builder,
React SDK, CLI, Protocol, and HTTP Client affected suites and typechecks
- [x] Run real Builder E2E for source lifecycle, live file refresh,
empty insertion, Enter/slash focus, Link props, delete, keyboard
reorder, switching, disconnect, conflicts, unresolved templates, dynamic
sources, and repeated scopes
- [x] Regenerate shared project fixture bundles and verify a second
generation leaves the tree clean
- [x] Run lint, package boundaries, `pnpm oxfmt --check`, and `git diff
--check`
- [x] Review and update authoritative `docs/` content and navigation
- [ ] Run the complete agent evaluation suite — not run because it
requires separate explicit approval

### 14. Content-mode parity and disconnect handoff

- [x] Allow keyboard reordering in Content mode through the normal move
command while keeping the editable Content Block boundary
- [x] Let already accepted Asset writes drain independently before
resetting the source; disconnect does not load, flush, or convert the
file
- [x] Remove MDX-to-instance copying from disconnect
- [x] Reset `src`, remove the derived tree, and leave the ordinary
Content Block empty
- [x] Preserve automatic text-editor focus after inserting editable
content from the template menu
- [x] Remove Builder frontmatter-save UI and report complete-file save
failures through the Asset editor
- [x] Add regression coverage for keyboard movement, disconnect
teardown, source switching, insertion focus, and paragraph deletion
focus restoration
- [x] Run the complete six-scenario MDX Content Block Builder E2E suite

### 15. Final architecture review fixes

- [x] Keep one canonical ordered pending document per Asset so shared
Content Block occurrences cannot conflict with one another
- [x] Merge ordered local edits from different occurrences while
preserving text changes, insertions, deletions, and sibling reordering;
keep external revisions on the strict reload-required path
- [x] Centralize typed render-scope parsing, repeated-occurrence
detection, scoped variable lookup, and dynamic Asset resolution across
Canvas, Settings, source lifecycle, and copy/paste
- [x] Capture the owning Asset session when a save is accepted so normal
unmount/source-switch teardown cannot redirect or lose it
- [x] Dispose project-owned roots, queues, caches, generations, failure
state, and subscriptions when replacing the project Asset session
- [x] Add regression coverage for same-Asset edits from two occurrences,
same-node last-write ordering, structural reorder, insertion
preservation, scoped source resolution, and project cleanup
- [x] Re-run affected full unit suites, typechecks, production build,
static checks, and all six MDX Content Block Builder E2E scenarios


### 16. Final persistence and transfer hardening

- [x] Drain every accepted per-Asset edit before project-session
teardown while retaining the owning authorization context
- [x] Drain canvas serialization queues and revisioned Asset writes
before pre-publish audit and publication
- [x] Block publication on a queued serialization failure or Asset
revision conflict
- [x] Copy actual Asset bytes during cross-project paste, preserve MDX
reference names, rewrite direct and Collection-backed Asset IDs, and
clean up newly created Assets when insertion fails
- [x] Include every resolved dynamic MDX source when copying an entire
Collection and warn while preserving successful copies when individual
Assets cannot be transferred
- [x] Keep MDX editor save failures visible with retry and route
conflicts through the standard reload-required flow
- [x] Give keyboard nested reparenting the same editable Content Block
boundary used by pointer movement
- [x] Add regression coverage and rerun the complete Content Engine and
Builder unit suites plus type, lint, formatting, package-boundary, and
diff checks

## Current rebuild status

- The connected Content Block now owns one complete MDX document:
frontmatter feeds designed-shell bindings and the explicit Body outlet
owns the authored MDX body.
- Builder and published output resolve the same frontmatter document
graph, including nested document references and Asset references.
- Direct frontmatter text/property edits use the existing ordered
per-Asset session; referenced-document values remain file-owned.
- Repeated Collection occurrences keep isolated runtime shell/Body
identities and occurrence-specific document values.
- The local blog article layout has been upgraded to the document/Body
model and its bindings validate cleanly.
- Full affected package suites and typechecks pass; every new regression
was demonstrated fail/pass.
- `pnpm lint`, package boundaries, `pnpm oxfmt --check`, generated
API/docs checks, fixture builds, and `git diff --check` pass.
- Full fixture synchronization is blocked by the development API
rejecting this local CLI version before fixture generation; existing
fixture bundles build successfully.
- Complete agent evaluations remain pending explicit approval.

### 17. Treat Content Block source as a property

- [x] Declare Content Block `src` in component metadata as a Design-mode
initial property labeled **Source**
- [x] Add the short property description: “Create an .mdx file in
Assets, then select or bind it here.”
- [x] Accept only a direct Asset or an expression resolving to an Asset
ID; reject URL/string sources, missing Assets, and non-MDX Assets
- [x] Extract and reuse the Image-style Asset picker and binding
presentation without duplicating source-control UI
- [x] Render Source through the normal properties section and remove the
separate source-section placement and special `src` filtering
- [x] Route Source changes through Content Block connect, switch, and
disconnect lifecycle operations instead of generic prop updates
- [x] Keep connect confirmation, Open, diagnostics, and persistence
state attached to Source; remove frontmatter and disconnect UI
- [x] Add regression coverage for direct Assets, dynamic Asset bindings,
invalid resolved values, lifecycle operations, and Design/Content mode
visibility
- [x] Re-review authoritative `docs/` content for the Source-property UI
change and update it if required

### 18. Source property UX

- [x] Show one full-width bindable Connect .mdx file button while
disconnected
- [x] Show equal-width filename and Open buttons while connected
- [x] Switch files from the filename button through the source lifecycle
- [x] Override PropertyLabel reset and Alt-click with the disconnect
lifecycle
- [x] Preserve confirmation, pending, failure, repeated-scope, and
dynamic-source behavior
- [x] Add regression coverage for connect, switch, binding, reset, and
disconnect
- [x] Update Content Block documentation
- [x] Run affected tests, typecheck, lint, formatting, package
boundaries, and diff checks


### 19. Reduced file-owned frontmatter follow-up

This checklist supersedes the earlier structured-frontmatter and
disconnect-copy behavior. Frontmatter remains file-owned and is edited
only in the MDX Asset editor.

#### A. Remove structured frontmatter UI

- [x] Remove the JSON frontmatter editor and unused Builder
controller/state
- [x] Keep source mutation Design-mode-only; show connected Source
read-only in Content mode and allow authorized editors to open the
complete file
- [x] Retain runtime frontmatter APIs only where compatibility requires
them

#### B. Preserve source and separate validation

- [x] Preserve the exact frontmatter envelope during every canvas body
save without parsing/stringifying YAML
- [x] Preserve comments, ordering, quoting, whitespace, BOM, newline
style, delimiters, and invalid but closed YAML
- [x] Keep the existing unsupported-node/property MDX recovery; do not
add an ad hoc partial parser
- [x] Parse body and frontmatter independently so either valid half
remains usable
- [x] Keep bounded Asset-query frontmatter extraction unchanged
- [x] Treat structurally malformed MDX as having no safe body tree

#### C. Share Asset state with the raw editor

- [x] Route canvas and complete-file changes through one ordered,
non-blocking per-Asset coordinator
- [x] Update canvas and clean raw editors immediately from shared
in-memory source
- [x] Reject stale dirty-editor saves, preserve unsaved text, and
require reload; never force-overwrite
- [x] Drain accepted writes after source replacement/disconnect and
report background failures globally
- [x] Dispose sessions only after they have no consumers and no queued
work

#### D. Connect, replace, and disconnect

- [x] Load and diagnose before connect; block only
unreadable/unauthorized/non-MDX Assets
- [x] Show **Abort / Connect** for non-empty blocks, including
diagnostics and the recoverable-content warning
- [x] Set `src` and delete persisted children through one runtime
mutation
- [x] Switch to a loaded source without waiting for the previous Asset's
network save
- [x] Remove disconnect confirmation and conversion
- [x] Reset `src` immediately, discard the derived tree, leave the block
empty, and preserve the Asset
- [x] Resetting a bound source leaves every repeated occurrence empty

#### E. Dynamic and shared sources

- [x] Keep one source/revision state per Asset and a separate derived
instances tree per occurrence
- [x] Resolve Templates independently per Content Block and preserve
Builder-only missing-template placeholders
- [x] Resolve bound `src` from the selected Collection occurrence
- [x] Make **Open** use the occurrence's resolved Asset; update the
filename and disable Open when unresolved
- [x] Add direct/bound tests including Collection item A opening `a.mdx`
and item B opening `b.mdx`

#### F. Diagnostics, MCP, and publication

- [x] Show an accessible warning icon beside Source with the first
diagnostic, line/column, and remaining count
- [x] Show precise YAML/MDX diagnostics in the Asset editor without
repetitive canvas/toast UI
- [x] Reuse parser/diagnostic/revision contracts in Builder, build,
publication, and MCP
- [x] Keep MCP frontmatter editing file-based and return structured
diagnostics for complete-source writes
- [x] Emit build warnings; published sites skip invalid/unsupported
content without diagnostic UI

#### G. Verification and documentation

- [x] Add fail/pass regressions for source preservation, independent
validation, stale raw saves, lifecycle, dynamic Open, shared Assets, and
cleanup
- [x] Run affected tests, typechecks, lint, formatting, package
boundaries, fixtures, and diff checks
- [x] Open Markdown and MDX editors at twice the default dialog size
with the right-side preview visible initially
- [x] Document canvas body editing, file-owned frontmatter, Content-mode
Asset access, and cloning an MDX Asset for a new post
- [x] Perform a final correctness, data-loss, DRY, simplification,
accessibility, and performance review
- [x] Keep this checklist synchronized with implementation status

### 20. Content-mode component properties

- [x] Show a connected Content Block Source read-only in Content mode
with an active **Open** action
- [x] Keep source switching, binding, reset, and disconnect available
only in Design mode
- [x] Expose HTML Embed code and preserve Code Text's code-editor
metadata on authored Text Content
- [x] Audit every registered component and expose authored payload while
keeping design, structure, and behavior controls in Design mode
- [x] Keep media dimensions and Code Text theme Design-only while
preserving adapter-owned intrinsic Image metadata during MDX round trips
- [x] Add exact component-permission and regression coverage
- [x] Update authoritative component documentation
- [x] Run affected full suites, typechecks, formatting, lint, package
boundaries, and generated contract/documentation checks


### 21. Designed document shell and Body outlet

- [x] Add one structural **Content Block Body** outlet and keep legacy
Content Blocks without it compatible
- [x] Materialize and persist only `document.body` children through the
Body outlet while preserving the surrounding designed shell and
Templates
- [x] Expose the connected document frontmatter through a Content Block
`document` parameter for shell text and property bindings
- [x] Write direct frontmatter-bound text and properties back through
the same ordered per-Asset session without blocking canvas mutations
- [x] Resolve document `$ref` dependencies and Asset references
consistently in Builder and published output
- [x] Keep referenced-document values read-only on canvas and direct
editors to the referenced file
- [x] Isolate repeated Collection occurrences while retaining
occurrence-specific source, frontmatter, shell, and Body content
- [x] Render the same designed shell, frontmatter values, and Body
substitution in generated/published output
- [x] Upgrade the local blog article layout to bind title, excerpt,
dates, author data, and images from frontmatter and render MDX through
Body
- [x] Add fail/pass regressions for nested Body discovery, lifecycle,
permissions, repeated values, document references, and generated shell
rendering
- [x] Update authoritative Content Block documentation
- [x] Run affected tests and typechecks, formatting, lint, package
boundaries, generated API/docs checks, and fixture builds


### 22. Final nested references and frontmatter editing

- [x] Keep direct frontmatter-bound text and property controls editable
in Content mode without exposing or replacing their bindings
- [x] Keep referenced-document values read-only and preserve legacy
instance-property name normalization
- [x] Render direct frontmatter text bindings through the plain-text
editing path
- [x] Resolve nested legacy and structured Asset references from
referenced Markdown/MDX frontmatter
- [x] Preserve missing Asset references, emit structured
`ASSET_NOT_FOUND` diagnostics, and keep them out of the document graph
- [x] Add fail/pass regressions for the real Settings control and
root/nested missing Asset-reference cases
- [x] Fix exact Content Block selection after the Body node appears in
Navigator
- [x] Run full affected suites/typechecks, all six MDX Content Block E2E
scenarios, deterministic fixture builds, generated checks, lint,
formatting, package boundaries, and diff checks

### Before merge

- [x] Resolve #6261: Nested Asset references in referenced Markdown
frontmatter work consistently in Assets queries, previews, publication,
and static export


### 23. Explicit writable expression bindings

- [x] Add an explicit `readwrite` mode to expression bindings; existing
expressions remain read-only by default
- [x] Accept `readwrite` only for direct static member paths and
centralize their parsing and validation
- [x] Require the expression root to provide a writable data source;
initially support the connected Content Block document frontmatter
- [x] Update Content Block permissions, canvas text editing, and
property controls to require a valid writable binding
- [x] Persist direct frontmatter edits through the existing ordered
per-Asset session
- [x] Create new direct frontmatter bindings with `readwrite` mode and
migrate existing compatible bindings
- [x] Preserve writable mode through schemas, serialization, loading,
copy/paste, transfer, and export
- [x] Add fail/pass regressions for valid paths, derived expressions,
unsupported sources, permissions, persistence, and compatibility
- [x] Review authoritative `docs/` content and update it if the
persisted binding contract affects user-facing documentation
- [x] Reject unsafe frontmatter paths and reject `readwrite` bindings
outside a connected Content Block document
- [x] Preserve expression binding mode through the public HTTP client
and document the MCP authoring contract
- [x] Limit frontmatter value write-back to Content mode so Design mode
retains normal binding replacement/reset behavior
- [x] Keep bindings inside Templates and across nested Content Block
ownership boundaries read-only
- [x] Upgrade compatible legacy bindings even when the Content Block is
disconnected during migration
- [x] Run affected tests, typechecks, formatting, lint, package
boundaries, fixture checks, and `git diff --check`

Verification: full SDK (567), Project Build (2,277), and Builder (2,695)
suites pass, along with affected Expression, Migration, Template, HTTP
Client, and MCP suites; typechecks, lint, formatting, package
boundaries, and diff checks pass. Generated contracts are refreshed.
Fixture synchronization remains blocked before generation because the
development API rejects this local CLI version as incompatible. The
checked-in bundles pass the current Protocol suite and every fixture
build with the current bundle version.
 2026-08-29 20:15:20 +02:00
Oleg Isonen 7e7c14868a fix: Preserve read-only guides and publish targets (#6266) 
## Summary

- Route explicit read-only `meta.guide` requests to focused discovery
tools and exclude mutations, side-effecting session tools, broad design
context, and visual verification.
- Require explicit task scope, specialized workflow, authored-fragment
routing, and design-system reuse instead of inferring behavior from
brief prose.
- Persist the requested staging or production target in new SaaS
deployment records.
- Return the same target from publish list and job status, with
backward-compatible inference for historical records.

## Todo

- [x] Reproduce the read-only custom-code audit misrouting from #6262.
- [x] Add failing regressions and implement strict read-only routing.
- [x] Remove behavior-affecting prose inference from CLI/MCP guide
routing.
- [x] Reproduce the single-domain production target mismatch from #6263.
- [x] Add create-to-list/status regressions and persist the explicit
target.
- [x] Cover historical deployment records that predate the stored
target.
- [x] Verify production create → list → get-job through the staging
deployment.
- [x] Regenerate and verify project fixtures against the changed bundle
contract.
- [x] Review documentation impact and update the generated MCP manual.
- [x] Review the complete diff for correctness, compatibility,
duplication, and unnecessary complexity.

## Verification

- Real staging publish for #6263 — passed; create, list, and get-job
consistently reported `production` for a one-domain publish. The
supplied project has no custom domains, so the custom-domain variant was
not available. The job remained `pending` during this check.
- Real local CLI/MCP verification for #6262 — passed; no mutation or
side-effecting tools leaked.
- `pnpm --filter @webstudio-is/protocol test -- src/fixtures.test.ts` —
17 passed
- `pnpm -r fixtures:build` — passed for all eight fixtures
- `pnpm --filter @webstudio-is/project-build test` — 2,134 passed
- `pnpm --filter webstudio test` — 1,037 passed; the one payload-size
regression was fixed and its targeted test passed
- `pnpm --filter @webstudio-is/domain test` — 24 passed
- `pnpm --filter @webstudio-is/sdk test` — 519 passed
- Builder API router test — 30 passed
- Relevant package typechecks — passed
- `pnpm lint`
- `pnpm check:package-boundaries`
- `pnpm check:generated-api`
- `pnpm check:generated-docs`
- Regression tests were demonstrated failing against the previous
behavior and passing with the fixes.
- Agent evaluations were not run because they require separate explicit
approval.

Closes #6262
Closes #6263
 2026-08-29 18:54:05 +02:00
Oleg Isonen 471f504087 feat: Select unused assets before deleting (#6245) 
Closes #6243

## Summary

- Add checked-by-default controls for every asset in the **Delete unused
assets** dialog.
- Add a footer **Select all** control that reflects the complete
selection and toggles all assets.
- Delete only selected assets that remain unused and disable deletion
when the selection is empty.
- Keep the Assets documentation aligned with the selective deletion
workflow.

## Implementation

The dialog owns a set of selected asset IDs initialized from the
unused-assets result. Individual checkboxes update that set, while
**Select all** replaces it with either every currently unused ID or an
empty set.

Asset usage inputs remain subscribed while the dialog is open. The
deletion payload is intersected with the current unused-assets list, and
this cleanup path does not force deletion. If an asset becomes
referenced before the mutation executes, the runtime rejects its
deletion instead of removing a now-used asset.

## Todo

- [x] Add per-asset selection controls.
- [x] Add and position the **Select all** footer control.
- [x] Limit deletion to selected asset IDs that remain unused.
- [x] Protect against assets becoming referenced while the dialog is
open.
- [x] Add regression tests for individual selection, select-all
behavior, empty-selection protection, selective deletion, and live usage
changes.
- [x] Review documentation impact and update the Assets guides.
- [x] Verify formatting, lint, types, and Asset Manager/assets tests.

## Verification

- `pnpm --filter @webstudio-is/builder exec vitest run
app/builder/shared/asset-manager app/builder/shared/assets` — 26 files,
221 tests passed.
- `pnpm --filter @webstudio-is/builder typecheck` — passed.
- `pnpm exec oxlint --deny-warnings
apps/builder/app/builder/shared/assets/delete-assets.ts
apps/builder/app/builder/shared/asset-manager/delete-unused-assets.tsx
apps/builder/app/builder/shared/asset-manager/delete-unused-assets.test.tsx`
— passed.
- `pnpm exec oxfmt --check
apps/builder/app/builder/shared/assets/delete-assets.ts
apps/builder/app/builder/shared/asset-manager/delete-unused-assets.tsx
apps/builder/app/builder/shared/asset-manager/delete-unused-assets.test.tsx`
— passed.
- Both regression tests were confirmed failing before their respective
implementations and passing afterward.

## Compatibility and follow-ups

No data migrations, public API changes, fixture changes, or known
follow-ups.
 2026-08-26 12:14:06 +02:00
Oleg Isonen db2175f333 fix: transfer pasted assets across deployments (#6228) 
Closes #6227

## What changed

Cross-deployment copy/paste now transfers referenced assets into the
destination project before inserting instances, pages, templates, or
folders.

- Add optional source deployment metadata to the existing v0.1 clipboard
formats.
- Keep old v0.1 payloads readable and keep new payloads readable by
older builders, which ignore the additive field.
- Reuse the existing asset upload-by-URL flow for images, fonts, videos,
and files.
- Preserve source video dimensions through the existing upload route.
- Replace source asset IDs and font-family references with destination
values before runtime insertion.
- Include and remap page social images and marketplace thumbnails.
- Reject current clipboard payloads whose page metadata references
omitted assets.
- Leave the public `pageTransfer.insert` runtime input unchanged.
- Abort insertion with a paste error when an asset cannot be imported.

No clipboard version increase, custom network fetcher, address
blocklist, rollback framework, or global paste wrapper is introduced. If
a multi-asset import partially uploads before a later upload fails, the
completed uploads may remain unused.

## Checklist

- [x] Rebased onto `main`
- [x] Preserve the existing v0.1 clipboard format
- [x] Transfer and remap instance assets
- [x] Transfer and remap page, template, and folder assets
- [x] Preserve video metadata
- [x] Keep runtime page-transfer API compatible
- [x] Add regression coverage
- [x] Review documentation impact and update page-copy documentation
- [x] Run Builder tests and typecheck
- [x] Run project-build tests and typecheck
- [x] Run lint, package-boundary, and generated-API checks

## Verification

- Builder full suite: 219 files, 2,552 tests passed
- Project-build full suite: 78 files, 2,129 tests passed
- Final affected suite: 84 tests passed
- New regressions were confirmed failing before the fixes and passing
afterward
- Builder and project-build typechecks passed
- `pnpm lint` passed
- `pnpm check:package-boundaries` passed
- `pnpm check:generated-api` passed
- Fixture inputs and generated fixture bundles are unaffected
 2026-08-25 16:44:38 +02:00
Oleg Isonen 130d371759 perf: Skip resources in statically hidden subtrees (#6233) 
## Summary

- Skip read-time resources scoped to an instance with literal
`show=false` or any descendant of that instance.
- Use the same static reachability rule in the Builder client preloader
and CLI-generated published resource map.
- Keep dynamic `show` expressions on the existing loading path because
their result may depend on resource data.

## Data path and performance

In the Builder, the client-side resource request set now excludes
statically hidden subtrees before requests enter the preloader.

For generated sites, the CLI still scans project instances and props
during build, but removes statically hidden subtrees from the page build
data. Their read resources therefore never enter the generated
server-side resource map and no corresponding resource network request
runs during page rendering.

This removes request and server/network cost for literal-`false`
subtrees. It does not remove build-time project scanning, and resources
under dynamic conditions still load.

## Compatibility

- No project schema or persisted data changes.
- No migrations.
- Action-bound resources retain their existing explicit-trigger
behavior.
- Dynamic `show` expressions retain current behavior.

## Checklist

- [x] Add shared static-subtree reachability logic.
- [x] Apply it to Builder preview resource preloading.
- [x] Apply it to generated/published resource collection.
- [x] Add Builder and generated-output regressions for direct,
descendant, visible, and dynamic cases.
- [x] Demonstrate the new regressions fail against the previous behavior
and pass with the implementation.
- [x] Review authoritative `docs/` content; no documentation update is
required because the UI and authoring workflow are unchanged.
- [x] Review fixture inputs and outputs; no fixture contains this
resource/show combination, so fixture regeneration is not required.

## Verification

- `pnpm --filter @webstudio-is/sdk test`
- `pnpm --filter @webstudio-is/react-sdk test`
- `pnpm --filter @webstudio-is/builder exec vitest run
app/shared/nano-states/props.test.tsx`
- `pnpm --filter webstudio build:content-runtime`
- `pnpm --filter webstudio exec vitest run src/prebuild.test.ts`
- Typechecks for SDK, React SDK, Builder, and CLI
- Targeted `oxlint --deny-warnings`
- `git diff --check`

Agent evaluations were not run because they require separate explicit
approval.

Closes #6232
 2026-08-24 18:45:46 +02:00
Oleg Isonen 8e3f8e3234 fix: Avoid Builder E2E registry throttling (#6231) 
## Summary

- pull the pinned Supabase Postgres image from its official GHCR package
instead of anonymous Public ECR
- pin the multi-platform index digest shared by both registries

## Technical choice

The existing Public ECR and GHCR tags resolve to the same multi-platform
index digest, so changing registries preserves the amd64 and arm64 image
contents. GHCR public pulls require no long-lived CI credentials.

## Checklist

- [x] Verify Public ECR and GHCR resolve to the same image digest
- [x] Switch the Compose database image to the pinned GHCR digest
- [x] Validate shell syntax and rendered Compose configuration
- [x] Review documentation impact; no user-facing documentation is
affected
- [x] Review fixture impact; no fixture inputs or outputs are affected

## Verification

- `bash -n apps/builder/dev/backend.sh apps/builder/e2e/run-local.sh`
- `docker compose ... config --images`
- `docker buildx imagetools inspect` digest comparison for both
registries
- `git diff --check`

The full Builder E2E suite was not run locally because the change only
selects an identical image from a different registry.

Closes #6230
 2026-08-24 16:07:21 +02:00
Oleg Isonen df3601c26a fix: Capture preview failure diagnostics (#6206) 
## Summary

- Give invalid project bundles a stable `PROJECT_BUNDLE_INVALID` error
code and structured validation issues.
- Remember the latest failed MCP tool call and attach sanitized
diagnostics to automatic issue reports.
- Render the tool, error code, bundle contract, and invalid schema paths
in a dedicated issue section.

## Scope and issue assessment

This PR improves diagnostics; it does not change which asset shapes
`preview.start` accepts.

Issue #6198 came from CLI 0.290.0 and did not include the rejected
field, value, or bundle contract, so its exact root cause cannot be
established. Node 22 is not implicated by the available evidence.
Canonical video assets were added separately in #6189, which may explain
an old-CLI schema mismatch, but that relationship is an inference rather
than a confirmed reproduction.

Per project triage policy, this PR closes #6198 despite being
diagnostics-only. If the failure still exists on a current CLI, the
enriched report will provide enough evidence to reopen or file a new
actionable issue.

## Implementation

Bundle validation exposes structured paths such as `assets.0.type`. The
MCP host records the latest failure and injects it into CLI-owned
runtime metadata when `report-issue` is called.

The MCP registry supplies either a canonical known tool name or
`unknown`. Stable error codes are validated once by the CLI error-code
boundary. Detailed paths are included only for `PROJECT_BUNDLE_INVALID`,
capped at 30 issues, and omit received values, error messages, examples,
filenames, IDs, details, and stack traces. The bundle-contract field
remains optional so reports from older CLI versions still validate.

CLI-owned runtime metadata is not advertised as model-authored MCP
input, keeping the handshake compact and making ownership explicit.

## Checklist

- [x] Add structured bundle validation diagnostics.
- [x] Capture the latest MCP failure in issue-report runtime metadata.
- [x] Render deterministic diagnostics in GitHub issue bodies.
- [x] Add fail/pass regression coverage and privacy constraints.
- [x] Keep CLI-owned runtime fields out of the model-facing MCP schema.
- [x] Regenerate the server-only API contract.
- [x] Review documentation impact — no product documentation changes are
required; this only enriches internal anonymous issue reports.
- [ ] Run agent evaluations — not run because repository policy requires
explicit approval.

## Verification

- Real Node 22.22.1 stdio MCP run with an isolated malformed local
bundle returned `PROJECT_BUNDLE_INVALID`, path `assets.0.type`, code
`invalid_value`, and constraint `one_of:["image"]`.
- The following real `report-issue` call was accepted by the server and
deduplicated to existing issue #6198; it created no duplicate issue and
changed no linked project.
- Project-build: 2,115 tests passed.
- CLI: all 1,005 unaffected tests passed; one unrelated
process-supervisor timing test failed once and its exact two
owner-disconnect cases passed on immediate rerun.
- Project-build and CLI typechecks passed.
- Formatter and diff checks passed.
- All latest GitHub workflows passed, including both test/typecheck
matrices, visual regression, size and generated checks, packaged SSG,
and all six Builder E2E shards.

Closes #6198
 2026-08-20 11:58:17 +02:00
Oleg Isonen 0760d6cfec fix: upload videos from Builder and MCP (#6189) 
## Summary

Video uploads failed because file detection returned `video` while the
canonical Asset schema accepted only `font | image | file`. The mismatch
was introduced when video detection was added in #5539; the recent JSON
work only exposed the adjacent contract gap and did not cause it.

This makes `video` a first-class `Asset["type"]` across Builder, SDK,
storage formatting, project bundles, HTTP uploads, and MCP. There is no
separate upload-only type and no conversion from `video` to `file`.

## Implementation

- Add the canonical `video` Asset variant with width/height metadata.
- Make Builder UI reservations and optimistic assets preserve `video`.
- Make MCP and HTTP upload schemas derive their enum from the canonical
SDK `assetType`.
- Preserve video metadata through S3 and filesystem upload paths.
- Include canonical video assets in project/canvas bundles.
- Keep legacy video records without dimension metadata readable as
generic `file` assets.
- Fall back to a labeled file preview when the browser cannot decode a
video format such as AVI; browser-supported formats continue to render
video thumbnails.
- Regenerate runtime operation contracts and sync fixtures from the PR
deployment.

## Contract protection

- Assert every detected upload category parses as a canonical Asset
type.
- Exercise Builder reservations for every value in `assetType.options`.
- Assert MCP's upload enum exactly equals `assetType.options`.
- Exercise direct HTTP video upload metadata and URL construction.
- Cover canonical video formatting, legacy fallback, preview assets, and
canvas serialization.
- Verify a video decode error replaces the blank media element with a
visible format-labeled fallback.
- Store canonical WebM and MP4 records with dimensions in the
`webstudio-features` fixture and verify generated runtime metadata.

## Checklist

- [x] Make video a canonical Asset type
- [x] Verify Builder UI upload flow
- [x] Verify MCP and direct HTTP upload contracts
- [x] Add fail/pass regression coverage for the broken paths
- [x] Add an Assets panel fallback for browser-unsupported video formats
- [x] Regenerate runtime contracts
- [x] Sync and build fixtures against the PR deployment
- [x] Run affected package tests, typechecks, lint, package-boundary,
generated API, and generated docs checks
- [x] Review documentation impact — existing Assets documentation
already lists supported video formats and upload behavior; no
user-facing documentation change is required

## Verification

- Builder: 2,543 tests passed
- Builder asset thumbnail regression: 14 tests passed
- Project build/MCP runtime: 2,111 tests passed
- CLI: 999 tests passed
- SDK: 516 tests passed
- Asset uploader: 199 tests passed
- Protocol: 92 tests passed
- HTTP client: 38 tests passed
- Typechecks passed for Builder, SDK, asset uploader, project build,
protocol, HTTP client, and CLI
- Lint and package-boundary checks passed
- Generated API and documentation checks passed
- Fixture sync completed against
`fix-video-up-be4d8a0.development.webstudio.is`
- All fixture builds passed after synchronization

Closes #6186
 2026-08-19 00:21:11 +02:00
Oleg Isonen 5a57234c75 revert: Store Content Block content in MDX assets 
Revert the accidental merge of #6125 so the implementation can continue in draft PR #6181.
 2026-08-18 13:23:32 +01:00
Oleg Isonen 5ec3547d36 fix: Support JSON content files in Builder (#6168) 
## Summary

- make the complete asset filename editable in Asset settings
- convert existing editable text assets when their extension changes
while preserving the asset ID
- accept every JSON root type and normalize JSON-compatible syntax into
formatted strict JSON
- expose the same optional extension change through MCP/API
`update-asset-content`
- initialize new JSON files and empty text-to-JSON conversions with a
valid empty object
- distinguish browser URL imports from JSON file uploads so JSON objects
with a root `url` field remain file content
- update the Assets and Content Engine documentation

## Root cause

Asset settings rendered the extension as a fixed suffix outside the name
input. The metadata rename mutation could only change the display
basename, while content revisions always inherited the current storage
extension. As a result, changing `.md` to `.json` was impossible in the
UI and unavailable through the shared MCP/API revision command.

Asset settings now edits the complete filename. Ordinary renames keep
using the metadata mutation. A text-extension change loads the current
content, validates it for the target format, and writes a new immutable
revision through the existing conflict-safe content endpoint. The
existing Acorn-backed `parseJsonExpression` utility accepts
JSON-compatible syntax without evaluating code and serializes it as
strict JSON. Objects, arrays, strings, numbers, booleans, and `null` are
all accepted.

## Checklist

- [x] Create JSON files by entering a `.json` filename
- [x] Edit the complete filename in Asset settings
- [x] Convert editable text-file extensions while preserving the asset
ID
- [x] Expose extension conversion through `update-asset-content`
- [x] Create JSON files and convert empty text files with a valid empty
object
- [x] Normalize safe JSON-compatible syntax to strict JSON
- [x] Accept objects, arrays, and scalar JSON roots
- [x] Reject executable or incomplete content
- [x] Preserve browser-uploaded JSON object shapes
- [x] Review and update authoritative Builder documentation
- [x] Add focused UI, revision, HTTP-client, CLI/MCP, protocol, and
upload regression tests
- [x] Run affected typechecks, lint, documentation checks, and the
Builder production build
- [ ] Manually verify create, edit, rename `.md` to `.json`, reload
persistence, and Content Engine query in a browser
- [ ] Run the complete local agent evaluation suite if approved

## Verification

- 217 focused tests passed across Builder, asset revisions, HTTP client,
CLI/MCP, and protocol
- Builder, asset-uploader, HTTP-client, protocol, CLI, and project-build
typechecks passed
- Builder production build passed
- MCP and Content Engine documentation checks passed
- `oxlint` on changed TypeScript files passed
- `git diff --check` passed
- Regression tests were demonstrated failing before the implementation
and passing afterward

A disposable E2E database and PostgREST stack bootstrapped successfully
in the earlier real-path test, but UI automation remained blocked
because no controllable browser was available. Fixture inputs and
generated fixture outputs are not affected. Agent evaluations were not
run because approval is required and was not requested.

Closes #6167
 2026-08-18 11:39:01 +02:00
Oleg Isonen 86b6310fff feat: Store Content Block content in MDX assets 2026-08-17 15:22:20 +01:00
Oleg Isonen ab5717c9cf feat: Support CSS properties of every status (#6166) 
## Summary

- Make every CSS property in the current MDN dataset available in the
Advanced style workflow, regardless of standards status.
- Mark experimental, nonstandard, and obsolete properties with a yellow
warning icon, status-specific tooltip, and MDN link.
- Add complete CSS anchor-positioning property and value support.
- Update `mdn-data` from 2.28.0 to 2.33.0 and `css-tree` from 3.1.0 to
3.2.1.

## Root cause

Webstudio generated its supported property list from MDN data but
filtered by status and then manually allowed a few experimental and
nonstandard entries. A property could therefore exist in the installed
MDN package without being exposed by the Builder.

Runtime validation had a separate grammar gap. CSS-tree 3.2.1 now
includes the anchor functions and alignment values, but its bundled data
predates the current `position-anchor: normal | match-parent` additions.

## Implementation

- Remove status-based property filtering and the nonstandard-property
allowlist.
- Generate one property-to-status map for 486 standard, 40 experimental,
119 nonstandard, and 15 obsolete properties.
- Show status-specific browser-compatibility guidance for every
non-standard status. Active declarations keep the warning after the
value with 3px left spacing.
- Infer MDN links from existing property metadata and fall back to MDN
search for entries without a direct URL.
- Preserve known vendor prefixes from the generated property inventory
instead of maintaining a prefix allowlist.
- Generate WebKit text fill/stroke longhands directly from MDN instead
of preserving them through parser exceptions.
- Preserve unsupported initial-value shapes as unparsed CSS rather than
excluding their properties.
- Use CSS-tree 3.2.1's native anchor-function and alignment grammars.
Keep one narrow extension for the current `position-anchor` keywords
missing from its bundled MDN data.
- Generate recursively flattened shorthand longhands and initial values
from MDN.
- Expand syntax-defined shorthands generically from CSS-tree's matched
property captures, including function-valued components.
- Decline generic expansion when CSS-tree and MDN disagree about the
longhand set or a longhand repeats.
- Update the font grammar reference and CSS math validation for CSS-tree
3.2.1 compatibility.
- Regenerate CSS metadata and property-variable fixtures.

## Checklist

- [x] Include MDN properties regardless of status.
- [x] Generate status metadata for standard, experimental, nonstandard,
and obsolete properties.
- [x] Show warning icons in Add styles results and active declarations.
- [x] Add status-specific consequences and MDN links.
- [x] Preserve generated vendor-prefixed properties during CSS import.
- [x] Generate WebKit text fill/stroke longhands without parser
exceptions.
- [x] Support `anchor()`, `anchor-size()`, and `anchor-center`.
- [x] Support current `position-anchor` keywords.
- [x] Expand syntax-compatible shorthands through generic MDN metadata
and CSS-tree captures.
- [x] Cover ordered optional, unordered, slash-separated,
function-valued, omitted, CSS-wide, variable, invalid, and
incompatible-data cases.
- [x] Verify new status, warning, prefix, and shorthand tests fail
without their implementations and pass when restored.
- [x] Update to CSS-tree 3.2.1.
- [x] Regenerate CSS metadata and property-variable fixtures.
- [x] Review documentation impact; no authoritative Builder
documentation lists individual supported CSS properties, so no update is
needed.
- [x] Review fixture impact; 668 fixtures and 2,030 valid cases were
regenerated with zero unsupported.

## Verification

- `pnpm --filter @webstudio-is/css-data test` — 4,618 passed, 1 skipped.
- Builder CSS editor and property-status tests — 25 passed.
- CSS data, CSS engine, Project Build, and Builder typechecks passed.
- Lint, formatting, repeated regeneration, and diff checks passed.

## Known limitation

This change covers CSS declarations and values. It does not add a
dedicated Builder editor for the `@position-try` at-rule.

Closes #6162
 2026-08-17 14:54:11 +02:00
Oleg Isonen a836f29a69 fix: Explain asset count limits (#6149) 
## Summary

Return a specific, safe error when a project reaches its plan's
asset-count limit instead of reporting an internal Assets API failure.

## Root cause

The uploader enforced `maxAssetsPerProject` by throwing a plain `Error`.
The REST boundary correctly treated unknown errors as internal failures,
returned HTTP 500, and redacted the useful limit explanation. The
Builder then displayed that generic response.

## Technical choices

- Represent the expected quota condition with
`AssetUploadCountLimitError`.
- Map that domain error to HTTP 409 while preserving its existing
message.
- Keep the generic HTTP 500 response and server logging for unexpected
failures.
- Reuse the Builder's existing API-error handling; it already displays
the returned `errors` value in the upload toast.

## Checklist

- [x] Reproduce the generic uploader and REST error behavior
- [x] Add regressions and verify they fail before the behavior change
- [x] Throw a typed error when the asset-count limit is reached
- [x] Return the safe limit explanation through the Assets API
- [x] Verify unexpected internal errors remain redacted
- [x] Review documentation impact — no documentation change is needed
for this error-message correction
- [x] Review fixture impact — no fixture inputs or generated outputs are
affected
- [x] Review the final diff for unrelated changes and duplicated logic

## Verification

- `pnpm --filter @webstudio-is/asset-uploader test` — 196 tests passed
- Asset uploader typecheck passed
- Targeted Builder Assets API and upload client tests — 37 tests passed
- Builder typecheck passed
- `pnpm lint`

Closes #6148
 2026-08-14 23:32:07 +02:00
Oleg Isonen cb51d477e1 fix: Refresh resource diagnostics metadata (#6113) 
## Summary

Follow up on #6095 so explicit Diagnostics refreshes update both
diagnostic and performance metadata, stale refreshes are cancelled
during invalidation, and Assets batch measurements are labeled
consistently in the Builder.

Ref #6092

## Technical choices

- Cache response diagnostics and performance metadata through one shared
path.
- Attach an AbortController to each explicit diagnostics request and
reject stale results after invalidation.
- Accept additive performance fields from newer servers.
- Clarify that Assets timings and counters cover the combined batch.
- Reuse the measured view-authorization path across repository entry
points.

## Implementation checklist

- [x] Refresh performance metadata with explicit diagnostics.
- [x] Cancel and ignore stale diagnostics requests.
- [x] Keep performance parsing forward-compatible with additive fields.
- [x] Clarify Assets batch labels and descriptions.
- [x] Add regression coverage for refresh, invalidation, and parsing
behavior.
- [x] Review documentation impact — authoritative `docs/` does not cover
this internal Diagnostics tooling, so no update is required.
- [ ] Run affected Builder and asset-uploader typechecks.
- [ ] Confirm GitHub checks pass.

## Verification

- Builder diagnostics/resource tests: 27 passed.
- Asset repository tests: 39 passed.
- `git diff --check origin/main...HEAD` — passed.
 2026-08-12 11:46:58 +02:00
Oleg Isonen db2d092268 ci: replace Lost Pixel with self-hosted visual regression (#6086) 
## Summary

Replace the defunct Lost Pixel service and the interim Chromatic
workflow with a fast, self-hosted visual comparison pipeline.

The workflow indexes the existing CSF stories without building
Storybook, builds and serves a lightweight browser app with Vite,
renders the merge base and current revision through Webstudio's shared
browser tooling, and compares screenshots with Webstudio's pixel-diff
and OCR engine. It uses GitHub Actions cache and artifacts only—no R2
bucket or external visual-testing service.

Closes #6064

## Implementation

- [x] Remove the Lost Pixel workflow, configuration, acknowledgement,
and global two-second wait.
- [x] Remove the interim Chromatic workflow and settings.
- [x] Extract source-agnostic capture, diff, OCR, audit, cache, and
report tooling into `@webstudio-is/vision`; rename the
Webstudio-specific preview boundary to
`@webstudio-is/project-build/preview`.
- [x] Expose the reusable browser capture session through the shared
vision boundary.
- [x] Make OCR and diff-artifact generation conditional on detected
pixel changes.
- [x] Render existing CSF stories through a typed, lightweight browser
harness built with Vite.
- [x] Replace the embedded HTML and handwritten HTTP/MIME server with a
real HTML entry, Vite production build, and Vite preview server; isolate
declarative build configuration in `story-build.ts`.
- [x] Apply Storybook preview annotations, decorators, global CSS,
backgrounds, and story metadata.
- [x] Store story scopes as shared data and load each revision's own
configuration.
- [x] Discover changed, added, and removed stories from both revisions.
- [x] Populate private story sources symmetrically in the baseline
worktree.
- [x] Isolate every story with a fresh React root and deterministic
time, randomness, intervals, focus, animation, and layout readiness.
- [x] Preserve story-owned visual options for masking, settling, and
interval control.
- [x] Reuse Vite preview servers, one Chromium process, grouped browser
pages, production bundles, and a bounded CPU worker pool for image
comparison.
- [x] Recheck initial pixel differences with serial captures, then
recapture persistent baseline/current pairs on the same browser page so
renderer variance cannot create false failures.
- [x] Report individual story render failures without discarding
successful captures.
- [x] Enforce a 0.1 per-pixel threshold and 0.001% total mismatch
tolerance.
- [x] Run OCR only for changed comparisons and include text-change
evidence.
- [x] Move the generic report model, writer, static viewer, and opener
into the shared Vision package for reuse by CI and future CLI
comparisons.
- [x] Bundle report HTML, styling, and browser behavior as static Vision
assets without runtime source-file lookups.
- [x] Copy external screenshot inputs into self-contained report
artifacts for CLI reuse.
- [x] Handle cached removed-only comparisons without rendering the
removed story on the current server.
- [x] Support focused local reproduction and opening the report with one
command.
- [x] Pin CI to the lockfile's Playwright Chromium, cache complete
revision screenshots with matching restore/save paths and automatically
derived runtime invalidation, and render both revisions on a miss.
- [x] Keep cache archives bounded to one revision instead of
accumulating the cache history.
- [x] Run the complete visual suite in one CI job and upload one report
as a 14-day artifact.
- [x] Fail PR checks on visual differences, empty runs, missing
screenshots, missing or empty report artifacts, and infrastructure
failures.
- [x] Rerun fork checks when `safe-to-deploy` is added.
- [x] Allow reviewed visual-only differences through
`visual-change-approved` and invalidate approval after new commits.
- [x] Keep capture, pixel diff, OCR, and report generation
source-agnostic for future draft-versus-published project comparisons.
- [x] Use Vite for the complete story HTML, module, asset,
production-build, and preview-server lifecycle; inline the selected
stories and skip ineffective tree-shaking to keep the cached run under
one minute.
- [x] Review Builder product documentation impact: no user-facing
Builder documentation is affected; contributor usage is documented in
`scripts/visual-regression/README.md`.
- [x] Review fixture impact: no fixture inputs or generated fixture
outputs are affected.

## Verification

- [x] Project-build Vision tests, including the reusable report test,
and entrypoint contract tests pass.
- [x] CLI screenshot, preview, and browser-session tests pass.
- [x] Project-build, CLI, Builder, design-system, and visual-runner
typechecks pass.
- [x] Visual runner tests pass (10), and the complete Vision package
passes (79), including demonstrated fail-before/pass-after coverage.
- [x] Full Storybook production build passes.
- [x] Full local Vite run completes all 238 stories in 69.9s cold and
50.3s with a cached baseline plus two same-page serial verifications,
with zero differences; identical-image comparison remains sub-second.
- [x] The report writer passes a bundled-output smoke test and
materializes external screenshot assets.
- [x] The generated report validates all 238 entries and referenced
images.
- [x] Hosted cached-baseline comparison completes all 238 stories in
52.9s with zero differences; the cache-miss fallback completes in
101.3s.
- [x] Cache-miss fallback renders both Git revisions on demand.
- [x] Visual differences produce baseline, current, diff, OCR, JSON, and
HTML evidence and fail without approval.
- [x] Hosted single-job visual workflow restores both the pinned browser
and baseline cache, validates all 238 public and private stories with
zero differences, and uploads the report.
- [x] The final Vite production build renders all 238 stories without
dependency invalidation or duplicate React runtimes, and restores
`NODE_ENV` so cached serial fallback installs the baseline's dev
dependencies.
- [x] Root lint, package-boundary validation, and relevant formatting
checks pass.
- [x] Workflow YAML parses successfully.
- [x] `git diff --check`, staged-diff review, and final worktree review
pass.

## Usage

Run the complete local comparison and open the visual report:

```sh
pnpm visual-regression:ui
```

Focus one component while iterating:

```sh
pnpm visual-regression:ui -- --grep "Design system/Button"
```

Reopen the latest report without rerunning:

```sh
pnpm visual-regression:report
```

The CI report is available as the `visual-regression-report` artifact.

## Tradeoffs

- Baseline screenshots are a disposable acceleration cache, not source
of truth. A missing or invalid cache renders the merge base directly
from Git.
- GitHub artifact storage provides downloadable one-click reports;
permanently hosted browser links would require another hosting surface.
- OCR runs after pixel triage because invoking Tesseract for every
unchanged story would make the common path unnecessarily slow.
- Vite production builds add Rollup's module graph and HTML/asset
pipeline. The runner emits one bundle because every selected story is
rendered, and skips tree-shaking because it removed little while
increasing build time.
- The verified local path is 50.3s cached and 69.9s cold. The hosted
path is 52.9s cached and 101.3s cold. A new browser/runtime cache must
render its baseline once; the pinned browser keeps later runners on the
reusable path.
- The complete suite stays in one Actions job; browser-page concurrency
is internal and bounded.
 2026-08-12 10:55:52 +02:00
Oleg Isonen 11d76eb43d perf: Diagnose and reduce Assets resource latency (#6095) 
## Summary

Adds Builder-only resource performance metrics to grouped Diagnostics
tables, removes duplicate object-store reads from the Assets query path,
scopes compiled-artifact reuse to one repository request, and compiles
unpinned Builder previews from their concrete query values instead of
hydrating the complete saved dynamic deployment plan. Nothing is
precomputed or retained in a warm serverless process after the request
finishes.

Ref #6092

## Technical choices

- Keep `__performance__` as internal response metadata and strip it
before resource values enter expression-visible caches.
- Record Builder round-trip, server duration, response size, Assets
phases, document work, completed storage reads, and cumulative
compiler/document-graph storage-read durations.
- Present Diagnostics as collapsible sections ordered Database and
sizes, Timing, Query work, then detailed database/result inspectors.
Open only the first available section by default. Every metric keeps an
explanatory tooltip, with info icons revealed only when the label cell
is hovered or keyboard-focused.
- Add a copy action to every section header. It copies that section's
raw structured data as formatted JSON without opening a collapsed
section.
- Keep Index preparation as an inclusive parent duration and explain
that relationship in Diagnostics.
- Compile ordinary unpinned Builder preview requests from a union of
their concrete query values. A detail route therefore hydrates only
documents that can satisfy that request.
- Continue using the saved artifact-producing plan for revision-pinned
requests.
- Keep publication and prepublication preparation unchanged.
- Load only `props`, `dataSources`, and `resources` when constructing
the single-query Builder preview plan, matching the already-narrow batch
path instead of downloading and parsing the complete dev Build row.
- Build the deployment-wide saved plan only when detailed Diagnostics
requests its Published database statistics. Report that work as the
separate inclusive `Published diagnostics` timing and exclude its child
phases and fetch counters from the normal query measurements.
- Reuse successfully read content bytes across compiler preparation,
document graph construction, and runtime resolution within one query or
batch. Graph-fetched bytes are size-validated before entering the
revision-keyed request map, and graph additions stop when the request
cache reaches the 2 MiB hydrated-content limit.
- Before repairing canonical metadata, load candidate metadata once and
prove its revision, base document, and prepared metadata tier match the
open source snapshot. Skip synchronization when it is already current;
retain recovery and repair for stale, missing, insufficient-tier, or
inconsistent rows.
- Key request-local bytes by content reference and revision so
source-change retries cannot reuse stale data.
- Instantiate compiled-artifact reuse per `PostgresAssetRepository`;
repeated preparations inside one request can reuse or coalesce work, but
later requests and other serverless instances cannot.
- Remove the module-level 32-entry compiled-artifact cache introduced in
#5946.
- Introduce no precomputed, persisted, process-local, or cross-request
content cache.

## Findings and measurements

The supplied production post resource was bimodal before removing the
process cache:

- Warm process: approximately 1.0–1.5 seconds.
- Cold process: approximately 5.7–6.7 seconds.
- Ordinary response: approximately 4.6 kB.

The earlier development capture was a process-cache hit: 542.6 ms server
time and 932.5 ms Builder round trip. A later hit measured 491.5 ms
server time and 739.1 ms round trip. Those results demonstrated the old
module cache's nondeterministic serverless behavior rather than
cold-path performance.

After request-scoped reuse replaced the process cache, the supplied cold
capture measured 3,848.5 ms server time. Index preparation was 3,259.7
ms. The saved dynamic plan caused 25 compiler content fetches and 301 kB
of compiler input even though the concrete post query resolved only two
documents. The published database contained 28 documents and used 45.4
kB bounded / 512 kB unbounded.

Tracing compilation also exposed duplicate request-time I/O: selected
Markdown was fetched for compiler input, document graph construction,
and runtime document resolution. Request-local byte reuse changes that
representative path from three object-store reads to one.

The remaining dominant issue was therefore query breadth, not response
serialization or runtime result resolution. The Builder was compiling
the deployment-wide dynamic plan to answer a concrete post request. The
new preview path compiles the concrete values; regression coverage
demonstrates that a saved two-post dynamic plan reads only `post-a` when
previewing `post-a`.

The next cold capture confirmed that change: Builder round trip fell
from 4,391.4 ms to 2,245.2 ms, server duration from 3,848.5 ms to
1,601.8 ms, and index preparation from 3,259.7 ms to 1,044.9 ms.
Compiler content fetches fell from 25 to 1 and document-graph content
fetches from 3 to 1. Canonical metadata remained 239.2 ms because an
unchanged request still performed a repair synchronization round and
then loaded the same stored metadata again. The new fast path reduces
that normal case to one recovery-safe candidate metadata read.

A subsequent capture confirmed the canonical metadata fast path: server
duration fell from 1,601.8 ms to 1,401.0 ms, index preparation from
1,044.9 ms to 684.8 ms, and canonical metadata from 239.2 ms to 74.7 ms.
The post query remained narrow at one compiler fetch and one
document-graph fetch. Builder round trip rose from 2,245.2 ms to 2,411.1
ms while the server improved, so that difference is outside the measured
server work and is consistent with
browser/network/development-environment variance.

At that point `buildPlan` was the largest remaining isolated phase at
242.8 ms. Single-query preview was still loading every dev Build column
and parsing pages, styles, instances, deployment data, and other fields
that are not inputs to the Assets compilation plan. The narrow loader
now requests and parses only the three content-engine fields used by
plan construction.

The next capture confirmed that optimization: build plan fell from 242.8
ms to 99.0 ms, server duration from 1,401.0 ms to 1,056.8 ms, and the
concrete post query remained at one compiler fetch and one
document-graph fetch. Index preparation stayed effectively flat at 679.7
ms, making compiler entries (142.3 ms), source snapshot (119.2 ms),
source validation (119.1 ms), and document graph (114.0 ms) the next
candidates. The new child timings separate cumulative storage waiting
from the remaining compiler/document-graph work before choosing another
behavioral optimization.

The storage timing capture then showed that I/O dominated the remaining
compiler and graph phases: compiler storage was 114.6 ms of 125.9 ms,
while document-graph storage was 114.0 ms of 147.6 ms. The graph path
consulted compiler-cached bytes but did not retain newly fetched graph
documents, so runtime reference resolution fetched the same two-document
chain again. Retaining validated graph bytes in the same request reduces
the regression fixture from four physical object-store reads to two
without changing its resolved result.

Final three-run measurements after same-request graph-byte reuse were
stable: median Builder round trip was 926.2 ms, median server duration
was 681.9 ms, median index preparation was 407.9 ms, and median document
resolution was 3.7 ms. Compared with the uncached 3,848.5 ms server /
3,259.7 ms index baseline, that is approximately 82% less server time
and 87% less index-preparation time for this post query.

A final DRY/correctness review centralized request-local byte-cache
keying and its 2 MiB accounting, removed repeated Diagnostics size rows
and SDK metadata assembly, typed phase keys, and corrected response-size
measurement so internal performance metadata is excluded as the UI
promises.

The module cache was also only bounded for completed entries. Many
simultaneous unique-project compilations in one process could exceed 32
pending entries until they settled. Scoping reuse to one repository
request removes cross-user retention and tenant cache competition while
preserving duplicate-work elimination inside the request.

## Todo

- [x] Add generic resource timing and response-size metrics.
- [x] Add Assets phase, document graph, source validation, storage-read
count/byte metrics, and compiler/document-graph storage-read durations.
- [x] Render ordered collapsible Diagnostics sections with only the
first open by default, per-metric explanations, interaction-only info
icons, and per-section JSON copy actions.
- [x] Reuse compiler and document-graph content bytes through runtime
resolution within the current request, without precomputation.
- [x] Remove cross-request process caching and retain request-scoped
compilation reuse.
- [x] Compile unpinned previews from concrete query values.
- [x] Separate deployment-wide diagnostic preparation from normal query
phase counters.
- [x] Skip redundant canonical metadata synchronization when stored
candidates already match the source snapshot and requested tier.
- [x] Narrow single-query Build reads to the three fields required by
Assets plan construction.
- [x] Expose cumulative compiler and document-graph storage-read timing
as explained child rows.
- [x] Add regression coverage, including red/green verification.
- [x] Review the final diff for correctness, duplication, and
unnecessary complexity.
- [x] Verify concrete-query compilation against the supplied post
resource: compiler fetches fell from 25 to 1.
- [x] Remeasure the canonical metadata fast path: canonical metadata
fell from 239.2 ms to 74.7 ms and index preparation from 1,044.9 ms to
684.8 ms.
- [x] Review documentation impact: no authoritative `docs/` page covers
this internal Diagnostics tooling, so no documentation update is
required.
- [x] Run focused tests, affected package typechecks, lint, formatting,
and the fixture pipeline.

## Verification

- Content engine tests: 23 passed.
- Asset uploader tests: 195 passed, including repository, canonical
synchronization, request-cache, and storage-timing coverage.
- Builder focused Diagnostics tests: 10 passed in the latest
storage-timing pass; 15 passed in the preceding final pass and 35 across
the earlier instrumentation scope. The collapsible view test verifies
section order, default state, and deferred content interaction.
- Project tests: 84 passed, including 5 focused preview tests. The
focused regression failed against the full-Build loader and passed after
switching the single-query path to the narrow content-engine loader.
- SDK resource loader tests: 24 passed.
- Current canonical metadata regressions failed under the old
always-synchronize expectations and pass with one metadata load and no
synchronization; stale metadata still exercises and passes the repair
fallback.
- The Timing clipboard test failed with an empty payload and passed with
the complete raw duration/phase JSON; copying a collapsed section is
also verified not to open it.
- The collapsible default-state test failed when Timing was temporarily
opened alongside Database and sizes, then passed after restoring the
first-section-only behavior.
- The concrete-preview regression failed before the optimization by
reading both `post-a` and `post-b`, then passed by reading only
`post-a`.
- A repository-reuse regression failed without the request cache and
passed when reuse became the repository default.
- The content-read regression failed before the original compiler reuse
optimization with three storage calls and passed afterward with one.
- The two-document graph regression failed with four physical storage
calls and passed with two after graph reads populated the request-local
runtime cache.
- The response-size regression failed at 127 bytes when internal
performance metadata was included and passed at the actual 66-byte
payload after separating that metadata.
- Builder, content engine, asset uploader, project, and SDK typechecks
passed.
- Oxfmt, oxlint, and `git diff --check` passed.
- The full fixture link/sync/build pipeline passed with no tracked
fixture changes after each runtime change.

## Known limitations

- Detailed Diagnostics intentionally prepares both the concrete query
database and the deployment-wide Published database. `Published
diagnostics` reports the latter separately; normal resource loads do not
perform it.
- Builder round-trip and server duration cover the complete resource
batch and therefore include Published diagnostics when the detailed view
requests it.
- Phase durations are aggregated when a batch contains multiple Assets
queries.
- Compiler and document-graph storage-read timings sum completed fetch
durations. They exclude request-local byte-cache hits and can exceed
wall-clock parent phases when reads run concurrently.
- Content-read counters report completed normal-query reads;
deployment-wide diagnostic reads are excluded, and failed storage reads
are represented by the enclosing phase timing and error.
- Request-local byte reuse retains selected content only until the
current query or batch finishes. Document-graph additions are capped by
the existing 2 MiB hydrated-content limit; content beyond the cap is
still streamed and can be fetched again at runtime.
- Source snapshot and final source validation remain separate reads. The
metadata fast path removes redundant repair work without weakening that
concurrent-change check.

## Measurement gate

PR deployments use the development backend. The supplied production
project returns 500 on both the branch deployment and
`main.development.webstudio.is`, while the production host returns 200.
Final phase timings for the exact project require its corresponding
development project/session or an explicitly authorized
production-backed deployment. An unrelated fixture will not be used to
claim the production improvement.
 2026-08-11 12:31:23 +02:00
Oleg Isonen db3b11970f fix: Optimize images in Markdown Embed (#6089) 
## Summary

Markdown Embed now renders image tags with the same optimized attributes
and canvas behavior as the SDK Image component.

The implementation shares the complete attribute-resolution path without
rendering React inside the Markdown pipeline:

- `@webstudio-is/image` exposes one framework-independent
`getImageAttributes` function for optimization, placeholder fallback,
loading, decoding, dimensions, alternative text, and responsive sources.
- React-only element props such as `className`, `style`, handlers,
context, and refs stay in React callers.
- The SDK Image component and Markdown renderer both use
`getSdkImageProps` for canvas-specific loading, decoding, asset
identity, and incomplete-upload handling.
- Markdown retains only the necessary HTML attribute-name conversion,
sanitization, and Safari-safe `sizes` / `srcset` / `src` ordering.
- The redundant low-level React Image primitive was removed. Its callers
now render native `img` elements, and the image package no longer has
React runtime or peer dependencies.

## Compatibility and tradeoffs

- The SDK Image component remains the public React component because it
owns React context and ref forwarding.
- The removed low-level `@webstudio-is/image` Image export is replaced
by the pure attribute function for internal callers.
- No data migration or schema change is required.
- Loader output is sanitized after transformation, while unsafe input
URLs are sanitized before reaching the loader.

## Todo

- [x] Share image attribute resolution between native, SDK, and Markdown
images.
- [x] Preserve SDK canvas behavior for Markdown images.
- [x] Preserve safe URL sanitization and Safari-safe responsive
attribute ordering.
- [x] Remove the redundant low-level React image primitive and migrate
its callers.
- [x] Remove the overlapping `getImageProps` abstraction.
- [x] Add regression coverage comparing Markdown, SDK Image, and native
attribute output.
- [x] Review documentation impact and update the Markdown Embed
documentation.
- [x] Run targeted tests, typechecks, builds, lint, formatting,
package-boundary checks, and fixture generation.

## Verification

- `pnpm --filter @webstudio-is/image test`
- `pnpm --filter @webstudio-is/image typecheck`
- `pnpm --filter @webstudio-is/image build`
- `pnpm --filter @webstudio-is/image dts`
- `pnpm --filter @webstudio-is/sdk-components-react test`
- `pnpm --filter @webstudio-is/sdk-components-react typecheck`
- `pnpm --filter @webstudio-is/sdk-components-react build`
- `pnpm --filter @webstudio-is/sdk-components-react dts`
- `pnpm --filter @webstudio-is/builder typecheck`
- `pnpm --filter @webstudio-is/design-system typecheck`
- Targeted Oxlint and Oxfmt checks
- `pnpm check:package-boundaries`
- `pnpm fixtures` (no fixture diff)

Ref #6014
 2026-08-10 18:21:45 +02:00
Oleg Isonen 41c8283a70 fix: Allow removal of invalid audited selectors (#6074) 
## Summary

Allow all targeted style-deletion paths to remove an invalid state
selector that already exists, including Builder UI deletion of local and
token declarations and CLI/MCP design-token remediation. Creation and
update validation remain strict.

## Root cause

The style audit reads legacy declarations directly and can report
selectors that the current selector validator rejects. Deletion reused
the creation/update validator, so the system could identify an invalid
stored declaration but could not represent its coordinates for targeted
removal. The first fix addressed only design-token deletion, leaving
Builder UI and the generic style APIs inconsistent.

## Implementation

- [x] Give style deletions a shared deletion-only state-selector
contract.
- [x] Accept an invalid selector only when the exact target, breakpoint,
property, and state already exist.
- [x] Reject arbitrary invalid selectors that do not identify an
existing declaration.
- [x] Reuse one matcher for local, selected/token, and design-token
deletions.
- [x] Keep creation and update selector validation strict.
- [x] Add Builder UI regression coverage for token and local declaration
deletion.
- [x] Add an audit regression that deletes the reported design-token
selector, applies the patch, and confirms the warning is cleared.
- [x] Regenerate the public runtime operation contract.
- [x] Complete the DRY review: deletion matching, validation, key
deduplication, and stored-style scanning are centralized.
- [x] Review documentation impact: no authored documentation changes are
required; generated operation contracts describe the behavior.
- [x] Review fixture impact: no fixture inputs or generated fixture
bundles are affected.
- [x] Run relevant tests, typechecks, lint, formatting,
package-boundary, generated-API, and generated-documentation checks.
- [ ] Agent evaluations — not run because repository policy requires
separate explicit approval.

## Verification

- The two Builder UI regression tests failed with `Operation input is
invalid` before the shared fix and passed afterward.
- Project-build suite: 82 files, 2,097 tests passed.
- CLI suite: 60 files, 968 tests passed.
- Builder style-data target: 7 tests passed.
- Audit and styles targets: 195 tests passed.
- Project-build, Builder, and CLI typechecks passed.
- Full repository lint passed.
- Package-boundary, generated-API, and generated-documentation checks
passed.
- Formatting and `git diff --check` passed.

Closes #6062
 2026-08-09 01:46:42 +02:00
Oleg Isonen d5b3f3ab80 fix: Improve MCP authoring reliability (#6056) 
## Summary

This consolidates the reported MCP reliability fixes into one delivery.

- validate all inserted fragment expressions with the shared runtime
rules and reject text/expression children when component content models
require elements
- return an actionable error for unsupported existing-resource
references in JSX
- preserve siblings rendered before expression children
- accept unresolved CSS-variable shorthand expansion and eliminate
dynamic-alt/`htmlFor` audit false positives
- resolve exposed underscore MCP names during focused discovery
- omit unavailable stopped-preview fields and restart stale iterative
previews for external clients
- bound single and responsive screenshot capture, return
`SCREENSHOT_TIMEOUT`, and discard failed reusable browser sessions
- dispose previews when shared MCP runs fail or stdio clients
disconnect, including complete Windows process-tree cleanup
- resolve generated route templates from the CLI-managed preview
workspace during responsive verification
- retry a browser that exits during rendered-audit startup and report
sanitized exit diagnostics with a concrete browser-path recovery action
- materialize complete current session data before preview generation so
route order and project styles cannot collapse from missing lazy
namespaces
- always include CLI and Node.js version fields in generated issue
reports, with explicit legacy-client fallbacks
- document the updated preview and screenshot contracts and regenerate
the CLI manual

The current branch also retains the protections already present on
`main` for dry-run isolation, CSS rewrite truncation metadata,
stderr-only progress, shortcut argument forwarding, strict MCP schemas,
and refreshed conflict diagnostics.

## Technical choices

- MCP command orchestration stays in `mcp.ts`; preview port/retry
behavior lives with `mcp-preview`, and persisted checkpoint storage
lives in `mcp-checkpoint`. Each module owns its matching tests.
- Expression validation lives on the shared fragment input schema,
reuses `listFragmentExpressions`, and emits the same structured
validation issues as prop bindings, so JSX-derived and structured
fragments follow one contract across fragment, collection, and runtime
UI insertion.
- Element-child requirements come from component registry content models
during fragment insertion; no component names are hardcoded in the JSX
parser. `CollapsibleTrigger` metadata now matches its enforced `asChild`
implementation.
- Stopped preview status returns only `running: false`; unavailable
`url`, `pid`, and `mode` fields are optional and omitted from JSON.
- Explicit `preview.start` restarts a stale compatible iterative
preview; managed screenshots keep their optimized refresh path.
- Screenshot operations share one timeout utility. A single capture or
complete responsive batch is bounded after preview readiness;
lower-level CDP deadlines use the same stable error code.
- Failed managed captures close and forget their browser session. A
closed browser session cannot restart itself while cancellation is in
progress, which prevents orphaned Chromium processes.
- One ownership helper disposes one-shot and multi-project MCP hosts on
success or failure. Shared `mcp run` hosts dispose in one `finally`
boundary, long-running MCP hosts dispose on connection close, and
Windows uses `taskkill /T /F` instead of killing only the npm wrapper.
- Framework template readers receive the absolute template directory
captured from the build root, so later async work cannot redirect them
to the linked project's cwd. The same option drives cleanup for React
Router, Remix, and Vike.
- Browser startup gets one bounded retry before a rendered capture batch
fails. Persistent failures report only the exit code, signal, or spawn
error code plus the supported `WEBSTUDIO_BROWSER_PATH` recovery action;
local paths and raw browser output are not exposed.
- CLI and MCP preview share one session materializer that ensures the
canonical Builder namespace set before writing `.webstudio/data.json`.
Serialization rejects incomplete snapshots instead of converting missing
style/instance namespaces into valid-looking empty arrays, and preview
freshness records the post-materialization project version.
- The CLI injects its package version and `process.versions.node`; issue
formatting now keeps both fields visible even for legacy reports that
omit runtime metadata.

## Checklist

- [x] Add regression coverage for every changed behavior
- [x] Demonstrate new regression tests fail without the corresponding
fixes and pass with them
- [x] Run affected package test suites and typechecks
- [x] Run lint, package-boundary, and generated API checks
- [x] Review fixture impact — no fixture inputs or outputs are affected
- [x] Review documentation impact and update the authoritative MCP CLI
manual source/generated output
- [x] Run the complete local agent evaluation suite

## Verification

- `@webstudio-is/project-build`: 2,090 tests passed
- MCP asset query result modes are preserved across create, read, and
update serialization
- complete local agent evaluation suite: all 5 fixtures passed,
1,308,276 aggregate tokens
- `@webstudio-is/react-sdk`: 140 tests passed
- `@webstudio-is/template`: 43 tests passed
- `webstudio` CLI: 956 tests passed
- issue reporter: 5 targeted tests passed
- MCP host disposal regression was demonstrated failing without the
shared ownership boundary and passing with it
- relevant package typechecks passed
- `pnpm lint` passed
- `pnpm check:package-boundaries` passed
- `pnpm check:generated-api` passed
- generated-doc tests, typecheck, sync, and regeneration passed; the
aggregate clean-tree command reports the intentionally changed generated
CLI manual
- stalled single capture, stalled four-viewport capture, startup retry
across a four-viewport batch, sanitized browser-exit diagnostics,
complete preview materialization, post-refresh version tracking,
closed-session restart, Windows process-tree cleanup, and
preview-workspace regressions were demonstrated failing without their
fixes and passing with them

## Known limitations

- External `baseUrl` screenshots remain caller-owned and intentionally
do not regenerate, restart, or stop that external server.

Closes #6034
Closes #6035
Closes #6038
Closes #6042
Closes #6043
Closes #6044
Closes #6045
Closes #6046
Closes #6047
Closes #6048
Closes #6049
Closes #6050
Closes #6051
Closes #6052
Closes #6059
Closes #6060
 2026-08-08 22:22:25 +02:00
Oleg Isonen 2ec78c183d feat: Add single-result modes to Assets queries (#6040) 
## Summary

- Add `many`, `one`, `first`, and `last` result modes to Assets queries
while keeping `many` as the backward-compatible default.
- Return direct items for single-result modes, validate deterministic
ordering, and report duplicate `one` matches as a structured
`MULTIPLE_RESULTS` error.
- Update Builder controls, preview/published runtime handling, SDK
resource encoding, MCP/CLI guidance, generated contracts, and fixtures.

## Implementation TODOs

- [x] Add result-mode schemas and core execution behavior.
- [x] Select single results before content hydration and preserve
deterministic ID tie-breaking.
- [x] Add structured duplicate-match transport errors.
- [x] Update Builder controls and hide/reset pagination for
single-result modes.
- [x] Keep existing settings first, then Output and Result, without a
duplicate Return label.
- [x] Preserve legacy saved resources with an implicit `many` default.
- [x] Update preview, published runtime, SDK, protocol, MCP, and CLI
behavior.
- [x] Regenerate runtime contracts, server metadata, CLI docs, and
fixtures.
- [x] Review documentation impact and update the authoritative MCP/CLI
guidance.
- [x] Add regression coverage and verify the new tests fail without the
implementation.
- [ ] Run the agent evaluation suite (not run; requires explicit user
approval).

## Compatibility and behavior

- Existing queries without `result` continue to behave as `many` and
retain the ID-keyed collection shape.
- `one` returns an item or `null`, but fails when more than one asset
matches.
- `first` and `last` require explicit sorting; the existing asset-ID
tie-breaker makes equal sort values deterministic.
- Single-result modes ignore and do not persist pagination controls.
Materialized collection queries remain collection-only; single-result
queries execute dynamically.

## Verification

- Content engine typecheck and full test suite (291 tests).
- Query builder, query-builder React, SDK, protocol, asset uploader,
project build, Builder, and CLI targeted typechecks/tests.
- Root lint.
- MCP documentation tests, typecheck, sync check, and generated-doc
checks.
- Generated API contract checks.
- Fixture generation pipeline.
- Final focused content-engine regression run (53 tests).
- Final DRY review: affected typechecks, focused tests, lint, package
boundaries, generated checks, and fixtures.

Closes #6039
 2026-08-07 19:13:19 +02:00
Oleg Isonen d8491ea736 fix: Make bound fields read-only (#6028) 
Closes #6027

## Summary

Bound values currently remain editable when the expression directly
references a mutable variable. Editing the field writes a temporary
runtime override while leaving the binding in place, which makes the
source of truth unclear and behaves differently across Builder fields.

This change makes every expression-bound field read-only. To enter a
literal value, remove the binding first.

## Implementation

- Simplify binding state to two states: unbound/editable and
bound/read-only.
- Remove temporary data-variable override writes and the
overwritten/reset binding UI.
- Remove `allowBindingOverwrite` and bound-value parsing APIs and their
call-site exceptions.
- Guard pending draft saves so a draft started before binding cannot
replace the new binding.
- Add query-builder regression coverage for string and numeric filters
bound to mutable variables.

## Checklist

- [x] Add regression tests and confirm they fail with the previous
behavior.
- [x] Make all expression-bound controls read-only.
- [x] Remove temporary override state and UI.
- [x] Verify literal controls remain editable.
- [x] Run the full Builder unit test suite.
- [x] Run Builder typecheck and repository lint.
- [x] Review documentation impact: no documentation change is needed;
this makes the existing remove-binding-before-edit interaction
consistent.
- [x] Review fixture impact: no fixture inputs or generated fixture
outputs are affected.

## Verification

- `pnpm --filter @webstudio-is/builder test` — 2,509 tests passed across
217 files.
- `pnpm --filter @webstudio-is/builder typecheck` — passed.
- `pnpm lint` — passed.
- Agent evaluations were not run because they require separate explicit
approval and this change does not modify agent-facing behavior.
 2026-08-07 12:34:11 +02:00
Oleg Isonen 8e7a8a21aa feat: Report anonymous MCP reliability issues (#6021) 
Closes #6020

## Summary

Adds the first end-to-end slice for anonymous, LLM-authored reliability
reports from the CLI and MCP.

The shared report contract captures the generalized user story,
attempted workflow, expectation, actual result, recovery attempts, user
impact, technical context, acceptance criteria, and agent runtime
(client, provider, model, version, and reasoning effort). Guidance
requires removing any user- or project-identifying information before
submission.

## Architecture

- Adds a project-authenticated `api.reports.issue` tRPC mutation as the
reusable product boundary.
- Maps that mutation to the public `report-issue` operation for CLI and
MCP callers.
- Keeps the endpoint directly callable from a future Builder UI through
the existing tRPC client.
- Publishes through a server-side GitHub App installation, so end users
do not need GitHub accounts or tokens.
- Discovers the App installation for the fixed Webstudio reporting
repository instead of requiring deployment-specific installation or
repository configuration.
- Searches for a hidden anonymous technical marker before creation to
avoid obvious duplicates.

## Todo

1. [x] Define a shared anonymous LLM-authored report contract.
2. [x] Include the user story, attempted workflow, expected and actual
behavior, recovery attempts, impact, technical context, and acceptance
criteria.
3. [x] Include the LLM client, provider, model, client version when
known, and reasoning effort.
4. [x] Attach sanitized CLI, Node.js, operating-system, architecture,
execution-mode, and API-contract metadata automatically.
5. [x] Add the Builder-reusable project tRPC mutation.
6. [x] Expose the mutation to CLI and MCP as `report-issue`.
7. [x] Add GitHub App authentication and issue publication.
8. [x] Reduce deployment configuration to the App ID and private key.
9. [x] Add marker-based duplicate detection.
10. [x] Add guidance for explicit user-requested reports and qualifying
repeated-friction reports.
11. [x] Review and simplify repeated GitHub request handling and public
protocol exports.
12. [ ] Add server-side throttling and harden concurrent duplicate
creation.
13. [x] Configure and install the reporting GitHub App in deployed
environments.
14. [x] Exercise the deployed endpoint with a controlled test report.
15. [ ] Run the complete agent evaluation suite after explicit approval.
16. [x] Review documentation impact and update the generated MCP
documentation.

## Verification

- Protocol issue-report contract tests: passed (3)
- Builder GitHub reporting tests: passed (4)
- Builder API router tests: passed (30)
- CLI schema, command-doc, MCP, and project-session tests: passed
- Protocol, HTTP client, project-build, CLI, and Builder typechecks:
passed
- Root lint: passed
- Generated API and documentation checks: passed
- New and changed regression tests were demonstrated failing against the
previous behavior, then passing after implementation
- No fixture inputs or outputs are affected
- Agent evaluations were not run because they require explicit approval

## Deployment notes

The Builder only needs `GITHUB_ISSUE_REPORT_APP_ID` and
`GITHUB_ISSUE_REPORT_PRIVATE_KEY`. The server discovers the App
installation for `webstudio-is/webstudio` and creates a short-lived
installation token. The App needs repository metadata read access and
Issues write access.

## Known limitations

Duplicate detection currently uses GitHub issue search and can race when
identical reports arrive concurrently. The endpoint is
project-authenticated, but dedicated server-side throttling remains a
follow-up before production rollout. No Builder UI is included yet; the
tRPC boundary is intentionally ready for one.
 2026-08-06 00:32:51 +02:00
Oleg Isonen c44ef705cf perf: Speed up Builder resource loading (#6011) 
## Problem

Assets-backed Builder pages can take 6–7 seconds to become usable even
when their Assets resources only request metadata. On the
production-derived Blog project, a single Assets query measured 8,703 ms
and an immediate repeat measured 8,110 ms. Four sibling Assets resources
repeated the same authorization, development-build load/parse,
compilation, canonical-asset preparation, runtime metadata load, and
document work.

## Architecture

The loader now treats remote resources holistically while preserving
provider boundaries:

- The client scheduler is resource-provider neutral. It versions logical
requests, frees capacity when obsolete requests are replaced, ignores
stale results, and aborts physical batches only when every member is
obsolete.
- Ordinary Assets requests in one server resource-loader call use a
deferred Assets provider and execute as one repository batch.
Independent HTTP, GraphQL, and other resources start immediately instead
of waiting for Assets preparation.
- Arbitrary remote responses are **not** coalesced or persistently
cached. Their auth headers, cookies, upstream freshness, and `Vary`
semantics remain isolated.
- The Assets provider authorizes and creates its API context once per
batch, then loads only `props`, `dataSources`, and `resources` from the
development Build.
- Queries covered by the saved Build compilation plan reuse that exact
plan/artifact, preserving published byte limits, truncation, and
pinned-revision behavior. Uncovered unpinned queries use a deterministic
literal union and safely fall back to their existing individual path if
shared preparation cannot preserve behavior.
- Request-scoped document loading uses npm's byte-bounded `lru-cache` so
sibling queries do not reread the same referenced author/content files.
- Sibling dynamic queries with the same filter, sort, output, and
content shape stitch into one document-graph resolution while keeping
filter values, pagination windows, results, and failures independent.
Aggregate document/byte-limit failures fall back to isolated execution.
- Provider-neutral batching lives in the SDK runtime, and project
asset-query preview orchestration lives in the shared project package so
Builder and MCP paths can reuse the same behavior.
- Cancellation flows from the client batch through the
SDK/server/build/query path, with checkpoints between expensive phases.
- Runtime asset metadata remains conservatively artifact-wide because
URL/size filters, structured references, markdown hydration,
materialized rows, and saved plans have different reachability rules. To
make 1,000-asset artifacts safe, PostgREST `.in(...)` lookups are
deduplicated and split into 100-ID chunks with bounded concurrency.

## Implementation checklist

- [x] Trace the production Blog resource path and identify repeated work
- [x] Copy the production Blog project into an ignored disposable local
project
- [x] Record production and local baselines
- [x] Add provider-neutral client scheduling, stale-result protection,
and cancellation
- [x] Batch ordinary Assets queries without coupling or caching
HTTP/GraphQL responses
- [x] Load only Build fields required to create the saved content
compilation plan
- [x] Preserve saved-plan, truncation, fallback, diagnostics, and
revision semantics
- [x] Share Assets artifacts, runtime metadata, and revision-keyed
document reads per request
- [x] Stitch compatible query shapes into one graph resolution with
isolated fallback
- [x] Extract reusable SDK batching and project preview orchestration
for MCP/runtime consumers
- [x] Bound large runtime asset metadata lookups for the 1,000-asset
project limit
- [x] Add regression coverage and demonstrate new tests failing before
their fixes
- [x] Run focused and broad tests, type checks, lint, format,
package-boundary, and diff checks
- [x] Review documentation impact; no authoritative `docs/` content
changes are required
- [x] Regenerate and verify the affected asset-query runtime fixture; no
production data is changed
- [x] Address all actionable PR review comments
- [x] Rerun the final parity-preserving implementation against the
copied Blog project

## Measurements

Production trace before the change:

- Updates Assets query: 8,703 ms
- Immediate identical repeat: 8,110 ms
- Query artifact: 9,344 bytes / 7 documents
- Saved source database: 42,086 bytes / 28 documents

Copied-project local baseline:

- Builder navigation until resource loading: 5,111 ms
- Exact four-resource batches: 1,448 / 797 / 1,377 / 774 / 3,018 ms

During implementation, before the final saved-plan parity tightening:

- Exact four-resource batch: 174 ms first run, then 51–80 ms
- Actual Builder resource-loader requests: 234–246 ms

Final parity-preserving copied-project rerun:

- Preflight: 10 ms
- Exact four-resource batch: 435 ms cold, then 119 / 97 / 90 / 79 ms
warm
- Every response returned HTTP 200 with the same 16,884-byte payload

The final cold batch is 70% faster than the best copied-project cold
baseline, and the fastest warm batch is 90% faster than the best warm
baseline. The production trace remains the authoritative production
baseline; these before/after comparisons use the same disposable local
project and captured request payload.

## Verification

Passed:

- Full package suites: Builder (212 files / 2,496 tests), asset-uploader
(22 / 193), content-engine (40 / 274), project (7 / 84), SDK (29 / 508),
and project-build (82 / 2,070)
- Final focused regressions: Builder preview/API/resource integration (3
files / 42 tests), SDK resource loader (23 tests), and document-graph
resolution (6 tests)
- Builder, asset-uploader, content-engine, project, SDK, and
project-build type checks
- Fixture regeneration is deterministic; the generated vendor exactly
matches the source runtime bundle
- Fixture typecheck and production build
- Changed-file Oxfmt, repository Oxlint, and package-boundary checks
- `git diff --check`
- `git diff --name-only HEAD --exit-code`

New behavioral tests were demonstrated to fail before implementation and
pass afterward, including Assets batch coordination, saved-plan wiring,
truncation and pinned revisions, shared document reads, cancellation
checkpoints, stale/replaced client requests, and large PostgREST ID
filters.

Agent evaluations were not run because they require separate explicit
approval.

## Compatibility and remaining work

The public resource result/error shape and on-demand diagnostics
contract stay compatible. Generic HTTP/GraphQL caching semantics are
unchanged. Compilation already in progress is not force-cancelled
because safely interrupting shared compilation requires
ownership/reference tracking; cancellation prevents later phases from
starting.

Closes #6003
 2026-08-04 18:39:41 +02:00
Oleg Isonen 6385e290c6 feat: Add syntax highlighting to Code Text (#6007) 
Closes #4119

Supersedes #5999 after GitHub closed it during the head-branch rename.

## Approach

Code Text uses one optional public highlighting boundary:

- `@webstudio-is/sdk-components-react/code-text` → `src/code-text.tsx`

The ordinary `components` entry remains Shiki-free. Canvas loading,
metadata options, theme variables, and the plain semantic `<code>`
component remain private implementation details.

**Code** uses the code editor. **Language** and **Theme** use normal
bindable selects. Fixed selections generate direct imports for only the
per-page language/theme union; plaintext imports no grammar. Bound
selections generate Shiki's lazy language/theme registries.

SSR and client rendering share the same factory. Missing bound assets
suspend the first render while SSR resolves the selected modules. SSG
now waits for Suspense before serializing HTML. During hydration the
completed server markup stays in place while the browser fetches only
the selected language and theme. Later client-side binding changes load
only the new selections and keep the same `<code>` node and focus.

Highlighted output preserves the semantic root, ref, author
classes/styles, whitespace, escaped source, and legacy children/native
`lang` behavior. Shiki's implicit tab stop is disabled. A color-only
transformer strips theme typography so asynchronous highlighting cannot
change text geometry.

## Bundle impact

Measured from same-commit production React Router + Cloudflare builds
containing one JavaScript / GitHub Light Code Text instance. Sizes are
the sum of emitted JavaScript files; gzip is the sum of each file
compressed independently.

| Worker output | Files | Raw | Sum gzip |
| --- | ---: | ---: | ---: |
| Fixed selections | 3 | 1,341,851 B | 288,342 B |
| Bound selections | 310 | 10,372,112 B | 1,872,273 B |
| Increase | +307 | +9,030,261 B (8.61 MiB) | +1,583,931 B (1.51 MiB) |

The bound browser build also emits the catalog as lazy chunks:
+9,012,594 raw bytes / +1,578,779 sum-gzip bytes across the complete
deployed output. Those chunks are not all delivered on page load. The
production SSG regression verifies 100+ catalog chunks are emitted while
the HTML directly module-preloads only one shared runtime chunk; the
browser fetches the runtime-selected language/theme modules on demand.

## Tradeoffs and compatibility

- Instances with both highlighting properties unset keep the legacy
plain renderer.
- Static highlighting still requires both **Language** and **Theme**.
- Binding either selection includes that catalog in deploy output
because its runtime value is not known at publish time.
- The syntax-highlighting property is `language`; legacy native HTML
`lang` remains preserved.
- Invalid or unavailable runtime selections render as plain code.
- Fixed selections retain the smallest publish output; bound selections
trade deploy/worker size for runtime flexibility.

## Todo

- [x] Restore the specialized Code Text component and semantic plain
fallback
- [x] Use a code editor for Code and bindable selects for Language and
Theme
- [x] Persist JavaScript / GitHub Light template defaults
- [x] Collapse highlighting to one public `./code-text` entry
- [x] Keep canvas loaders and option/theme helpers private
- [x] Generate direct imports for fixed selections and omit plaintext
grammar
- [x] Generate lazy catalogs when Language or Theme is bound
- [x] Preserve highlighted SSR markup during hydration without
node/focus movement
- [x] Support Suspense-aware highlighted SSG output
- [x] Preserve escaping, accessibility, legacy children, and native
`lang`
- [x] Measure fixed versus bound Cloudflare worker output
- [x] Deep-review correctness, DRYness, package surface, and obsolete
binding infrastructure
- [x] Review documentation impact and update Code Text documentation
- [x] Run focused tests, typechecks, production builds, formatting, and
staged diff review
- [ ] Fixture pipeline — blocked during sync because the current API
rejects this local CLI version as incompatible; no fixture output
changed

## Verification

- [x] New/changed regressions were demonstrated failing without their
behavior and passing after restoration
- [x] SDK Code Text metadata, rendering, hydration, and bundle tests: 17
passed
- [x] Component registry tests: 3 passed
- [x] CLI asset collection and full prebuild suite: 59 passed
- [x] SDK components, component registry, SDK, and CLI typechecks
- [x] SDK components declaration and production builds
- [x] Component registry production build
- [x] Production Vite + Vike prerender emits highlighted bound SSG HTML
with one doctype
- [x] Dynamic catalog chunks remain outside the SSG page's direct
preload set
- [x] Documentation reviewed and updated
- [x] Formatting and staged diff checks
 2026-08-04 18:25:36 +02:00
Oleg Isonen 08e1e19b3f fix: expose existing text bindings in mixed content (#6008) 
Closes #5992

## Summary

Expose the existing **Text content** binding control when a directly
text-capable selected instance contains expressions alongside other
content.

The supplied reading-time span keeps its existing child sequence:

- static prefix text
- bound expression

No data migration or new Settings editor is introduced. Normalization
happens only when the user explicitly resets a binding.
Expression-bearing nodes are excluded from canvas text editing because
the canvas editor cannot represent bindings safely. Literal descendants
in mixed trees remain canvas-editable, while bindings remain editable
through Settings.

## Implementation

- Reuse the existing scalar Text content and binding UI.
- Target expression children with indexed `instances.updateText`
mutations, preserving instance boundaries and unrelated siblings.
- Fail closed when the selected instance is missing or no longer has a
safe text target.
- Evaluate expressions through the binding control's existing live
scope.
- Centralize text-content target classification in project-build for
reuse by the engine and Builder.
- Reset every expression atomically through the engine's
`instances.setTextContent` operation, validate the complete replacement
set, and coalesce adjacent text nodes without crossing nested-instance
boundaries.
- Centralize expression-free canvas text-editor targeting in the
project-build lookup module.
- Route double-click, Enter, post-insert editing, cursor generation, and
keyboard navigation through the shared lookup rules.
- Exclude single-expression and mixed/nested expression targets from the
canvas editor in every mode.
- Preserve canvas editing for literal descendants inside otherwise mixed
content.
- Share canvas fixtures and simplify binding-removal capability to an
optional handler.

## Verification

- [x] Settings exposes and edits mixed text/expression targets without
replacing siblings.
- [x] Reset binding converts every expression on the instance and
normalizes adjacent text.
- [x] Missing, unsupported, stale, and incomplete replacement sets fail
closed.
- [x] Double-click and Enter do not open bound nodes in the canvas
editor.
- [x] Literal descendants within mixed trees remain canvas-editable.
- [x] Complete Builder suite passes (2,497 tests).
- [x] Complete project-build suite passes (2,070 tests).
- [x] New targeted Builder and project-build regression tests pass (175
tests).
- [x] Builder and project-build typechecks pass.
- [x] Repository lint passes.
- [x] Package-boundary validation passes.
- [x] Generated runtime operation contracts are updated from source.
- [x] Fixture inputs and outputs are unaffected.
- [x] Documentation impact reviewed: no documentation update is needed.
- [ ] Interactive browser smoke test was unavailable because the in-app
browser failed to initialize.
- [ ] Agent evaluations were not run because they require separate
approval.

## CI note

The previous head's Main workflow failed while pulling the E2E database
image with Docker Hub `toomanyrequests: Rate exceeded`. Lost Pixel
failed separately with a certificate error. Both are external
infrastructure failures; branch formatting, lint, package boundaries,
generated contracts/docs, tests, and typechecks were successful in that
run.

Supersedes #5993, which GitHub closed automatically when its head branch
was renamed.
 2026-08-04 15:36:19 +02:00
Oleg Isonen 17947e47fa fix: Constrain the asset settings popover width (#6002) 
## Summary

- Set the asset settings popover to a fixed 250px width using the
existing `css` prop.

## Root cause

The popover used a minimum width while its shared content style allowed
intrinsic `max-content` sizing. Long descriptions could therefore expand
the popover beyond the intended layout.

## Impact

Asset settings now stay consistently sized, and long descriptions remain
contained within the popover.

## Todo

- [x] Constrain the asset settings popover width.
- [x] Keep styling on the existing `css` prop.
- [x] Avoid a constant-only regression test, in line with the existing
agent testing guidance.
- [x] Confirm fixture inputs and outputs are unaffected.
- [x] Review documentation impact; no user-facing documentation changes
are needed.
- [x] Run targeted tests, formatting, linting, and Builder typecheck.

## Verification

- `pnpm --filter=@webstudio-is/builder test -- asset-settings.test.tsx`
- `pnpm exec oxfmt --check
apps/builder/app/builder/shared/asset-manager/asset-settings.tsx`
- `pnpm exec oxlint --deny-warnings
apps/builder/app/builder/shared/asset-manager/asset-settings.tsx`
- `pnpm --filter=@webstudio-is/builder typecheck`

Closes #6001
 2026-08-04 12:34:59 +02:00
Oleg Isonen cf0e76f96a fix: Validate authored fragments against HTML content models (#5988) 
## Summary

Prevent authored fragment insertion from creating invalid HTML
structures while preserving the Builder's warning-only behavior for
legacy clipboard content.

Also show an explicit loading state while Assets resource diagnostics
are fetched lazily, instead of briefly reporting that no diagnostics are
available.

Allow established text assets to be edited without their preserved
creation timestamp being mistaken for an expired upload reservation.

## Root cause

The shared insert-fragment runtime mutation did not validate the final
HTML content model.

Builder copy and paste uses the same mutation for legacy project
content. The compatibility path must tolerate only violations already
present in the copied fragment while still rejecting new violations
caused by its destination.

The diagnostics inspector had no pending state, and an unkeyed pending
state could be cleared by an older request after the inspected resource
changed.

Asset revisions preserve the original file's `createdAt` value, but
upload finalization used `createdAt` to enforce the 30-minute
upload-ticket lifetime.

## Implementation

- [x] Enforce HTML content-model validation at the shared
fragment-insertion mutation boundary.
- [x] Centralize fragment placement warning collection with the existing
matcher logic.
- [x] Keep the legacy compatibility switch out of the public runtime/MCP
contract.
- [x] Pass compatibility through trusted Builder runtime context only.
- [x] Ignore only violations already present in legacy clipboard content
and reject new destination violations.
- [x] Preserve warning-only behavior across the shared Builder paste
helper and instance-copy path.
- [x] Key diagnostics loading state to the active resource request.
- [x] Validate upload-ticket freshness with `updatedAt` while preserving
logical asset `createdAt`.
- [x] Regenerate the runtime operation contract.
- [x] Add regression coverage for strict insertion, legacy
compatibility, destination validation, paste context propagation,
diagnostics request races, and established text assets.

## Impact

MCP and other public authored-fragment callers remain strictly validated
and cannot opt out.

Legacy Builder clipboard content remains pasteable with warnings, but
compatibility cannot introduce a new violation at the selected
destination.

Assets diagnostics retain their loading state when an older request
settles, and established text assets can be updated after 30 minutes
without changing their displayed creation date.

## Verification

- [x] Project-build targeted tests: 80 passed.
- [x] Builder targeted tests: 86 passed.
- [x] Asset-uploader tests: 184 passed.
- [x] Builder typecheck passed.
- [x] Project-build typecheck passed.
- [x] Asset-uploader typecheck passed.
- [x] Generated runtime operation contracts are current.
- [x] New regression tests were demonstrated failing without their fixes
and passing after restoration.
- [x] Fresh GitHub Main workflow passed, including both test/typecheck
matrices and all six Builder E2E shards. Shard 3 was rerun successfully
after Docker Hub rate-limited its initial database-image pull.

## External CI status

- Lost Pixel failed on an external TLS certificate handshake after
Storybook built successfully.
- The staging deployment workflow failed before executing job steps,
independently of code checks.
 2026-08-03 17:38:17 +02:00
Oleg Isonen 5cefbc0320 perf: Speed up Builder resource loading (#5980) 
## Summary

Remove avoidable latency from Builder resource loading and make Assets
resources use the content database efficiently.

## Why

Builder currently applies a one-second debounce to every uncached
resource request. Assets queries also perform server-side preparation
work that should be reused or deferred, making pagination and route
changes noticeably slower than expected.

## Implementation plan

- [x] Dispatch the complete set of currently computable page resources
without the one-second debounce.
- [x] Preserve batching, deduplication, exact-request caching,
invalidation, and bounded queue draining.
- [x] Add regression tests for immediate dispatch, batching,
deduplication, invalidation, and queue draining.
- [x] Cache compiled Assets content databases by the relevant
project/build and asset revision.
- [x] Run pagination, filtering, sorting, and metadata selection against
the cached database without rebuilding it for each request.
- [x] Fetch only referenced document content that is required by the
query and keep graph reads parallel.
- [x] Avoid generating and returning full diagnostics artifacts for
ordinary canvas resource loads; load detailed diagnostics on demand.
- [x] Preserve distinct loading and confirmed-empty states for article
and author resources.
- [x] Add coverage for paginated overview and article navigation
behavior.
- [x] Deep-review the complete change and remove redundant plan
construction, duplicated JSON comparison, path-based diagnostics
inference, and synthetic canvas diagnostics.
- [x] Make resource and diagnostics invalidation race-safe, including
invalidation during in-flight requests.
- [x] Run formatting, tests, lint, type checking, package-boundary
checks, and affected fixtures.

## Verification

- `pnpm oxfmt --check`
- `pnpm lint`
- `pnpm check:package-boundaries`
- `pnpm --filter @webstudio-is/content-engine test` (262 tests)
- `pnpm --filter @webstudio-is/asset-uploader test` (184 tests)
- `pnpm --filter @webstudio-is/builder test` (2,483 tests)
- Typechecks for Content Engine, Project Build, Asset Uploader, and
Builder
- `pnpm fixtures` (passed; no fixture diff)

The new invalidation regression tests were verified to fail against the
previous behavior and pass with the fix. Agent evaluations were not run
because they require separate approval.

## Known limitations

None identified.
 2026-08-03 01:15:56 +02:00
Oleg Isonen 92084aeea3 fix: Remove MCP workflow friction (#5974) 
## Summary

- Make MCP discoverable from the CLI README and expose direct MCP tool
calls as restart-free CLI subcommands.
- Preserve required discriminator fields in compact MCP SDK schemas and
surface the `set-text-content` contract.
- Return instance props by default from `inspect-instance` and improve
the `list-instances` input hint.
- Add a Builder context-menu **Move** submenu as an accessible
alternative to drag-and-drop sibling reordering.

Companion generated documentation:
https://github.com/webstudio-is/webstudio-community/pull/274

## Technical choices and compatibility

- Compact SDK schemas retain required fields only inside root
composition branches, avoiding broad schema expansion.
- `inspect-instance` now defaults to `props` only; larger styles,
bindings, sources, children, and ancestors remain opt-in.
- Existing move commands and shortcuts back the new menu, so command
behavior stays centralized.
- No migrations, fixture inputs, generated fixture bundles, or generated
API contracts are affected.

## Todo

- [x] Add direct CLI MCP discovery and restart guidance.
- [x] Preserve discriminated-union requirements for SDK clients.
- [x] Improve instance inspection and input diagnostics.
- [x] Add accessible menu-based instance movement.
- [x] Add regression coverage and verify fail/pass behavior.
- [x] Run targeted tests, typechecks, lint, package-boundary, and
generated-doc checks.
- [ ] Deferred: run `pnpm evaluations` after separate explicit approval,
as required by repository policy.

## Verification

- `@webstudio-is/project-build`: 172 targeted tests passed.
- `@webstudio-is/protocol`: 8 targeted tests passed; typecheck passed.
- `@webstudio-is/builder`: 8 targeted tests passed; typecheck passed.
- CLI and project-build typechecks passed.
- Root lint, package-boundary check, generated-doc check, and diff
checks passed.
- New and modified regression tests were demonstrated failing before
their fixes and passing afterward.

## Known limitations

Synthetic drag-and-drop behavior is unchanged. The new menu and existing
keyboard shortcuts provide the automatable and accessible reordering
path. Agent evaluations were not run because explicit approval was not
provided.
 2026-08-03 00:36:56 +02:00
Oleg Isonen aefe532a73 fix: Preserve instance deep links on Builder reload (#5977) 
## Summary

Preserve exact Builder instance selections during initial load,
authentication, and hard reload, including shared content rendered
through multiple Slot instances.

## Implementation

- [x] Store the complete selector in one comma-separated `instance` URL
parameter.
- [x] Reuse `canResolveInstanceSelector` to validate every selector path
before restoring it.
- [x] Resolve the selected page from the selector root.
- [x] Wait for the authoritative Builder data-loaded state before
applying initial URL state.
- [x] Keep selection-to-URL synchronization paused until initial URL
state is resolved.
- [x] Let the existing auth loader derive `returnTo` directly from
Builder requests.
- [x] Update generated Builder links, including pre-publish audit links,
to use full selectors.
- [x] Add regression coverage for incomplete sync data and duplicate
shared-slot child IDs.

## Verification

- [x] A fresh browser session authenticates and opens a direct selector
deep link.
- [x] Hard reload retains the exact selector URL and selection.
- [x] Clicking the first shared child creates an `instance` URL for the
first Slot path; reload restores the first occurrence.
- [x] Clicking the second shared child creates a distinct `instance` URL
for the second Slot path; reload restores the second occurrence.
- [x] Focused Vitest suite: 8 tests passed.
- [x] Builder TypeScript typecheck.
- [x] Focused oxlint.
- [x] Focused oxfmt check.
- [x] `git diff --check`.

## Compatibility and limitations

No schema, migration, or public API changes. This change uses one
canonical full-selector URL format; ID-only deep links are not retained
as a compatibility path.
 2026-08-03 00:16:26 +02:00
Oleg Isonen 9fe63c7501 fix: Submit webhook forms only once (#5979) 
## Summary

Webhook Form action resources were being persisted both as prop-bound
actions and render-time data sources. Generated sites could therefore
execute the POST while rendering and again when the visitor submitted
the form.

This change keeps prop-bound resources action-only and hardens
generation for existing builds that still contain a stale data-source
binding.

Ref #5806

## Implementation

- Keep `resources.upsertProp` resources out of render-time data and
remove stale exposure when an action is edited.
- Make the mutation's `dataSourceId` result optional and regenerate the
runtime contract.
- Exclude action-bound resources from the generated data-resource map,
preserving them only in the action map.
- Extend the Webhook Form E2E with a local request-counting endpoint.

## Todo

- [x] Prevent new Form actions from creating render-time data sources.
- [x] Protect existing persisted builds from loading action resources
during rendering.
- [x] Regenerate the runtime operation contract.
- [x] Add mutation, generation, Builder-store, and generated-site
regression coverage.
- [x] Verify zero calls on page load and one populated POST on submit.

## Verification

- [x] Focused Builder E2E: `Webhook Form action submits once and
persists after reload`
- [x] Project Build test suite: 2,062 tests
- [x] SDK test suite: 505 tests
- [x] Builder resource mutation test file: 25 tests
- [x] SDK, Project Build, and Builder typechecks
- [x] Changed-file lint, formatting, and diff checks
- [x] Generated API cleanliness check
- [ ] Agent evaluation suite (not run; requires explicit approval)

## Notes

No fixture inputs or generated fixture bundles are affected.
 2026-08-02 22:56:08 +02:00
Oleg Isonen 22642fc90b fix: Show publish feedback immediately (#5978) 
## Summary

Move the optimistic publishing state update ahead of the asynchronous
content-database diagnostics request. The publish button now shows its
pending animation immediately after a valid click instead of waiting for
that network round trip.

## Root cause

The content-database publish diagnostics were added before the function
that set `isPublishing`. Any latency in that request left the publish
button looking idle.

## Implementation

- Start the optimistic publishing state at the beginning of the publish
transition.
- Preserve the existing pre-publish audit, content-database warning,
publish request, polling, and error behavior.

## Todo

- [x] Trace the publish click and pending-state sequence.
- [x] Move pending feedback ahead of asynchronous diagnostics.
- [x] Run targeted lint and Builder typecheck.
- [x] Run the complete Builder test suite.

## Verification

- `pnpm exec oxlint --deny-warnings
apps/builder/app/builder/features/publish/publish.tsx`
- `pnpm --filter @webstudio-is/builder typecheck`
- `pnpm --filter @webstudio-is/builder test` — 211 files, 2,472 tests
passed

No fixture inputs or generated outputs are affected.

Closes #5976
 2026-08-02 17:13:23 +02:00
Oleg Isonen ca08919b13 fix: Confirm conflicting root styles on page paste (#5971) 
## Summary

Prevent page, template, folder, and marketplace page imports from
silently replacing local declarations on the shared global root. When
incoming root styles differ at the same property, breakpoint, and state,
Builder now asks whether to preserve the project values or use the
incoming values. Cancellation happens before the runtime mutation.

## Technical choices

- Compare CSS values by their serialized meaning so equivalent keyword
and unparsed values do not create false conflicts.
- Plan breakpoint remapping with the same insertion path used by
fragment paste, including ID collisions and breakpoint limits.
- Keep named-token conflict resolution separate because token conflicts
support suffixing and merging, while a shared root-local declaration has
only an existing or incoming value.
- Default the root-style dialog to **Keep existing** because global root
styles affect every page, and show the breakpoint plus current and
incoming values for each conflict.
- Re-read current project data after the asynchronous token dialog
before detecting root conflicts.
- Leave no-conflict resolution undefined so the runtime performs a final
conflict check instead of treating it as an explicit **Use incoming**
choice.
- Resolve root conflicts at the page-copy boundary before using the
generic fragment/style insertion path.
- Reuse one root-conflict UI resolver for clipboard paste and
marketplace imports, one page-transfer traversal helper in the UI and
runtime, and one presentational shell for token and root-style conflict
dialogs.
- Require an explicit `rootStyleConflictResolution` for conflicting
`pageTransfer.insert` and `pages.copy` runtime operations so non-UI
callers cannot overwrite global values accidentally.
- Document that `ours` keeps target-project values and `theirs` uses
incoming values in both generated runtime contracts.

## Todo

- [x] Detect conflicting root-local declarations by property,
breakpoint, and state
- [x] Add keep-existing, use-incoming, and cancel behavior
- [x] Cover page, template, nested-folder, marketplace, and direct
runtime transfer paths
- [x] Preserve non-conflicting incoming root declarations with either
resolution
- [x] Avoid false conflicts for equivalent CSS values and colliding
breakpoint IDs
- [x] Revalidate root styles after asynchronous token resolution
- [x] Preserve existing named-token conflict behavior
- [x] Remove duplicated transfer traversal, conflict resolution mapping,
and dialog layout
- [x] Regenerate the runtime operation contract
- [x] Verify regression tests fail when their corresponding behavior is
disabled
- [x] Run targeted tests, typechecks, formatting, and lint

## Verification

- `pnpm --filter @webstudio-is/project-build test --
src/runtime/style-copy.test.ts src/runtime/fragment.test.tsx
src/runtime/page-copy.test.tsx` — 80 passed
- `pnpm --filter @webstudio-is/builder test --
app/shared/copy-paste/plugin-page.test.ts
app/shared/root-style-conflict-dialog.test.tsx
app/shared/token-conflict-dialog.test.tsx` — 17 passed
- Retrospective fail/pass audit covered semantic CSS equality,
breakpoint ID collisions, both runtime guards, both resolution
directions, folder/template paths, conflict detail rendering, and stale
target data after token resolution
- `pnpm --filter @webstudio-is/project-build typecheck` — passed
- `pnpm --filter @webstudio-is/builder typecheck` — passed
- `pnpm lint` — passed
- Runtime contract regeneration was repeated and produced identical
output
- Fixture pipeline not run locally: no fixture inputs or outputs are
affected; GitHub fixture checks passed on the previous revision
- Agent evaluations not run: repository guidance requires separate
explicit approval before running them

## Compatibility and limitations

The new runtime inputs are optional for non-conflicting imports.
Existing imports continue unchanged; only imports that would overwrite a
conflicting shared root-local declaration require an explicit
resolution. No migrations or persistence changes are required.

Closes #5970
 2026-08-02 12:33:02 +02:00
Oleg Isonen f335340a54 fix: Correct content database sizing and Markdown references (#5972) 
## Summary

- Report only serialized content-database bytes that are actually
included in the published bundle.
- Replace the ambiguous `markdown-body` query mode with the strict
`markdown-body-ref` mode.
- Require every selected Markdown body to have a document-graph node;
compilation now fails instead of embedding source bytes as a fallback.
- Apply the same Markdown eligibility rule during compilation and
runtime query execution, excluding non-Markdown files from
reference-only databases and results.
- Highlight database sizes in red only when documents were omitted
because of the size limit.
- Refresh Assets diagnostics after an Assets resource edit is confirmed
by the server.
- Update CLI/MCP guidance, examples, schemas, generated contracts, and
fixtures to use the reference-only mode.

## Root cause

Dynamic Markdown-body queries conservatively treated unrelated files,
including remote images, as possible hydration candidates. The old mode
could also fall back to embedding a Markdown body when graph resolution
was unavailable, despite its intended storage-backed behavior.
Compilation and runtime selection did not previously share the same
content-mode eligibility check. Diagnostics displayed the unbounded
pre-limit estimate instead of the serialized bundled size, did not carry
the omission reason into query-preview diagnostics, and an early cached
preview could leave the merged measurement stale until reload.

## Impact

Content database diagnostics and publish warnings now reflect bundled
database bytes rather than remote source-file sizes. Saved query filters
trigger a post-sync diagnostics refresh. Markdown detail queries must
use `content.mode: "markdown-body-ref"`; the old `markdown-body` value
is rejected, missing graph support is an explicit compilation error, and
non-Markdown assets cannot be selected or counted by a reference-only
query.

Ref #5966

## Verification

- Content engine: 39 files / 261 tests passed.
- Asset uploader: 22 files / 183 tests passed.
- SDK: 29 files / 505 tests passed.
- Protocol: 9 files / 86 tests passed.
- Project build: 82 files / 2,055 tests passed.
- CLI: full suite previously passed (57 files / 933 tests); final
prebuild verification passed 48 tests.
- Focused Builder diagnostics tests: 4 files / 10 tests passed.
- Typechecks passed for Builder, content engine, asset uploader, SDK,
protocol, project build, and CLI.
- Generated API contracts and docs are current.
- Fixture build passed and is reproducible after the final review
changes.
- Formatting, lint, package-boundary checks, and diff checks passed.
- New strict-reference, runtime-selection, compilation-selection, and
size-highlight regression tests were demonstrated failing without their
implementations and passing with them.
- Agent evaluations were not run because this follow-up did not receive
separate evaluation approval.
 2026-08-02 12:20:10 +02:00
Oleg Isonen 56a22fde8a feat: Add storage-backed document graph resolution (#5966) 
## Summary

Introduce a document graph architecture for Assets resources so
structured documents can reference one another while keeping full
document payloads on CDN-backed storage.

The content database will retain the searchable index and dependency
graph. At runtime, the server will select a root document, determine its
complete dependency set before performing I/O, fetch each unique
document concurrently, and assemble the final result server-side.

## Implementation todos

### Document graph foundation

- [x] Add normalized internal reference contracts for whole documents,
JSON paths, Markdown body, and Markdown frontmatter.
- [x] Add an immutable graph abstraction with deterministic ordering,
dependency traversal, revision validation, and cycle detection.
- [x] Add a storage-independent resolver with dependency closure,
bounded parallel loading, shared-content deduplication, representation
selection, assembly hooks, cancellation, and structured errors.
- [x] Add focused unit coverage for the isolated document-graph
abstraction.
- [x] Define and validate the external remote-document reference syntax.
- [x] Add a pure graph compiler that resolves canonical document URLs
into IDs and revisions and validates missing, duplicate, and cyclic
dependencies.

### Build-time graph

- [x] Add a storage-independent source-catalog compiler for bounded
parallel reference discovery and deterministic graph compilation.

- [x] Add a versioned deterministic document graph artifact with
canonical serialization and integrity verification.

- [x] Discover and normalize statically declared references while Assets
resources are processed.
- [x] Resolve document identities and revisions and report missing,
stale, or cyclic dependencies with document context.
- [x] Persist document nodes and dependency edges alongside searchable
resource metadata.
- [x] Include persisted graph data in artifact integrity and size
validation.

### Document representations

- [x] Add a standalone JSON document adapter for exact reference
markers, compiler occurrences, JSON path selection, and immutable
assembly.
- [x] Add a standalone Markdown document adapter for bounded decoding,
frontmatter references, immutable assembly, and source, body, and
frontmatter selection.

- [x] Parse fetched JSON and Markdown documents through format-specific
adapters.
- [x] Select complete documents, JSON fragments, Markdown body, and
Markdown frontmatter through the shared reference model.
- [x] Assemble nested and shared references into the final server-side
value.

### Runtime resolution

- [x] Enforce reachable-document, per-document byte, and aggregate-byte
limits before parsing and assembly.

- [x] Add a revision-keyed bounded source-cache wrapper with replayable
bytes, stale-entry fallback, and in-flight deduplication.

- [x] Add a storage-neutral loaded-source contract that rejects revision
mismatches before parsing.

- [x] Resolve graph-backed property paths before filtering, sorting,
pagination, and projection; skip graph loading for unrelated queries.
- [x] Add an injected HTTP/CDN document loader with revision metadata
validation.
- [x] Apply runtime payload limits, bounded concurrency, caching, and
structured failure mapping.
- [x] Ensure shared documents are fetched and parsed once per
resolution.

### Product integration

- [x] Integrate equivalent behavior in local preview, SSR, SSG, and
published runtime paths.
- [x] Preserve existing embedded Assets resource behavior without
requiring migration.
- [x] Add observability for graph loading, document fetching, caching,
and resolution failures.
- [x] Document the public reference and resolution behavior.

### Review hardening

- [x] Enforce graph, artifact, cache, and published-storage identity
consistency.
- [x] Compile only graph sources reachable from selected build entries.
- [x] Resolve graph-backed fields before query evaluation and avoid
unnecessary graph loading.
- [x] Consolidate duplicated graph schemas, utilities, and document
content-type detection.
- [x] Correct runtime documentation and use structural evaluation
validation.

### Verification

- [x] Keep a deterministic high-impact agent evaluation for
Markdown-to-JSON references across the dynamic blog overview and article
routes.
- [x] Add a separate discoverability evaluation that must learn the
reference workflow from MCP guidance without prompt-supplied syntax,
queries, bindings, or call order.
- [ ] Re-run the complete local agent evaluation suite after the
discoverability fixture change and compare every fixture with its
accepted baseline.

- [x] Add integration coverage for CDN-backed hydration, parallel
fetching, caching, and shared dependencies.
- [x] Add integration coverage for missing references, cycles, revision
changes, limits, and cancellation.
- [x] Verify equivalent behavior across preview, SSR, SSG, and published
deployments.
- [x] Verify that existing embedded Assets resources retain their
current behavior.

## Intended outcome

- Storage-backed documents do not require their complete payloads to be
embedded in the content database.
- The server knows the complete statically discoverable dependency set
before fetching document content.
- Shared dependencies are fetched once per resolution and independent
resources can be fetched in parallel.
- JSON fragments and Markdown body/frontmatter representations use the
same document-resolution system.
- Reference and payload revisions cannot be mixed silently.
- Existing Assets resources continue to work without migration.

## Verification results

- `@webstudio-is/content-engine`: 37 files / 244 tests passed; typecheck
passed.
- `@webstudio-is/asset-uploader`: 22 files / 182 tests passed; typecheck
passed.
- CLI: 57 files / 925 tests passed, including generated SSR and SSG
build/prerender coverage; typecheck passed.
- Repository-wide `pnpm oxfmt --check` passed across 2,757 files.
- Package-boundary validation passed.
- Previous `pnpm evaluations` run: all 4 original high-impact fixtures
passed. The complete 5-fixture rerun is pending explicit approval after
adding the discoverability fixture.
 2026-08-02 10:39:06 +02:00
Oleg Isonen 3ccb46cec7 fix: restore Marketplace settings and optimize Assets publishing (#5964) 
## Outcome

Restore the Marketplace section for projects without existing
marketplace metadata, make the complete review-submission workflow
available over MCP, clarify shared Assets database diagnostics, and
reduce published database size by materializing fully static Assets
queries.

## Implementation

### Marketplace and MCP

- Initialize the Marketplace form from a safe partial default instead of
rendering nothing when metadata is unconfigured.
- Keep review submission disabled until the complete metadata schema is
valid.
- Allow `get-marketplace-product` to return `null` and
`update-marketplace-product` to create the first marketplace product
through ProjectSession/MCP.
- Add a confirmed `submit-marketplace-product` server operation that
validates configured metadata before setting review status to `PENDING`.
- Document the MCP workflow: upload the thumbnail, update metadata,
publish production, then submit.
- Correct the Marketplace thumbnail upload example to use the current
MCP input field.

### Assets database

- Show query size and merged published-database size separately in
resource diagnostics.
- Clarify that only the merged database counts toward the publish limit.
- Allow the Assets query editor to scroll long query source.
- Materialize fully static metadata-only Assets queries at publish time.
- Store materialized results as compact field columns and rows keyed by
a deterministic normalized-query hash.
- Serve materialized rows through the existing Assets response contract
without candidate documents or a runtime field catalog.
- Keep route- or variable-dependent queries on the existing
document-backed runtime path.
- Exclude overview-only fields from dynamic detail documents so the
merged database does not duplicate them.
- Preserve byte-limit selection, truncation diagnostics, integrity
verification, caching, and backward compatibility with existing
artifacts.

### Release

- Keep the release lockfile stable when preparing the CLI package.
- Regenerate runtime contracts, server-only API metadata, and CLI
documentation where required.

## Todo

- [x] Restore the empty Marketplace settings state
- [x] Prevent empty review submissions
- [x] Support first-time Marketplace metadata creation over MCP
- [x] Add MCP review submission
- [x] Document the end-to-end MCP workflow
- [x] Correct query and merged-database diagnostics
- [x] Make long Assets query source scrollable
- [x] Materialize static overview queries
- [x] Preserve dynamic detail-query behavior
- [x] Add regression coverage with fail/pass proof
- [x] Run relevant package and integration checks

## Verification

- [x] Content engine tests: 158 passed
- [x] Asset uploader tests: 179 passed
- [x] SDK tests: 505 passed
- [x] CLI tests: 921 passed
- [x] CLI prebuild integration tests: 46 passed
- [x] Builder Assets resource and diagnostics tests: 14 passed
- [x] Project runtime and ProjectSession tests: 133 passed
- [x] Builder API router tests: 30 passed
- [x] Protocol operation tests: 18 passed
- [x] Content engine build and declaration generation
- [x] Content engine, asset uploader, SDK, Builder, and CLI typechecks
- [x] Repository lint and formatting checks
- [x] Generated API and documentation checks
- [x] New materialization tests demonstrated failing with the
implementation disabled and passing after restoration

## Notes

- The representative five-post overview fixture shrinks from 2,122 bytes
to 1,197 bytes while returning the same sorted and paginated result.
- Materialization is intentionally limited to static, metadata-only
field queries without runtime asset fields. Unsupported or
data-incompatible queries safely use the existing runtime database.
- The complete `pnpm evaluations` agent suite was not run because
repository policy requires separate explicit approval.
- No database schema or migration changes are required.
 2026-07-31 13:16:21 +02:00
Oleg Isonen efbc3098c4 feat: add queryable Assets resources (#5946) 
Closes #5919
Ref #5920

## Outcome

Extend the existing Assets system resource with optional queries for
Markdown, JSON, and standard file metadata. Unconfigured Assets
resources retain the existing fetch-all behavior and activate none of
the content-query infrastructure.

Files are the authoring model. A standalone content engine converts an
immutable snapshot of only the files required by configured queries into
one bounded, deterministic runtime database. Builder preview, SSR, and
SSG use the same query semantics without requiring a database service
for a published site.

## Final architecture

```text
Markdown / JSON files
        │
        ▼
ContentSource adapter
(FS or Webstudio Assets)
        │ immutable snapshot
        ▼
ContentCompilationPlan
(union of configured query requirements)
        │
        ▼
ContentCompiler
(parse, normalize, discover fields, enforce budget)
        │
        ▼
ContentArtifact
(bounded, deterministic, immutable JSON)
        │
        ▼
ContentDatabase
(filter, OR groups, sort, paginate, project, read content)
        │
        ├── Builder preview
        ├── SSR runtime
        └── SSG build only
```

The authoring/storage layer owns mutable files. The content engine
consumes immutable snapshots and produces immutable, disposable
artifacts.


Query authoring is a separate OpenAPI-driven path:

```text
Assets REST OpenAPI operation
+ standard request JSON Schema
+ dynamic field paths
        │
        ▼
Generic OpenAPI adapter
        │ resolved schemas, controls, paths, and transport mapping
        ▼
Generic query-builder core and React renderer
        │ arbitrary provider-defined value
        ▼
Assets REST request adapter
        │
        ▼
{ query: value }
```

The selected standard OpenAPI operation is the only source of query UI
semantics. Request parameters and JSON Schema describe the editable
data; the generic renderer derives controls and owns layout.

### Activation and compilation

- The compiler does not index every asset. It combines all reachable
configured Assets-resource queries into one `ContentCompilationPlan`.
- Static constraints such as folder, path, and file type select
candidate files. Dynamic values such as a route-bound slug cannot reduce
the build-time candidate set.
- The plan contains the union of fields used by filtering, sorting,
routing, projection, excerpts, and requested full content.
- Documents shared by multiple resources are stored once in one
deployment-wide runtime database.
- Markdown/frontmatter, JSON properties, excerpts, and full bodies are
prepared only when required.
- No configured content query means no parsing, artifact, or shipped
runtime dependency. Fresh generated apps need no content-runtime
cleanup; incremental generation removes stale integration if a
previously configured query is deleted.

### Runtime database limit

- `CONTENT_DATABASE_MAX_BYTES` limits the complete serialized runtime
database, not source files, images, or unrelated assets.
- The default is `500 * 1024` bytes and is presented as 500 KiB.
- Counted data includes indexed metadata, frontmatter/JSON properties,
excerpts, the field catalog, serialization overhead, and full Markdown
bodies only when requested.
- Prepublish validation builds and serializes the exact database that
will be attached to the immutable build, so it knows the unbounded size,
bounded size, included documents, omitted documents, and affected
resources.
- Overflow never stores partial documents. Candidates are ordered by
`createdAt` descending, canonical path ascending, and stable document ID
ascending. Missing dates follow dated documents. A document that does
not fit is skipped while smaller later documents are still considered.
- Exceeding the limit warns but does not block publishing.
- Builder preview can report exact bounded versus unbounded results when
dynamic expressions have concrete preview values. Prepublish warnings
remain conservative for future route values.

### Snapshot consistency

The source contract guarantees a consistent compilation, while adapters
provide the mechanism:

- Webstudio Assets captures asset metadata and immutable content
references.
- Filesystem sources capture file identity and verify files did not
change before accepting the artifact.
- A changed source retries once, then fails clearly instead of producing
a mixed-revision database.
- This requires no universal snapshot service, source-tree copy, or new
versioned-asset infrastructure.

### Package ownership

- `@webstudio-is/content-engine`: runtime artifact schema, query
execution, field catalog, compact runtime statistics.
- `@webstudio-is/content-engine/compiler`: source snapshots, compilation
plans, Markdown/JSON parsing, deterministic compilation, byte
accounting, and compiler diagnostics. This entry exists to keep
Remark/YAML/future MDX dependencies out of runtime bundles.
- `@webstudio-is/query-builder`: resolves standard OpenAPI operation
schemas into an internal provider-neutral editor model, validation,
mutations, and source serialization. It owns no Assets,
provider-authored UI contract, or fixed REST-query semantics.
- `@webstudio-is/query-builder/react`: container-neutral schema renderer
whose layout is renderer policy. It remains separate from the content
engine.
- SDK: Webstudio Assets/Resource domain models and mapping Webstudio
expressions into generic content requirements.
- Protocol: external Assets REST/OpenAPI transport contracts.
- Asset uploader/repository: asset and folder mutations, generic
metadata, immutable byte references, and storage access. Its hosted
adapter supplies Webstudio Assets snapshots and parsed-metadata cache
persistence to the content engine; parsing, artifact creation, and query
semantics remain engine-owned.
- Builder and generated-app integrations decide when to compile, cache,
embed, load, or discard a database.

### Runtime lifecycle

- Builder caches immutable `ContentDatabase` instances by source
revision plus compilation-plan hash and coalesces concurrent
compilation. Matching operations reuse the same instance; field
discovery uses a dedicated minimal all-properties plan that excludes
excerpts and bodies.
- SSR attaches the validated bounded database to the build and includes
the small runtime only when required.
- SSG uses the database only while generating pages and omits it and the
runtime from final output unless a future runtime-query feature requires
them.
- Local development compiles from the filesystem and uses identical
query semantics.
- Published pages query their local artifact; they do not call Builder
REST endpoints.
- Published artifacts need no external index store, invalidation,
garbage collection, SaaS content proxy, new bucket, or new environment
variable.

## Implemented foundation

These existing parts remain useful and should be moved behind the final
ownership boundaries rather than reimplemented.

- [x] Upgrade the existing Assets system resource in place and preserve
unconfigured fetch-all behavior.
- [x] Implement typed filtering, nested boolean groups, deterministic
sorting, pagination, projections, dynamic values, and content hydration.
- [x] Add Markdown frontmatter, excerpt, JSON-property, and
field-catalog extraction.
- [x] Add reusable query-control implementations and a separately
imported React UI as a foundation for the final OpenAPI-driven contract.
- [x] Add Assets query authoring and dynamic field discovery as existing
behavior to migrate into the Assets OpenAPI operation.
- [x] Add synchronized visual and editable expression authoring.
- [x] Unify Assets REST operations under `/rest/assets/*`.
- [x] Define authenticated collection, item, content, upload, folder,
query, field-catalog, and OpenAPI operations.
- [x] Support Builder cookie authentication and public API token
authentication.
- [x] Add Builder preview, MCP/CLI operations, SSR execution, SSG path
generation, and conditional activation.
- [x] Add revision-keyed parsed metadata persistence for the hosted
Builder.
- [x] Add focused query, API, authorization, Builder, CLI, SSR, SSG,
migration, and fixture coverage.

## Remaining implementation plan

### 1. Establish the content engine

- [x] Rename the generic `@webstudio-is/asset-resource` implementation
to `@webstudio-is/content-engine`.
- [x] Move existing behavior instead of creating a parallel
implementation.
- [x] Keep the primary entry runtime-only and add only the compiler
entry needed for bundle separation.
- [x] Remove the old package after all consumers migrate.
- [x] Ensure the engine has no Builder, SDK, asset-uploader, PostgreSQL,
R2, or deployment-target dependency.

### 2. Define source snapshots and compilation plans

- [x] Introduce `ContentSource.openSnapshot()` and a revision-bearing
`ContentSourceSnapshot`.
- [x] Include canonical path, stable document ID, content type,
immutable content reference, size, and optional `createdAt`.
- [x] Implement Webstudio Assets and filesystem snapshot guarantees
using their adapter-specific mechanisms.
- [x] Introduce `ContentCompilationPlan` for the union of reachable
configured resource requirements.
- [x] Separate statically selectable candidates from dynamic runtime
predicates.
- [x] Add mutation-during-compilation tests for replacement, deletion,
folder movement, and filesystem changes.

### 3. Separate storage from content processing

- [x] Keep upload, replacement, folders, byte storage, revisions, and
generic metadata in the Assets repository/uploader.
- [x] Move parsing, excerpts, field discovery, query execution, and
artifact creation into the content engine.
- [x] Remove content-index coordination from asset mutation business
logic.
- [x] Preserve original source `createdAt` across content replacement
and expose it as optional queryable metadata.
- [x] Keep hosted parsed-document persistence behind the content
compiler's optional cache interface.

### 4. Build deterministic compilation

- [x] Move the existing Markdown, frontmatter, JSON, canonicalization,
hashing, document-building, and tests into the compiler.
- [x] Keep frontmatter behind a dedicated module and continue using the
existing Remark/Micromark pipeline.
- [x] Parse only candidate files and only the tiers required by the
compilation plan.
- [x] Deduplicate documents and fields required by multiple resources.
- [x] Define a versioned, deterministic `ContentArtifact` with integrity
information.
- [x] Keep verbose parse and omission diagnostics outside the runtime
artifact.
- [x] Reject incompatible artifact versions cleanly.

### 5. Enforce the deployment-wide database budget

- [x] Replace the old asset-index limit with
`CONTENT_DATABASE_MAX_BYTES`, defaulting to 500 KiB.
- [x] Have applications read configuration and pass the byte limit into
the compiler.
- [x] Measure the exact serialized database, including selected bodies
and overhead.
- [x] Implement deterministic skip-and-continue packing of complete
documents.
- [x] Return compiler diagnostics with unbounded/bounded bytes,
included/omitted counts, omitted identities, largest contributors, and
affected resources.
- [x] Expose only compact artifact statistics at runtime through
`ContentDatabase.getStats()`.
- [x] Add exact boundary, deterministic ordering, deduplication, and
oversized-document tests.

### 6. Finalize the runtime database

- [x] Define `ContentDatabase` over an immutable artifact.
- [x] Preserve filtering, nested AND/OR groups, sorting, offset/limit,
projection, field references, and content retrieval.
- [x] Expose `getFieldCatalog()` and compact `getStats()`.
- [x] Keep Markdown/YAML/compiler dependencies out of the runtime entry.
- [x] Add execution guards and maximum-size runtime benchmarks.

### 7. Make query authoring OpenAPI-driven

The selected standard OpenAPI operation is the complete query-authoring
contract. The query builder resolves parameters and request-body JSON
Schema, derives its internal editor model, and applies renderer-owned
layout policy. It must not embed or fetch a second capabilities/UI
language through `x-webstudio-query` or another side channel.

- [x] Remove `x-webstudio-query` and all vendor query-definition
metadata from the Assets OpenAPI operation.
- [x] Resolve local OpenAPI references and derive the editable request
value, defaults, validation, labels, choices, arrays, objects, and
unions solely from standard operation parameters and request-body JSON
Schema.
- [x] Represent dynamic Assets field paths with standard JSON Schema
vocabulary in the `queryAssets` request schema.
- [x] Keep the normalized editor representation internal to
query-builder; do not expose another provider-authored capabilities
contract.
- [x] Remove provider-controlled layout metadata such as `group`; make
compact scalar packing and all other layout renderer policy.
- [x] Keep Webstudio expression binding as a consumer/editor concern
layered over schema values, not part of the OpenAPI contract.
- [x] Preserve the existing Assets filter, sort, output selection,
full-content/range selection, limit/offset, editable source, and
configured-field behavior.
- [x] Keep the Assets resource adapter responsible for serializing the
edited query into its request body.
- [x] Remove the obsolete Assets query-capabilities endpoint, URL, and
UI-side capability parsing.
- [x] Add a non-Assets OpenAPI proof with differently named parameters
and no vendor metadata.
- [x] Verify the generic packages contain no Assets imports, fixed
Assets property contract, or provider-authored layout model.
- [x] Update focused core, React, SDK, protocol/OpenAPI, Builder, and
source round-trip tests.

### 8. Manage ContentDatabase lifecycle in Builder

- [x] Cache databases by source revision plus normalized
compilation-plan hash.
- [x] Coalesce concurrent compilation requests.
- [x] Reuse cached database instances across field discovery,
autocomplete, query preview, and response preview when their normalized
compilation plans match.
- [x] Replace it when source revision or query requirements change.
- [x] Preserve the optional revision-keyed hosted parsed-document cache
without making it a source of truth.
- [x] Show exact preview warnings when concrete values reveal missing
bounded results.

### 9. Materialize and verify immutable builds

- [x] Compile and serialize the exact deployment-wide database during
prepublish validation.
- [x] Attach that validated bounded database to the immutable build
without rebuilding it from later asset state.
- [x] Show non-blocking prepublish warnings with database bytes,
omissions, and potentially affected resources.
- [x] Avoid claiming exact future route matches when dynamic values are
unknown.
- [x] Include the database/runtime only when a reachable configured
query requires them.
- [x] Ensure detail resources requesting Markdown content include all
statically eligible bodies needed by dynamic routes.

### 10. Integrate all targets

- [x] SSR: load the build's bounded database and conditionally include
the small runtime.
- [x] SSG: query the same database during generation, then omit
database/runtime output when no runtime query is needed.
- [x] Local/CLI: compile from the filesystem source with the same
semantics.
- [x] Preserve overview pagination, alternative route fields,
draft-as-user-data behavior, and full article rendering.
- [x] Verify ordinary Assets and fresh query-free sites perform no
parser, artifact, content-runtime dependency install, or cleanup work.

### 11. Correct API and package ownership

- [x] Move external Assets REST/OpenAPI contracts to
`@webstudio-is/protocol`.
- [x] Keep Webstudio domain models and Assets system-resource
configuration in the SDK.
- [x] Keep both cookie and token authentication behavior on the unified
Assets REST surface.
- [x] Remove pass-through route wrappers/reexports that add no transport
behavior.
- [x] Keep published runtime queries local to the artifact rather than
routing them through Builder REST.

### 12. Remove superseded deployment infrastructure

- [x] Remove external deployment-index upload, fetch, invalidation,
cleanup, and garbage-collection code.
- [x] Remove the SaaS asset-origin adapter requirement; close
webstudio-is/webstudio-saas#652 as superseded because it has no
independent purpose.
- [x] Remove index-bucket compatibility configuration and avoid adding
Infisical variables.
- [x] Remove obsolete external-index paths, package dependencies,
routes, scripts, and tests.
- [x] Consolidate unreleased intermediate migrations where safe and
retain only schema required by the final hosted cache/API design.

### 13. Verification and release readiness

- [x] Move and preserve existing tests while changing ownership; add new
tests only for changed contracts and behavior.
- [x] Run content-engine compiler/runtime tests, query-builder tests,
SDK/protocol/API tests, Builder tests, CLI tests, and affected
typechecks.
- [x] Run formatting, generated-file idempotence, fixtures, clean
Supabase-compatible migrations, E2E shards, and complete agent
evaluations.
- [x] Verify SSR and SSG conditional inclusion and query-free generated
package manifests.
- [x] Run the five-article MCP blog workflow: authoring, field
discovery, overview, dynamic detail routes, full Markdown rendering,
publishing, and published-site verification.
- [x] Run a maximum-budget database test and a
mutation-during-compilation test.
- [x] Perform a final architecture, security, API compatibility,
migration, bundle, correctness, and DRY review.
- [x] Reconcile every checkbox and replace the verification section with
results from the final architecture before release.

## Future MDX work

MDX is intentionally separate from this release.

- The compiler contract should allow a future build-only MDX parser.
- MDX compilation must not enter the published runtime.
- Embedded JSX must use an explicit compiled representation rather than
runtime source evaluation.
- Webstudio components, tokens, styles, and component allowlists remain
Webstudio integration concerns.
- The future MDX work should reuse `ContentArtifact` and
`ContentDatabase` where practical.

## Superseded design

The following earlier design choices are no longer the target
architecture:

- GROQ or GraphQL as the Assets query language.
- One external or long-lived index object per resource/query.
- A deployment index fetched by the Worker from SaaS/CDN infrastructure.
- Lazy production reads of source Markdown bodies.
- A separate index bucket, asset origin, invalidation service, or
garbage collector.
- A 4 MiB published index limit.
- Rebuilding an old build's database from the current mutable asset set.
- Bundling query infrastructure into projects without a configured
content query.

## Verification baseline

The embedded runtime-database architecture previously passed the checks
below. The OpenAPI-driven query-authoring change additionally passed:

- Query-builder core: 4 files / 19 tests.
- Query-builder React: 1 file / 7 tests.
- SDK Assets source/config: 1 file / 7 tests.
- Protocol Assets OpenAPI: 1 file / 9 tests, including independent
OpenAPI validation.
- Builder Assets OpenAPI description: 1 file / 5 tests.
- Query-builder, React, SDK, protocol, and Builder typechecks.
- Package-boundary validation and formatting of every affected file.

Local verification baseline:

- Content engine: 15 files / 104 tests.
- Asset uploader: 24 files / 175 tests.
- Builder: 204 files / 2,423 tests.
- CLI: 54 files / 866 tests.
- Complete workspace typecheck, formatting, lint, package boundaries,
generated fixtures, and fixture builds.
- The complete migration history applied successfully to a fresh
Supabase-compatible database, including every migration after the
consolidated Assets migration.
- Builder production build and full local E2E suite passed.
- Complete agent evaluations passed: authenticated page (170,859 ms),
design input (207,101 ms), and font assets (67,050 ms).
- Final review additionally verified correct REST edit/view/build
permits, cookie and public-token read paths, immutable snapshot retries,
exact database budgeting, query-free dependency omission, and SSR/SSG
conditional runtime inclusion.

Final staging verification passed against the embedded runtime-database
architecture: MCP uploaded and updated five Markdown articles,
discovered dynamic frontmatter fields, previewed a five-item sorted
overview and one full-body slug query, updated both Assets resources,
found zero binding issues, rendered `/blog` and `/blog/dynamic-routes`
at HTTP 200, published build version 266, and verified both published
routes at HTTP 200 with the expected article metadata and Markdown body.
The workflow also found and fixed offset-form asset timestamps failing
canonical datetime validation and made MCP `preview.start({ port: 0 })`
correctly select an available local port.

## Release constraints

- No new SaaS content service, bucket, or environment variable.
- No published-site database dependency.
- No parser/compiler dependency in query-free sites or runtime-only
bundles.
- The validated content database is immutable build output.
- Publishing warnings do not block deployment.
- The selected standard OpenAPI operation and its request JSON Schema
are the only source of query UI semantics.
 2026-07-31 00:17:46 +02:00
Oleg Isonen c0687e3323 fix: Accept CSS strings in animation keyframes (#5956) 
## Summary

Allow MCP `update-props` callers to configure `animationAction`
keyframes with normal CSS strings, including values such as `transform:
"translateX(-75%)"`.

Use one SDK-owned animation authoring schema in both the Builder UI and
MCP so CSS parsing, timing defaults, and validation cannot drift between
entry points.

## Root cause

The Builder UI and MCP did not share an authoring contract. Builder
controls produced Webstudio's structured style values and animation
factories supplied timing objects, while MCP passed its input directly
toward the persisted prop schema. As a result, normal CSS strings failed
deep Zod validation and omitted timing objects were rejected.

## Implementation

- Add a shared `createAnimationActionInput` schema factory to the SDK.
- Inject the CSS parser into the schema factory to avoid adding an `sdk`
→ `css-data` dependency cycle.
- Use the shared schema in both Builder animation validation and MCP
`update-props`.
- Accept raw CSS strings or existing structured Webstudio values in
`keyframes[].styles`.
- Parse string values before persistence and default omitted animation
`timing` to an empty options object.
- Update Animation Group guidance and regenerate the runtime operation
contract.
- Preserve strict validation for existing structured animation style
values.

## Checklist

- [x] Add regression coverage for the reported horizontal scroll
payload.
- [x] Verify the regression fails before implementation and passes after
it.
- [x] Add SDK coverage for shared animation authoring normalization.
- [x] Use the shared authoring schema in Builder and MCP.
- [x] Verify package boundaries and generated runtime contracts.
- [x] Run affected typechecks, targeted tests, lint, and formatting.
- [x] Run the complete agent evaluation suite.
- [ ] Resolve evaluation read-budget failures: `authenticated-page-v1`
and `design-input-v1` failed only `boundedReads`.

## Validation

Passed:

- `pnpm --filter @webstudio-is/sdk test --
src/schema/animation-schema.test.ts`
- `pnpm --filter @webstudio-is/project-build test --
src/runtime/props.test.ts src/mcp.test.ts`
- `pnpm --filter @webstudio-is/sdk typecheck`
- `pnpm --filter @webstudio-is/project-build typecheck`
- `pnpm --filter @webstudio-is/builder typecheck`
- `pnpm check:package-boundaries`
- `pnpm exec oxlint --deny-warnings <changed source and test files>`
- `pnpm exec oxfmt --check <changed source and test files>`
- `pnpm check:generated-api`
- `font-assets-v1` agent evaluation

Agent evaluation findings:

- `authenticated-page-v1`: failed only `boundedReads`; all functional,
audit, binding, privacy, and visual checks passed.
- `design-input-v1`: failed only `boundedReads`; all functional, audit,
responsive, design-system, privacy, and visual checks passed.
- `font-assets-v1`: passed all checks.
 2026-07-29 17:37:56 +02:00
Oleg Isonen 0b192c7802 fix: Sort style sources with temporary local source (#5949) 
## Summary

- Allow style sources to be reordered when the displayed local source
has not been persisted yet.
- Submit only attached style source IDs to the runtime reorder
operation.
- Add regression coverage for reordering tokens alongside a temporary
local source.

## Root cause

The style panel displays a temporary local style source until local
styles are persisted. Sorting included that temporary ID in the runtime
mutation, but the runtime accepts only attached style source IDs and
rejected the entire reorder.

## Todo

- [x] Filter temporary style source IDs from reorder mutations.
- [x] Add regression coverage.
- [x] Run targeted tests, lint, formatting, and Builder typecheck.

## Verification

- `pnpm --filter @webstudio-is/builder test --
app/builder/features/style-panel/style-source-section.test.ts` — 12
tests passed
- `pnpm exec oxlint --deny-warnings
apps/builder/app/builder/features/style-panel/style-source-section.tsx
apps/builder/app/builder/features/style-panel/style-source-section.test.ts`
- `pnpm exec oxfmt --check
apps/builder/app/builder/features/style-panel/style-source-section.tsx
apps/builder/app/builder/features/style-panel/style-source-section.test.ts`
- `pnpm --filter @webstudio-is/builder typecheck`
 2026-07-24 15:45:48 +02:00
Oleg Isonen 7eb58664b0 fix: harden asset folder uploads and selection (#5939) 
## Summary

Fixes folder uploads and asset-manager selection behavior.

## Changes

- Preserve folder IDs for newly uploaded assets.
- Place duplicate assets into the dropped folder, including server-side
deduplication and assets already uploading.
- Handle browser `NotFoundError` cases while reading dropped
directories.
- Keep lasso selection anchored correctly while scrolling.
- Add `Cmd/Ctrl+A` to select all currently rendered assets and folders.
- Document the new shortcut in the keyboard shortcuts dialog.
- TIFF support was removed in the base branch.

## Verification

- [x] Asset upload and directory-drop tests
- [x] Asset-manager tests
- [x] Keyboard shortcuts dialog tests
- [x] Builder typecheck
- [x] Formatting and diff checks
 2026-07-21 16:35:04 +02:00
Oleg Isonen 085f705cc4 fix: remove unsupported TIFF assets (#5938) 
## Summary

- Remove `.tif` and `.tiff` from supported asset extensions.
- Stop categorizing TIFF files as images in the asset manager.
- Update SDK URL coverage to reflect the supported direct-served image
formats.

## Why

The asset manager classified TIFF files as images, but the browser and
image optimization pipeline cannot reliably render TIFF thumbnails.
Treating TIFF as supported created broken previews.

## Verification

- SDK asset tests: 95 passed
- Formatting completed
- Diff checked against `main`
 2026-07-21 15:44:14 +02:00
Oleg Isonen 54aa7be145 fix: Make MCP authoring reliable across assets, styles, and embeds (#5932) 
Closes #5924
Closes #5925
Closes #5926
Closes #5927
Closes #5928
Closes #5929

## Outcome

Makes MCP and CLI authoring predictable across project-wide CSS
variables, font uploads, retried asset uploads, destructive asset
selection, design-token collisions, and SVG HtmlEmbed edits.

## Delivered

### Project-global CSS variables

- [x] `define-css-variable` writes to `ROOT_INSTANCE_ID` / `:root`
instead of the home-page Body.
- [x] Breakpoint and overwrite behavior is preserved, and compact
results identify the root scope.
- [x] Existing body-local declarations remain untouched.
- [x] Runtime and Builder integration tests cover root selection and
generated project behavior.

### Font metadata and upload overrides

- [x] Static fonts prefer authoritative `OS/2.usWeightClass`.
- [x] Fallback matching normalizes spaces, hyphens, and underscores and
checks specific aliases before shorter ones.
- [x] Explicit format and partial metadata overrides travel through
HTTP, filesystem, and S3 upload paths and are validated after merging
with detected metadata.
- [x] Regression tests cover ExtraBold, Extra-Bold, Semi Bold Italic,
Ultra Light, valid weight classes, invalid weight classes, partial
overrides, and invalid overrides.

### Idempotent asset uploads

- [x] API uploads compute a project-scoped SHA-256 content hash.
- [x] Sequential and concurrent duplicate uploads reuse the existing
asset and return `deduplicated: true`.
- [x] `force: true` bypasses deduplication for intentional duplicates.
- [x] A partial unique index enforces one non-null content hash per
uploader project.
- [x] Stale reservations are cleaned up and active concurrent uploads
are bounded by polling and an actionable retry error.
- [x] HTTP, protocol, uploader, and CLI tests cover hash reuse,
uniqueness races, force propagation, and result markers.

The migration does not backfill historical files. Existing assets begin
participating in deduplication when uploaded through the new path;
matching content keeps the existing asset's metadata rather than
silently rewriting it.

### Consistent asset inputs and permissions output

- [x] `delete-asset` exposes exact `assetIds` and explicit
`assetIdPrefixes`.
- [x] The legacy `assetIdsOrPrefixes` input remains accepted and is
marked deprecated.
- [x] Exact ids no longer accidentally select longer prefixed ids.
- [x] `webstudio permissions` prints a readable role/capability summary
by default while `--json` remains unchanged for machine callers.
- [x] Help and discovery describe JSON output as optional.
- [x] Runtime, CLI, metadata, and generated-contract tests cover
canonical and compatibility behavior.

### Deterministic fragment design tokens

- [x] Fragment insertion reconciles incoming tokens against project
tokens and earlier tokens in the same fragment.
- [x] Equal name/style tokens are reused instead of creating suffixed
twins.
- [x] Same-name/different-style fragments require an explicit `ours`,
`theirs`, or `merge` policy at the public runtime boundary.
- [x] Builder's established internal paste behavior retains its explicit
incoming-value default.
- [x] `create-design-token` accepts documented CSS strings through the
property-aware CSS parser as well as typed style values.
- [x] Tests cover same-fragment reuse/conflicts, persisted tokens,
explicit policies, and CSS string parsing.

### Editable SVG HtmlEmbed content

- [x] HtmlEmbed validation uses parse5 parser errors and source
structure instead of serialization-string equality.
- [x] Valid inline SVG is accepted without silently normalizing authored
markup.
- [x] Fragment insertion now uses the same validator as prop update and
text replacement paths.
- [x] Malformed markup is rejected before mutation.
- [x] Regression tests cover realistic SVG, parser failures, and
no-partial-mutation behavior.

## Verification

- [x] `pnpm checks`: all workspace tests, typechecks, lint, package
boundaries, generated API/docs checks, and fixtures passed on the final
committed tree.
- [x] `pnpm --filter webstudio test:release-smoke`: packaged install,
compact comparison, preview, agent workflow, stdio MCP, and structured
commands passed.
- [x] Prisma schema validation passed with test connection variables.
- [x] Two minimally prompted weak-model subagents independently
exercised the changed CLI/MCP discovery and focused workflows.
- [x] The subagent-found stale `permissions --help` requirement was
fixed, regression-tested, and retested.
- [x] Final diff and worktree are clean.
 2026-07-21 14:36:54 +02:00
Oleg Isonen 918e569fd4 fix: Show publish warnings immediately (#5935) 
## Summary

Show client-side pre-publish audit findings as soon as the user starts
publishing, and apply the Builder's standard text styling to warning
banners.

## Root cause

The cloud publish audit ran inside an async form action, while the
static export audit ran inside an async transition. React deferred their
ordinary state updates until those async operations completed, so
warnings could appear only after publishing or export finished even
though validation itself ran synchronously on the client.

## Changes

- Run the cloud publish audit in the synchronous button click handler
before starting the async publish transition.
- Run the static export audit before starting its async transition.
- Keep warnings non-blocking while preserving blocking audit errors.
- Render warning-banner content with the design-system `Text` component.

## Checklist

- [x] Show cloud publish warnings before publishing starts
- [x] Show static export warnings before export starts
- [x] Style warning text consistently
- [x] Run Builder tests
- [x] Run Builder typecheck
- [x] Run lint and formatting checks

## Verification

- `pnpm --filter @webstudio-is/builder test` — 2,351 tests passed
- `pnpm --filter @webstudio-is/builder typecheck`
- `pnpm exec oxlint --deny-warnings
apps/builder/app/builder/features/publish/publish.tsx`
- `pnpm exec oxfmt
apps/builder/app/builder/features/publish/publish.tsx`
 2026-07-21 11:57:00 +02:00