admin/webstudio
1
0 Fork 0
mirror of https://github.com/webstudio-is/webstudio.git synced 2026-10-30 08:37:56 +09:00
Code Issues Packages Projects Releases Wiki Activity
6,651 Commits 122 Branches 310 Tags
0.294.0
Commit Graph
5 Commits
Author SHA1 Message Date
Oleg Isonen 4273b7001f refactor: Organize Builder development and test tooling (#6285) 
## Outcome

Builder development, backend, E2E, and CI tooling now share one
ownership model: Builder defines named commands and backend modes, while
CI only schedules those commands.

## Technical choices

- Group shared Compose files, PostgreSQL initialization, and backend
helpers under `apps/builder/backend`.
- Resolve `compose.<mode>.yaml` by convention instead of maintaining
Compose filename lists in scripts and workflows.
- Keep persistent development and disposable test Compose projects
separate by default, while allowing the standard `COMPOSE_PROJECT_NAME`
override.
- Make schema snapshots invalidate from discovered migration sources and
backend files.
- Move E2E behavior recipes into named Builder commands; retain concise
root proxies for developer entry points.
- Let Vitest allocate available browser ports instead of maintaining a
cross-package port registry.
- Keep test-suite scheduling explicit in `main.yml`, but move executable
test commands to root package scripts.
- Move fixture-size enforcement from inline GitHub JavaScript into a
locally runnable repository script.

## Todo

- [x] Consolidate Builder backend and Compose ownership.
- [x] Add named development, E2E, schema-cache, Supabase, and CI-shard
commands.
- [x] Remove Compose topology and behavior flags from CI.
- [x] Remove the static Vitest browser-port registry.
- [x] Extract fixture-size checking from the workflow.
- [x] Verify migration bootstrap, snapshot-backed E2E, tests, builds,
types, formatting, lint, generated contracts, and workflow syntax.
- [x] Review documentation impact: updated the E2E developer guide; no
user-facing `docs/` content is affected.

## Verification

- Full package, CLI, Builder core, and Builder app test suites
- Full workspace typecheck
- Builder production build
- Schema-cache generation through the complete migration history
- Supabase migration compatibility
- Snapshot-backed Playwright animation workflow
- Backend regression tests, including demonstrated fail/pass coverage
- All four Compose mode configurations
- `oxfmt`, `oxlint`, package boundaries, generated API, and generated
docs checks
- `actionlint` and Bash syntax checks

The fixture-size checker preserves the existing Linux `du` thresholds.
Its three inputs build locally; macOS allocation sizes are not directly
comparable to the Linux CI thresholds, so CI remains the authoritative
size result.
 2026-08-31 21:09:54 +02:00
Oleg Isonen 8718feb465 test: simplify browser testing infrastructure (#6271) 
## Summary

Unifies frontend testing around three intentional boundaries:

- Vitest Node projects for pure and server tests
- Vitest Browser Mode with Playwright/Chromium for DOM and component
tests
- Playwright Test for Builder end-to-end workflows

The Builder no longer uses jsdom. Ordinary E2E workflows use an
automatically managed isolated `page` fixture; explicit contexts remain
only for simultaneous identities and suite-scoped setup.

## Technical choices

- Split Builder tests into one Node project and independent browser
projects so each Chromium/Vite process has a bounded workload.
- Added shared Browser Mode configuration for the affected SDK React
packages.
- Preserve the Builder's production top-window bridge ownership; Browser
Mode uses explicit test-only installation seams.
- Centralized Playwright context options and kept action/navigation
timeouts bounded.
- Validate Playwright filters before Docker, migrations, browser
installation, or Builder builds start.
- Preserve the first failed E2E attempt's report, traces, screenshots,
and videos when CI retries a shard.
- Split static checks, typechecks, CLI tests, core package tests,
browser package tests, and Builder projects into parallel CI jobs with
five-minute hard ceilings.
- Distribute Builder E2E tests across ten isolated shards with two
workers; parallel suites create unique per-worker setup data.
- Reuse color contrast DOM/canvas samples so the exhaustive theme sweep
takes about four seconds instead of timing out at 30 seconds.
- The removed cost is jsdom behavior drift, oversized serial check jobs,
and unnecessary E2E startup for empty selections. Real browser startup
and E2E environment setup remain.

## Todo

- [x] Split Builder pure/server tests from browser/component tests.
- [x] Migrate Builder DOM tests from jsdom to Vitest Browser Mode.
- [x] Migrate affected SDK React DOM tests to the shared Browser Mode
setup.
- [x] Remove jsdom annotations, dependencies, and compatibility setup.
- [x] Replace ordinary manual E2E contexts with the isolated `page`
fixture.
- [x] Keep explicit contexts only for concurrent identities and
`beforeAll` setup.
- [x] Preserve setup authentication isolation from workflow pages.
- [x] Add bounded Playwright action and navigation timeouts.
- [x] Preserve first-attempt E2E failure evidence across CI retry.
- [x] Reject empty E2E selections before expensive environment setup.
- [x] Parallelize repository checks into bounded CI shards.
- [x] Isolate parallel E2E setup data across workers.
- [x] Remove the color contrast sweep bottleneck.
- [x] Update E2E contributor documentation.
- [x] Review product documentation impact; no user-facing product docs
are affected.
- [x] Run the full repository tests and typechecks.
- [x] Run formatting, lint, package-boundary, generated-file, and
fixture checks.
- [x] Complete final correctness and simplification review.

## Verification

- `pnpm --filter @webstudio-is/builder test`
- `pnpm -r typecheck`
- `pnpm lint`
- `pnpm check:package-boundaries`
- `pnpm check:generated-api`
- `pnpm check:generated-docs`
- `pnpm fixtures`
- Design-system: 222 tests pass locally.
- Builder: all seven Vitest projects pass locally.
- The formerly slow E2E shard passes locally in 1m55s.
- Full database-backed Builder E2E execution passes in CI.
- All CI jobs pass; the slowest completed in 4m58s.
- No fixture inputs/outputs or generated files changed.

Ref #6270
 2026-08-31 17:17:28 +02:00
Oleg Isonen f6c524572d feat: add Builder dark theme and rebuild colors 
## Summary

Rebuilds Webstudio’s color system, migrates the entire Builder to it,
and introduces a complete dark theme.

A small set of theme colors and contrast controls now drives the
interface through relative CSS colors:

```text
Theme colors and contrast
    ↓
Light/dark derivation bounds
    ↓
Derived color families
    ↓
Reusable semantic colors
    ↓
Component composition
    ↓
Builder interface
```

CSS is the runtime source of truth. Light and dark modes share the same
theme inputs, so changing a theme color automatically updates its
surfaces, content, borders, states, and component presentation. The
refactor turns dark mode into a first-class Builder theme rather than a
separately maintained palette.

This is an atomic migration. Legacy color names and compatibility
aliases will not remain after all consumers have moved.

## Scope

- The color source, derivation model, semantic vocabulary, validation,
typing, and Storybook overview.
- Every design-system component.
- Builder, dashboard, authentication, shared UI, Query Builder, stories,
and tests that consume the legacy color scale.
- Removal of the legacy Stitches color scale after its final consumer is
migrated.
- A complete dark theme across application UI while keeping authored
canvas content independent.
- Complete light and dark visual verification.
- Unified browser-test infrastructure: Vitest Browser Mode for Design
System browser contracts and standard Playwright Test for all Builder
E2E workflows.

Literal colors remain only where color is the actual content or belongs
to a deliberately separate system, such as user-authored values,
external previews, brand artwork, data visualization, functional canvas
overlays, and source assets. Every remaining exception must have an
explicit owner.

## Architecture decisions

- Theme authors control six color families and three bounded contrast
controls.
- Relative CSS colors derive the rest of the system.
- Dark mode changes derivation bounds instead of maintaining a second
independent palette.
- Components consume a small reusable semantic vocabulary.
- Component states and exceptional presentation are composed inside
components.
- Builder-specific categorical meaning uses focused composite recipes,
not intent semantics or a larger global vocabulary.
- TypeScript validates public semantic CSS-variable names but does not
generate runtime colors.
- No token-manager dependency, compatibility aliases, or
component-specific public color tokens.
- A new semantic or theme variable is added only when existing semantics
and local composition cannot represent a reusable cross-feature
decision.

## Todo

### Color-system foundation

- [x] Establish direct CSS as the runtime source of truth.
- [x] Define bounded theme colors and contrast controls.
- [x] Derive light and dark modes from the same theme inputs.
- [x] Reduce the semantic inventory to reusable decisions and explicit
accessibility pairs.
- [x] Generate only TypeScript names for public semantic CSS variables.
- [x] Validate layers, references, cycles, consumption, naming,
generated types, and browser-computed contrast.
- [x] Add the Storybook color-system overview and theme controls.
- [x] Document the architecture and its relationship to the public Craft
specification.

### Design-system migration

- [x] Replace every active legacy color consumer in design-system
components and Storybook chrome.
- [x] Compose component states locally without component-specific public
tokens.
- [x] Preserve intentional preview and sample-content colors.
- [x] Add complete positive, negative, warning, informative, and accent
accessibility pairings.
- [x] Validate light and dark scheme-bound references independently.
- [x] Verify the design-system package with typecheck, lint, formatting,
Storybook, and 222 tests.

### Inventory and contract

- [x] Inventory all 319 legacy references across 107 Builder, Query
Builder, story, test, and design-system files.
- [x] Classify every reference as a direct semantic replacement, local
composite recipe, documented literal exception, or removable code.
- [x] Record the intentional authored-content, external-preview,
functional-preview, data-visualization, brand, and source-asset
exceptions.
- [x] Treat the public semantic vocabulary as closed by default and
justify any proposed addition by cross-feature reuse and accessibility
need.
- [x] Keep Builder code limited to typed public variables accepted by
`cssVar()`.

### Theme infrastructure

- [x] Add `system` (default), `light`, and `dark` application
preferences.
- [x] Persist the preference in existing client settings and resolve
`system` through `matchMedia`.
- [x] Apply the resolved scheme before paint and respond to
operating-system changes.
- [x] Expose the persisted theme control under **View → Color scheme**.
- [x] Keep authored canvas documents independent from the application
theme.
- [x] Ensure portals, overlays, embedded surfaces, and third-party
controls receive the correct interface scheme.

### Generic interface migration

- [x] Migrate shared dialogs, project settings, notifications, controls,
and Query Builder.
- [x] Migrate dashboard, authentication, publishing, and shared
application UI.
- [x] Migrate the Builder loading state, topbar, sidebars, navigator,
workspace, footer, settings, pages, alerts, and menus.
- [x] Replace fixed interface literals alongside their owning surfaces.
- [x] Remove the old scrollbar variable and hardcoded loading color.

### Style panel and style-source recipes

- [x] Migrate ordinary style-panel inputs, labels, layouts, spacing,
alignment, menus, property states, and value editors.
- [x] Model style-source provenance, kind, interaction, and error as
separate axes.
- [x] Compose categorical style-source colors from neutral and
accent-relative colors.
- [x] Use negative semantics only for actual errors, not overwritten
provenance.
- [x] Reuse one feature-owned recipe across style-source controls,
badges, menus, and values without adding global tokens.

### Canvas and editors

- [x] Migrate canvas selection, reusable outlines, grid guides, resize
handles, labels, badges, and overlays.
- [x] Keep collaborator colors as data visualization while guaranteeing
readable labels.
- [x] Prevent Builder theme values from leaking into authored content.
- [x] Migrate editor surfaces, selections, autocomplete, lint states,
and tooltips.
- [x] Derive syntax visualization from neutral and accent-relative
colors; reserve intent colors for diagnostics.

### Explicit color domains

- [x] Isolate external search and social previews from the application
scheme.
- [x] Preserve user-authored CSS values, functional color previews, and
checkerboards.
- [x] Review brand surfaces independently in both schemes.
- [x] Document every retained literal in the exception ledger.

### Stories, tests, and legacy removal

- [x] Migrate all affected Builder stories and tests.
- [x] Replace the design-system test that intentionally references the
legacy scale.
- [x] Add theme-preference and scheme-resolution behavior tests.
- [x] Remove the legacy `design-tokens.color` registry.
- [x] Remove the Stitches `colors` scale and `rawTheme.colors`.
- [x] Remove color-name conversion code that exists only for the legacy
scale.
- [x] Add a focused repository check that rejects legacy references.
- [x] Confirm no compatibility mappings or deprecated aliases remain.

### Browser-test infrastructure

- [x] Run pure Design System tests in Node and browser-dependent tests
in Vitest Browser Mode with Playwright Chromium.
- [x] Migrate all 79 Builder E2E workflows from the custom harness to
standard Playwright Test.
- [x] Replace filename-based shard discovery with standard Playwright
sharding.
- [x] Preserve clean-backend shard retries and upload Playwright failure
artifacts in CI.
- [x] Remove the custom runner, registry, and timeout infrastructure.

### Verification and documentation

- [x] Compare light-mode visual regressions directly with `main`; review
every difference rather than accepting snapshots wholesale.
- [x] Add representative dark coverage for dashboard, Builder shell,
style panel, dialogs, editors, canvas tools, and status states.
- [x] Test default and extreme custom-theme presets.
- [x] Verify semantic and categorical-recipe contrast.
- [x] Test first paint, persistence, system changes, portals, and canvas
isolation.
- [x] Run affected tests, typechecks, lint, formatting, and Storybook
checks.
- [x] Review fixture impact; no fixture inputs or generated outputs are
affected.
- [x] Review authoritative `docs/` content and update it where needed.
- [x] Review the final diff and make this checklist reflect delivered
scope.

## Current verification

- Design-system typecheck and generated-type freshness: passed.
- Design-system tests: 222 passed, including the full theme-parameter
contrast sweep and browser component coverage.
- Builder typecheck and production build: passed.
- Theme preference and scheme-resolution tests: 10 passed.
- Scoped Oxlint, formatting, and diff checks: passed.
- Design-system Storybook build and representative light/dark visual
review: passed.
- Builder E2E: 80/80 Playwright entries passed (setup plus 79
workflows).

Ref #3818
Closes #6267
 2026-08-30 03:24:56 +01:00
Oleg Isonen 2ec78c183d feat: Add single-result modes to Assets queries (#6040) 
## Summary

- Add `many`, `one`, `first`, and `last` result modes to Assets queries
while keeping `many` as the backward-compatible default.
- Return direct items for single-result modes, validate deterministic
ordering, and report duplicate `one` matches as a structured
`MULTIPLE_RESULTS` error.
- Update Builder controls, preview/published runtime handling, SDK
resource encoding, MCP/CLI guidance, generated contracts, and fixtures.

## Implementation TODOs

- [x] Add result-mode schemas and core execution behavior.
- [x] Select single results before content hydration and preserve
deterministic ID tie-breaking.
- [x] Add structured duplicate-match transport errors.
- [x] Update Builder controls and hide/reset pagination for
single-result modes.
- [x] Keep existing settings first, then Output and Result, without a
duplicate Return label.
- [x] Preserve legacy saved resources with an implicit `many` default.
- [x] Update preview, published runtime, SDK, protocol, MCP, and CLI
behavior.
- [x] Regenerate runtime contracts, server metadata, CLI docs, and
fixtures.
- [x] Review documentation impact and update the authoritative MCP/CLI
guidance.
- [x] Add regression coverage and verify the new tests fail without the
implementation.
- [ ] Run the agent evaluation suite (not run; requires explicit user
approval).

## Compatibility and behavior

- Existing queries without `result` continue to behave as `many` and
retain the ID-keyed collection shape.
- `one` returns an item or `null`, but fails when more than one asset
matches.
- `first` and `last` require explicit sorting; the existing asset-ID
tie-breaker makes equal sort values deterministic.
- Single-result modes ignore and do not persist pagination controls.
Materialized collection queries remain collection-only; single-result
queries execute dynamically.

## Verification

- Content engine typecheck and full test suite (291 tests).
- Query builder, query-builder React, SDK, protocol, asset uploader,
project build, Builder, and CLI targeted typechecks/tests.
- Root lint.
- MCP documentation tests, typecheck, sync check, and generated-doc
checks.
- Generated API contract checks.
- Fixture generation pipeline.
- Final focused content-engine regression run (53 tests).
- Final DRY review: affected typechecks, focused tests, lint, package
boundaries, generated checks, and fixtures.

Closes #6039
 2026-08-07 19:13:19 +02:00
Oleg Isonen efbc3098c4 feat: add queryable Assets resources (#5946) 
Closes #5919
Ref #5920

## Outcome

Extend the existing Assets system resource with optional queries for
Markdown, JSON, and standard file metadata. Unconfigured Assets
resources retain the existing fetch-all behavior and activate none of
the content-query infrastructure.

Files are the authoring model. A standalone content engine converts an
immutable snapshot of only the files required by configured queries into
one bounded, deterministic runtime database. Builder preview, SSR, and
SSG use the same query semantics without requiring a database service
for a published site.

## Final architecture

```text
Markdown / JSON files
        │
        ▼
ContentSource adapter
(FS or Webstudio Assets)
        │ immutable snapshot
        ▼
ContentCompilationPlan
(union of configured query requirements)
        │
        ▼
ContentCompiler
(parse, normalize, discover fields, enforce budget)
        │
        ▼
ContentArtifact
(bounded, deterministic, immutable JSON)
        │
        ▼
ContentDatabase
(filter, OR groups, sort, paginate, project, read content)
        │
        ├── Builder preview
        ├── SSR runtime
        └── SSG build only
```

The authoring/storage layer owns mutable files. The content engine
consumes immutable snapshots and produces immutable, disposable
artifacts.


Query authoring is a separate OpenAPI-driven path:

```text
Assets REST OpenAPI operation
+ standard request JSON Schema
+ dynamic field paths
        │
        ▼
Generic OpenAPI adapter
        │ resolved schemas, controls, paths, and transport mapping
        ▼
Generic query-builder core and React renderer
        │ arbitrary provider-defined value
        ▼
Assets REST request adapter
        │
        ▼
{ query: value }
```

The selected standard OpenAPI operation is the only source of query UI
semantics. Request parameters and JSON Schema describe the editable
data; the generic renderer derives controls and owns layout.

### Activation and compilation

- The compiler does not index every asset. It combines all reachable
configured Assets-resource queries into one `ContentCompilationPlan`.
- Static constraints such as folder, path, and file type select
candidate files. Dynamic values such as a route-bound slug cannot reduce
the build-time candidate set.
- The plan contains the union of fields used by filtering, sorting,
routing, projection, excerpts, and requested full content.
- Documents shared by multiple resources are stored once in one
deployment-wide runtime database.
- Markdown/frontmatter, JSON properties, excerpts, and full bodies are
prepared only when required.
- No configured content query means no parsing, artifact, or shipped
runtime dependency. Fresh generated apps need no content-runtime
cleanup; incremental generation removes stale integration if a
previously configured query is deleted.

### Runtime database limit

- `CONTENT_DATABASE_MAX_BYTES` limits the complete serialized runtime
database, not source files, images, or unrelated assets.
- The default is `500 * 1024` bytes and is presented as 500 KiB.
- Counted data includes indexed metadata, frontmatter/JSON properties,
excerpts, the field catalog, serialization overhead, and full Markdown
bodies only when requested.
- Prepublish validation builds and serializes the exact database that
will be attached to the immutable build, so it knows the unbounded size,
bounded size, included documents, omitted documents, and affected
resources.
- Overflow never stores partial documents. Candidates are ordered by
`createdAt` descending, canonical path ascending, and stable document ID
ascending. Missing dates follow dated documents. A document that does
not fit is skipped while smaller later documents are still considered.
- Exceeding the limit warns but does not block publishing.
- Builder preview can report exact bounded versus unbounded results when
dynamic expressions have concrete preview values. Prepublish warnings
remain conservative for future route values.

### Snapshot consistency

The source contract guarantees a consistent compilation, while adapters
provide the mechanism:

- Webstudio Assets captures asset metadata and immutable content
references.
- Filesystem sources capture file identity and verify files did not
change before accepting the artifact.
- A changed source retries once, then fails clearly instead of producing
a mixed-revision database.
- This requires no universal snapshot service, source-tree copy, or new
versioned-asset infrastructure.

### Package ownership

- `@webstudio-is/content-engine`: runtime artifact schema, query
execution, field catalog, compact runtime statistics.
- `@webstudio-is/content-engine/compiler`: source snapshots, compilation
plans, Markdown/JSON parsing, deterministic compilation, byte
accounting, and compiler diagnostics. This entry exists to keep
Remark/YAML/future MDX dependencies out of runtime bundles.
- `@webstudio-is/query-builder`: resolves standard OpenAPI operation
schemas into an internal provider-neutral editor model, validation,
mutations, and source serialization. It owns no Assets,
provider-authored UI contract, or fixed REST-query semantics.
- `@webstudio-is/query-builder/react`: container-neutral schema renderer
whose layout is renderer policy. It remains separate from the content
engine.
- SDK: Webstudio Assets/Resource domain models and mapping Webstudio
expressions into generic content requirements.
- Protocol: external Assets REST/OpenAPI transport contracts.
- Asset uploader/repository: asset and folder mutations, generic
metadata, immutable byte references, and storage access. Its hosted
adapter supplies Webstudio Assets snapshots and parsed-metadata cache
persistence to the content engine; parsing, artifact creation, and query
semantics remain engine-owned.
- Builder and generated-app integrations decide when to compile, cache,
embed, load, or discard a database.

### Runtime lifecycle

- Builder caches immutable `ContentDatabase` instances by source
revision plus compilation-plan hash and coalesces concurrent
compilation. Matching operations reuse the same instance; field
discovery uses a dedicated minimal all-properties plan that excludes
excerpts and bodies.
- SSR attaches the validated bounded database to the build and includes
the small runtime only when required.
- SSG uses the database only while generating pages and omits it and the
runtime from final output unless a future runtime-query feature requires
them.
- Local development compiles from the filesystem and uses identical
query semantics.
- Published pages query their local artifact; they do not call Builder
REST endpoints.
- Published artifacts need no external index store, invalidation,
garbage collection, SaaS content proxy, new bucket, or new environment
variable.

## Implemented foundation

These existing parts remain useful and should be moved behind the final
ownership boundaries rather than reimplemented.

- [x] Upgrade the existing Assets system resource in place and preserve
unconfigured fetch-all behavior.
- [x] Implement typed filtering, nested boolean groups, deterministic
sorting, pagination, projections, dynamic values, and content hydration.
- [x] Add Markdown frontmatter, excerpt, JSON-property, and
field-catalog extraction.
- [x] Add reusable query-control implementations and a separately
imported React UI as a foundation for the final OpenAPI-driven contract.
- [x] Add Assets query authoring and dynamic field discovery as existing
behavior to migrate into the Assets OpenAPI operation.
- [x] Add synchronized visual and editable expression authoring.
- [x] Unify Assets REST operations under `/rest/assets/*`.
- [x] Define authenticated collection, item, content, upload, folder,
query, field-catalog, and OpenAPI operations.
- [x] Support Builder cookie authentication and public API token
authentication.
- [x] Add Builder preview, MCP/CLI operations, SSR execution, SSG path
generation, and conditional activation.
- [x] Add revision-keyed parsed metadata persistence for the hosted
Builder.
- [x] Add focused query, API, authorization, Builder, CLI, SSR, SSG,
migration, and fixture coverage.

## Remaining implementation plan

### 1. Establish the content engine

- [x] Rename the generic `@webstudio-is/asset-resource` implementation
to `@webstudio-is/content-engine`.
- [x] Move existing behavior instead of creating a parallel
implementation.
- [x] Keep the primary entry runtime-only and add only the compiler
entry needed for bundle separation.
- [x] Remove the old package after all consumers migrate.
- [x] Ensure the engine has no Builder, SDK, asset-uploader, PostgreSQL,
R2, or deployment-target dependency.

### 2. Define source snapshots and compilation plans

- [x] Introduce `ContentSource.openSnapshot()` and a revision-bearing
`ContentSourceSnapshot`.
- [x] Include canonical path, stable document ID, content type,
immutable content reference, size, and optional `createdAt`.
- [x] Implement Webstudio Assets and filesystem snapshot guarantees
using their adapter-specific mechanisms.
- [x] Introduce `ContentCompilationPlan` for the union of reachable
configured resource requirements.
- [x] Separate statically selectable candidates from dynamic runtime
predicates.
- [x] Add mutation-during-compilation tests for replacement, deletion,
folder movement, and filesystem changes.

### 3. Separate storage from content processing

- [x] Keep upload, replacement, folders, byte storage, revisions, and
generic metadata in the Assets repository/uploader.
- [x] Move parsing, excerpts, field discovery, query execution, and
artifact creation into the content engine.
- [x] Remove content-index coordination from asset mutation business
logic.
- [x] Preserve original source `createdAt` across content replacement
and expose it as optional queryable metadata.
- [x] Keep hosted parsed-document persistence behind the content
compiler's optional cache interface.

### 4. Build deterministic compilation

- [x] Move the existing Markdown, frontmatter, JSON, canonicalization,
hashing, document-building, and tests into the compiler.
- [x] Keep frontmatter behind a dedicated module and continue using the
existing Remark/Micromark pipeline.
- [x] Parse only candidate files and only the tiers required by the
compilation plan.
- [x] Deduplicate documents and fields required by multiple resources.
- [x] Define a versioned, deterministic `ContentArtifact` with integrity
information.
- [x] Keep verbose parse and omission diagnostics outside the runtime
artifact.
- [x] Reject incompatible artifact versions cleanly.

### 5. Enforce the deployment-wide database budget

- [x] Replace the old asset-index limit with
`CONTENT_DATABASE_MAX_BYTES`, defaulting to 500 KiB.
- [x] Have applications read configuration and pass the byte limit into
the compiler.
- [x] Measure the exact serialized database, including selected bodies
and overhead.
- [x] Implement deterministic skip-and-continue packing of complete
documents.
- [x] Return compiler diagnostics with unbounded/bounded bytes,
included/omitted counts, omitted identities, largest contributors, and
affected resources.
- [x] Expose only compact artifact statistics at runtime through
`ContentDatabase.getStats()`.
- [x] Add exact boundary, deterministic ordering, deduplication, and
oversized-document tests.

### 6. Finalize the runtime database

- [x] Define `ContentDatabase` over an immutable artifact.
- [x] Preserve filtering, nested AND/OR groups, sorting, offset/limit,
projection, field references, and content retrieval.
- [x] Expose `getFieldCatalog()` and compact `getStats()`.
- [x] Keep Markdown/YAML/compiler dependencies out of the runtime entry.
- [x] Add execution guards and maximum-size runtime benchmarks.

### 7. Make query authoring OpenAPI-driven

The selected standard OpenAPI operation is the complete query-authoring
contract. The query builder resolves parameters and request-body JSON
Schema, derives its internal editor model, and applies renderer-owned
layout policy. It must not embed or fetch a second capabilities/UI
language through `x-webstudio-query` or another side channel.

- [x] Remove `x-webstudio-query` and all vendor query-definition
metadata from the Assets OpenAPI operation.
- [x] Resolve local OpenAPI references and derive the editable request
value, defaults, validation, labels, choices, arrays, objects, and
unions solely from standard operation parameters and request-body JSON
Schema.
- [x] Represent dynamic Assets field paths with standard JSON Schema
vocabulary in the `queryAssets` request schema.
- [x] Keep the normalized editor representation internal to
query-builder; do not expose another provider-authored capabilities
contract.
- [x] Remove provider-controlled layout metadata such as `group`; make
compact scalar packing and all other layout renderer policy.
- [x] Keep Webstudio expression binding as a consumer/editor concern
layered over schema values, not part of the OpenAPI contract.
- [x] Preserve the existing Assets filter, sort, output selection,
full-content/range selection, limit/offset, editable source, and
configured-field behavior.
- [x] Keep the Assets resource adapter responsible for serializing the
edited query into its request body.
- [x] Remove the obsolete Assets query-capabilities endpoint, URL, and
UI-side capability parsing.
- [x] Add a non-Assets OpenAPI proof with differently named parameters
and no vendor metadata.
- [x] Verify the generic packages contain no Assets imports, fixed
Assets property contract, or provider-authored layout model.
- [x] Update focused core, React, SDK, protocol/OpenAPI, Builder, and
source round-trip tests.

### 8. Manage ContentDatabase lifecycle in Builder

- [x] Cache databases by source revision plus normalized
compilation-plan hash.
- [x] Coalesce concurrent compilation requests.
- [x] Reuse cached database instances across field discovery,
autocomplete, query preview, and response preview when their normalized
compilation plans match.
- [x] Replace it when source revision or query requirements change.
- [x] Preserve the optional revision-keyed hosted parsed-document cache
without making it a source of truth.
- [x] Show exact preview warnings when concrete values reveal missing
bounded results.

### 9. Materialize and verify immutable builds

- [x] Compile and serialize the exact deployment-wide database during
prepublish validation.
- [x] Attach that validated bounded database to the immutable build
without rebuilding it from later asset state.
- [x] Show non-blocking prepublish warnings with database bytes,
omissions, and potentially affected resources.
- [x] Avoid claiming exact future route matches when dynamic values are
unknown.
- [x] Include the database/runtime only when a reachable configured
query requires them.
- [x] Ensure detail resources requesting Markdown content include all
statically eligible bodies needed by dynamic routes.

### 10. Integrate all targets

- [x] SSR: load the build's bounded database and conditionally include
the small runtime.
- [x] SSG: query the same database during generation, then omit
database/runtime output when no runtime query is needed.
- [x] Local/CLI: compile from the filesystem source with the same
semantics.
- [x] Preserve overview pagination, alternative route fields,
draft-as-user-data behavior, and full article rendering.
- [x] Verify ordinary Assets and fresh query-free sites perform no
parser, artifact, content-runtime dependency install, or cleanup work.

### 11. Correct API and package ownership

- [x] Move external Assets REST/OpenAPI contracts to
`@webstudio-is/protocol`.
- [x] Keep Webstudio domain models and Assets system-resource
configuration in the SDK.
- [x] Keep both cookie and token authentication behavior on the unified
Assets REST surface.
- [x] Remove pass-through route wrappers/reexports that add no transport
behavior.
- [x] Keep published runtime queries local to the artifact rather than
routing them through Builder REST.

### 12. Remove superseded deployment infrastructure

- [x] Remove external deployment-index upload, fetch, invalidation,
cleanup, and garbage-collection code.
- [x] Remove the SaaS asset-origin adapter requirement; close
webstudio-is/webstudio-saas#652 as superseded because it has no
independent purpose.
- [x] Remove index-bucket compatibility configuration and avoid adding
Infisical variables.
- [x] Remove obsolete external-index paths, package dependencies,
routes, scripts, and tests.
- [x] Consolidate unreleased intermediate migrations where safe and
retain only schema required by the final hosted cache/API design.

### 13. Verification and release readiness

- [x] Move and preserve existing tests while changing ownership; add new
tests only for changed contracts and behavior.
- [x] Run content-engine compiler/runtime tests, query-builder tests,
SDK/protocol/API tests, Builder tests, CLI tests, and affected
typechecks.
- [x] Run formatting, generated-file idempotence, fixtures, clean
Supabase-compatible migrations, E2E shards, and complete agent
evaluations.
- [x] Verify SSR and SSG conditional inclusion and query-free generated
package manifests.
- [x] Run the five-article MCP blog workflow: authoring, field
discovery, overview, dynamic detail routes, full Markdown rendering,
publishing, and published-site verification.
- [x] Run a maximum-budget database test and a
mutation-during-compilation test.
- [x] Perform a final architecture, security, API compatibility,
migration, bundle, correctness, and DRY review.
- [x] Reconcile every checkbox and replace the verification section with
results from the final architecture before release.

## Future MDX work

MDX is intentionally separate from this release.

- The compiler contract should allow a future build-only MDX parser.
- MDX compilation must not enter the published runtime.
- Embedded JSX must use an explicit compiled representation rather than
runtime source evaluation.
- Webstudio components, tokens, styles, and component allowlists remain
Webstudio integration concerns.
- The future MDX work should reuse `ContentArtifact` and
`ContentDatabase` where practical.

## Superseded design

The following earlier design choices are no longer the target
architecture:

- GROQ or GraphQL as the Assets query language.
- One external or long-lived index object per resource/query.
- A deployment index fetched by the Worker from SaaS/CDN infrastructure.
- Lazy production reads of source Markdown bodies.
- A separate index bucket, asset origin, invalidation service, or
garbage collector.
- A 4 MiB published index limit.
- Rebuilding an old build's database from the current mutable asset set.
- Bundling query infrastructure into projects without a configured
content query.

## Verification baseline

The embedded runtime-database architecture previously passed the checks
below. The OpenAPI-driven query-authoring change additionally passed:

- Query-builder core: 4 files / 19 tests.
- Query-builder React: 1 file / 7 tests.
- SDK Assets source/config: 1 file / 7 tests.
- Protocol Assets OpenAPI: 1 file / 9 tests, including independent
OpenAPI validation.
- Builder Assets OpenAPI description: 1 file / 5 tests.
- Query-builder, React, SDK, protocol, and Builder typechecks.
- Package-boundary validation and formatting of every affected file.

Local verification baseline:

- Content engine: 15 files / 104 tests.
- Asset uploader: 24 files / 175 tests.
- Builder: 204 files / 2,423 tests.
- CLI: 54 files / 866 tests.
- Complete workspace typecheck, formatting, lint, package boundaries,
generated fixtures, and fixture builds.
- The complete migration history applied successfully to a fresh
Supabase-compatible database, including every migration after the
consolidated Assets migration.
- Builder production build and full local E2E suite passed.
- Complete agent evaluations passed: authenticated page (170,859 ms),
design input (207,101 ms), and font assets (67,050 ms).
- Final review additionally verified correct REST edit/view/build
permits, cookie and public-token read paths, immutable snapshot retries,
exact database budgeting, query-free dependency omission, and SSR/SSG
conditional runtime inclusion.

Final staging verification passed against the embedded runtime-database
architecture: MCP uploaded and updated five Markdown articles,
discovered dynamic frontmatter fields, previewed a five-item sorted
overview and one full-body slug query, updated both Assets resources,
found zero binding issues, rendered `/blog` and `/blog/dynamic-routes`
at HTTP 200, published build version 266, and verified both published
routes at HTTP 200 with the expected article metadata and Markdown body.
The workflow also found and fixed offset-form asset timestamps failing
canonical datetime validation and made MCP `preview.start({ port: 0 })`
correctly select an available local port.

## Release constraints

- No new SaaS content service, bucket, or environment variable.
- No published-site database dependency.
- No parser/compiler dependency in query-free sites or runtime-only
bundles.
- The validated content database is immutable build output.
- Publishing warnings do not block deployment.
- The selected standard OpenAPI operation and its request JSON Schema
are the only source of query UI semantics.
 2026-07-31 00:17:46 +02:00