mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-11-09 05:26:24 +09:00
main
13
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleg Isonen
|
13eb7527ef |
fix: route deprecated Webhook Forms through Form backend (#6553)
Deprecated Webhook Forms reuse the existing Form execution pipeline without teaching the modern backend about WebhookForm. ## User-facing behavior and how to test it - Existing saved Webhook Forms keep their markup, action settings, and success/error feedback. Submit a published Webhook Form and check both outcomes. - URL and Resource actions preserve configured methods, headers, dependencies, body overrides, and the last submitted value for repeated field names. Test a configured action and inspect the received request. - Default owner notifications use the Email Worker, including submitted files. Submit a Webhook Form without a custom action and inspect the owner email and attachment. - Configured Email actions validate recipients and retain threading metadata. Test valid recipients and a rejected recipient limit. ## Implementation - One deprecated Webhook Form adapter calls existing bot validation, Email preparation, recipient validation, resource loading, and response handling. - Both generated route templates call that adapter. Both paths reuse an internal resource-graph traversal helper; modern submission behavior is unchanged. - Root actions use the protected, uncached request override. Dependencies reuse existing generated Assets and Resource caching behavior. - Keep reachable-resource filtering because Email preparation runs before loading: unrelated Email settings must not break the selected action. - Remove dead `getDefaultActionResource` wiring from generated templates. ## Verification - Focused SDK tests: 116 passed (11 deprecated Webhook Form adapter and 105 modern submission, Email, and handler tests), including unrelated resources, missing dependencies, cycles, Email preflight/threading, and single-attempt Email failures. - Resource generator tests: 26 passed. - Full CLI prebuild suite: 139 passed after tracked fixture regeneration; focused deprecated Webhook Form adapter tests: 9 passed. - Repository fixture pipeline (`pnpm -r fixtures:build`) passed. All tracked generated `getDefaultActionResource` references were removed, and no Builder `N8N_FORM_EMAIL_HOOK` reference remains. - Final MCP-generated local Preview E2E rerun: 3/3 passed. - Focused saved Webhook Form custom-action E2E rerun: 2/2 passed (setup and test), including a single POST through the protected test receiver after Builder reload. - SDK, CLI, and Builder typechecks, focused lint and formatting, and `git diff --check` passed. - Deprecated component picker regression test passed (2/2); verified it fails when WebhookForm deprecation is disabled, then passes with deprecation restored. - Independent Astra review found no blocker, including the 62 generated fixture changes. No production deployment or live secret change was performed. - Required staging CI checks passed on previous head `15635fc` (Main workflow, visual regression, submodules, and lint). An unrelated existing MCP success-redirect E2E assertion failed once in shard 5, passed in a focused local rerun (2/2 including setup), then shard 5 passed on GitHub rerun without code changes. - Terminology pass (`4a69ab573`): renamed the dedicated SDK adapter and migration helper, updated both route templates and generated fixtures, corrected the active identity migration marker comments, and synchronized docs and test names. Focused tests passed: SDK 35, CLI prebuild 12 plus 2 after final title change, MCP 7, migrations 8, React browser 7, CLI docs 9. Seven affected package typechecks, fixture builds, formatting, and `git diff --check` passed. That historical head was superseded; latest-head CI results are recorded below. - ResourceForm identity migration (`29a21edbb`): focused migration (23), DB patch (25), project-build DB/session (93), Builder API/import/auth/patch (60), React SDK (162), runtime/MCP (239), registry (11), CLI evaluation (90), CLI session/sync (55), and protocol (15) tests passed; full CLI prebuild (139) passed. Affected typechecks, lint, formatting, and `git diff --check` passed. Independent Astra review found no blocker. No deployment was performed. - Prepublish Webhook Form warning (`25855e8f00`): audit checks every non-draft published page and links nested/shared instances to the correct page; modern ResourceForm, native forms, draft-only forms, and orphans are excluded. Focused audit tests (13), Builder navigation tests (6), project-build and Builder typechecks, lint, formatting, and `git diff --check` passed. Independent Astra review found no blocker; publishing remains nonblocking. No deployment was performed. - ResourceForm test and CI expectation repair (`e1a527fbcc`): focused React SDK (24), Builder preview/registry (38), project-build pages (1), and CLI fixture API (8) tests passed; Remix and React Router targeted browser suites, affected package typechecks, full formatting (3375 files), lint, and `git diff --check` passed. Astra review found no blocker. Persisted identity and runtime exports were unchanged. - Project bundle fixture repair (`70889ef27d`): eight bundles now carry the current contract version; the two unmarked historical Form instances in `webstudio-features` normalize to WebhookForm. Protocol fixture tests (17), protocol typecheck, CLI fixture build, formatting, lint, and diff check passed. Independent Astra review found no blocker. - Story/MCP E2E CI repair (`483614e6c5`): SDK CLI story tests (7), ResourceForm fragment tests (34), and focused local managed-form E2E (4 including setup) passed. SDK CLI and Builder typechecks, lint, formatting, and diff check passed. Astra review found no blocker. The initial local Preview build used an ignored stale React Router `lib` artifact; rebuilding it restored the passing E2E run. - Latest PR head `483614e6c5`: Main workflow [run 39317777032](https://github.com/webstudio-is/webstudio/actions/runs/39317777032) passed, including all package matrices and six Builder E2E shards. Visual regression [run 39317796922](https://github.com/webstudio-is/webstudio/actions/runs/39317796922), submodules, and PR lint checks passed. The overlapping unapproved visual run was cancelled by the approved rerun. - Shared bot-value generator (`ecc4cad1e9`): SDK helper tests (3), React Router WebhookForm/ResourceForm browser tests (5), SDK/Router/Remix typechecks and builds, SDK browser-boundary check, formatting, and diff check passed. Brave bypasses media queries, working media queries produce a fresh timestamp per submission, and failed queries produce `jsdom`; each managed/legacy submission has one bot field. Independent Astra review found no blocker. Latest-head Main workflow [run 39325677117](https://github.com/webstudio-is/webstudio/actions/runs/39325677117) passed, including all six Builder E2E shards; approved visual regression [run 39325702835](https://github.com/webstudio-is/webstudio/actions/runs/39325702835), submodules, and PR lint passed. The superseded visual run was cancelled by the approved rerun.\n\n- Historical Form clipboard guard (`e864283d1c`): instance/page transfer parsers reject ambiguous legacy `Form` before asset import; direct fragment/page/folder mutations preflight before changes. Canonical ResourceForm, WebhookForm, and NativeForm remain accepted. Project-build tests (89), Builder paste browser tests (69), both typechecks, focused lint/formatting, and diff check passed. Independent Astra review found no blocker. Latest-head Main workflow [run 39329184890](https://github.com/webstudio-is/webstudio/actions/runs/39329184890) passed, including all six Builder E2E shards; approved visual regression [run 39329562692](https://github.com/webstudio-is/webstudio/actions/runs/39329562692), submodules, and PR lint passed. The earlier unapproved visual run reported two expected story differences.\n\n## Compatibility notes - Root HTTP actions reject private and same-origin destinations. Dependencies retain their existing fetch/cache behavior. - Routes retain the 25 MiB request limit. Shared execution applies bot validation, a 10-second Resource timeout, and one retry for eligible HTTP failures; an ambiguous failure can duplicate an HTTP action. Email actions are not retried. - Default email uses the Email Worker's sender, plain text, recipient limits, and attachment limits instead of the legacy n8n HTML/basic-auth path. - Self-hosted sites need normal Email sender configuration. Direct Builder draft Preview handling is outside this published-route change. - Coordinate this CLI release with [SaaS PR #669](https://github.com/webstudio-is/webstudio-saas/pull/669), which removes the n8n helper and `N8N_FORM_EMAIL_HOOK` from new published-site builds. Publish with a CLI artifact containing this adapter. Existing deployed workers retain their bundled code and injected hook until republished, so keep the actual n8n endpoint available while those sites remain active. ## Implementation checklist - [x] Align saved Webhook Form custom-action E2E with the protected receiver and verify one submission. - [x] Regenerate tracked published-site fixtures with the updated CLI and confirm no `getDefaultActionResource` or `N8N_FORM_EMAIL_HOOK` references remain in Builder source/config/fixtures. - [x] Review authoritative `docs/` impact of removing the obsolete Builder hook wiring; related deprecation and integration docs were updated. - [x] Route deprecated Webhook Form submissions through the shared execution pipeline with one adapter. - [x] Preserve configured actions, dependencies, repeated fields, body overrides, and saved feedback. - [x] Prepare and validate Email actions using existing shared logic. - [x] Remove duplicated orchestration and dead generated context wiring. - [x] Verify generated routes and MCP-generated Preview submissions. - [x] Mark the Webhook Form component and its dedicated dependencies visibly deprecated, while leaving shared Form infrastructure active. - [x] Group Webhook Form-only SDK code for straightforward removal while preserving public APIs and generated output. - [x] Use deprecated terminology consistently for Webhook Form-only PR code, and clarify the active identity migration marker. - [x] Confirm required staging CI checks. - [x] Use ResourceForm as the one active modern component identity across Builder, SDK, runtime, templates, generated code, and docs. - [x] Migrate historical Form, NativeForm, ResourceForm, and WebhookForm identities in one idempotent pass using the historical pages marker. - [x] Remove the marker from ordinary project and snapshot APIs while preserving historical input readers. - [x] Persist pages and instances together when a historical marked project is saved, including a pages-only edit. - [x] Verify migration, export/reimport, old CLI and server snapshots, page-only save, generation, and affected typechecks. - [x] Complete an independent architecture review before commit or push. - [x] Warn at prepublish for deprecated Webhook Forms on published pages, with a link to each affected element. - [x] Verify modern ResourceForm is excluded and nested/shared Webhook Forms navigate to the correct page. - [x] Complete independent review, focused tests, and affected typechecks before push. - [x] Remove local `ResourceForm as Form` aliases from React SDK tests and fixtures; use the canonical component name throughout. - [x] Fix stale identity expectations found in CI and verify affected tests/typechecks. - [x] Complete independent review before push. - [x] Refresh tracked project bundle fixtures to the current bundle contract and canonical WebhookForm identity; verify protocol and affected CLI flows. - [x] Align SDK CLI story assertions and MCP E2E fragment fixtures with the persisted ResourceForm component name; rerun affected checks. - [x] Share one browser bot-value generator across ResourceForm submission and deprecated Webhook Form wrappers, preserving field ownership and behavior. - [x] Verify bot timestamps, Brave/media failure handling, repeated submissions, affected typechecks, and independent review before push. - [x] Reject ambiguous historical `Form` clipboard fragments before any insertion or transfer mutation, with clear recopy guidance; preserve canonical ResourceForm action remapping.\n- [x] Verify instance/page/template/folder paste regressions, independent review, and latest-head CI.\n\n - [x] Explain Email attachment size limits in the Email action UI and give a useful error when the aggregate encoded attachments exceed the local guard. - [x] Check Preview and published behavior, boundary tests, and authoritative Form docs; verify the focused change and get independent review. Ref #4093 - [x] Preserve safe email codes/messages through MCP, including local attachment and upload limit errors. - [x] Correct Form documentation for each enforced limit and verify Preview/published/MCP paths. - [x] Verify Builder SDK and MCP focused tests, typechecks, and independent review before push. - Email limits and failure copy (`6d215f3ec3`): the shared SDK checks 32 attachments and 7 MiB aggregate Base64 data before dispatch; the 25 MiB multipart response and Email action guidance use approved copy. Managed Email error codes pass through Form and MCP results with resource IDs, while MCP still omits resource bodies. Form docs distinguish the Worker 7 MiB JSON cap from Cloudflare's 5 MiB complete message and verified-destination exception, and correct the no-retry behavior. Focused SDK tests 89, Builder API tests 44, UI browser tests 42, SDK/Builder/CLI/docs typechecks, lint, formatting, and generated MCP docs check passed. Astra found no blocker. No real email or production deployment occurred. - Latest Email limits head `6d215f3ec3`: Main workflow [run 39393037290](https://github.com/webstudio-is/webstudio/actions/runs/39393037290) passed on rerun, including all six Builder E2E shards; approved visual regression [run 39393456618](https://github.com/webstudio-is/webstudio/actions/runs/39393456618), submodules, and PR lint passed. The initial unapproved visual run showed the same two previously accepted story differences. - [x] Address review: normalize previous-Builder `Form` instance patches before persistence, preserve historical WebhookForm identity, and cover marked/unmarked cases. Documentation impact reviewed: this restores intended save behavior, so no user guide change is needed. - Review finding follow-up (`ce05dd733a`): three Form identity patch regressions passed red/green; full project build-patch tests 28/28, project typecheck, lint, formatting, and diff check passed. Independent Astra review found no blocker. Incoming PATCH `Form` is treated as the immediately previous Builder's modern form alias; already stored unmarked historical `Form` continues to migrate to WebhookForm. The old patch protocol has no client-release identifier, so pre-October-14 live clients adding new ambiguous IDs cannot be identified separately. - Latest review-fix head `ce05dd733a`: Main workflow [run 39397337356](https://github.com/webstudio-is/webstudio/actions/runs/39397337356) passed, including six E2E shards after a targeted shard-4 rerun for an unrelated MDX slash-menu timeout. Approved visual regression [run 39397395206](https://github.com/webstudio-is/webstudio/actions/runs/39397395206), submodules, and PR lint passed. The unapproved synchronize visual run was cancelled by the approved rerun. |
||
|
Oleg Isonen
|
8718feb465 |
test: simplify browser testing infrastructure (#6271)
## Summary Unifies frontend testing around three intentional boundaries: - Vitest Node projects for pure and server tests - Vitest Browser Mode with Playwright/Chromium for DOM and component tests - Playwright Test for Builder end-to-end workflows The Builder no longer uses jsdom. Ordinary E2E workflows use an automatically managed isolated `page` fixture; explicit contexts remain only for simultaneous identities and suite-scoped setup. ## Technical choices - Split Builder tests into one Node project and independent browser projects so each Chromium/Vite process has a bounded workload. - Added shared Browser Mode configuration for the affected SDK React packages. - Preserve the Builder's production top-window bridge ownership; Browser Mode uses explicit test-only installation seams. - Centralized Playwright context options and kept action/navigation timeouts bounded. - Validate Playwright filters before Docker, migrations, browser installation, or Builder builds start. - Preserve the first failed E2E attempt's report, traces, screenshots, and videos when CI retries a shard. - Split static checks, typechecks, CLI tests, core package tests, browser package tests, and Builder projects into parallel CI jobs with five-minute hard ceilings. - Distribute Builder E2E tests across ten isolated shards with two workers; parallel suites create unique per-worker setup data. - Reuse color contrast DOM/canvas samples so the exhaustive theme sweep takes about four seconds instead of timing out at 30 seconds. - The removed cost is jsdom behavior drift, oversized serial check jobs, and unnecessary E2E startup for empty selections. Real browser startup and E2E environment setup remain. ## Todo - [x] Split Builder pure/server tests from browser/component tests. - [x] Migrate Builder DOM tests from jsdom to Vitest Browser Mode. - [x] Migrate affected SDK React DOM tests to the shared Browser Mode setup. - [x] Remove jsdom annotations, dependencies, and compatibility setup. - [x] Replace ordinary manual E2E contexts with the isolated `page` fixture. - [x] Keep explicit contexts only for concurrent identities and `beforeAll` setup. - [x] Preserve setup authentication isolation from workflow pages. - [x] Add bounded Playwright action and navigation timeouts. - [x] Preserve first-attempt E2E failure evidence across CI retry. - [x] Reject empty E2E selections before expensive environment setup. - [x] Parallelize repository checks into bounded CI shards. - [x] Isolate parallel E2E setup data across workers. - [x] Remove the color contrast sweep bottleneck. - [x] Update E2E contributor documentation. - [x] Review product documentation impact; no user-facing product docs are affected. - [x] Run the full repository tests and typechecks. - [x] Run formatting, lint, package-boundary, generated-file, and fixture checks. - [x] Complete final correctness and simplification review. ## Verification - `pnpm --filter @webstudio-is/builder test` - `pnpm -r typecheck` - `pnpm lint` - `pnpm check:package-boundaries` - `pnpm check:generated-api` - `pnpm check:generated-docs` - `pnpm fixtures` - Design-system: 222 tests pass locally. - Builder: all seven Vitest projects pass locally. - The formerly slow E2E shard passes locally in 1m55s. - Full database-backed Builder E2E execution passes in CI. - All CI jobs pass; the slowest completed in 4m58s. - No fixture inputs/outputs or generated files changed. Ref #6270 |
||
|
Oleg Isonen
|
f9681dc30f |
feat: MDX-backed Content Blocks (#6234)
## Summary Supersede #6182 with a clean implementation of MDX-backed Content Blocks on top of the unchanged `main` Content Block editing model. A connected `.mdx` Asset changes where authored children are loaded and saved; it must not change how Content mode inserts, edits, selects, moves, copies, or deletes instances. The previous implementation remains available in `agent/mdx-content-block-6124` and `agent/mdx-content-block-6124-review`. Reuse its independently tested parsers, converters, schemas, Asset repository, template resolution, publication, MCP, and UI work where those pieces fit the corrected architecture. Do not port its per-operation storage routing, projected editing stores, MDX undo/history, or atomic multi-root machinery. Ref #6124 Ref #6182 Closes #6261 ## Locked behavior - `.md` keeps its existing Markdown behavior; `.mdx` enables MDX. - Standard Markdown structures are stored as Markdown. JSX is used only for structures Markdown cannot represent losslessly. - Generic elements use `<ws.element ws:tag="…">`; template instances use `<ws.element ws:name="…">`. - Invalid, unsafe, or unsupported MDX never prevents editing the file or publishing the project. Builder and publication use the same recovery result: recoverable nodes are skipped with source-located warnings, while an unrecoverable document renders no authored file content in either environment. - Connecting a source never serializes existing Content Block children into the file. **Connect** removes ordinary persisted children, preserves Templates, sets `src`, and uses file content. - Disconnecting resets `src`, discards derived instances, leaves the Content Block empty, and never changes the Asset. - While connected, normal Content Block mutations update in-memory instances synchronously. Asset persistence happens afterward and never blocks rendering. - Genuine external revision conflicts use the standard reload-required flow. Rapid local edits are ordered/coalesced and must never conflict with each other. - Repeated scopes use Content Block ID plus Collection item identity. - Multiple Content Blocks may intentionally share one MDX Asset. They share one revisioned document session and edits propagate to every connected occurrence. - Missing templates show warnings/placeholders only in Builder and are skipped on published sites. ## Reuse strategy - [x] Audit all 62 original implementation commits and 5 later review/fix commits - [x] Reuse the shared Content Engine Markdown/MDX AST, parser, serializer, diagnostics, reference discovery, and tests - [x] Reuse SDK source schemas, instance-name utilities, unique template naming, and source-integrity validation - [x] Reuse template resolution/materialization and authored-tree conversion after removing storage-operation coupling - [x] Reuse revisioned Asset content repository, authorization, limits, and browser/server transport - [x] Reuse publication, document-graph, CLI, MCP, source-control UI, frontmatter UI, and E2E work where contracts remain valid - [x] Reuse tests as behavioral specifications; rewrite tests that assert the rejected projection/routing architecture - [x] Do not port per-operation MDX routing, alternate projected mutation stores, semantic reconciliation after each command, MDX Builder history/undo, atomic multi-file rollback, or canvas loading/saving UI ## Old-branch porting map — do not rewrite Before implementing any checklist batch, inspect the named implementation and tests on `agent/mdx-content-block-6124-review`. Move the implementation and its tests first; change only the boundary that conflicts with the rebuilt architecture. Do not create a second implementation under a new name. When a test encodes the rejected projected-store architecture, retain its behavioral assertion and replace only its setup. ### Already ported - `parseMarkdownAst`, `createMdxDocumentFromMarkdown`, `parseMdxDocument`, `serializeMdxDocument`, `preferMarkdownSyntax`, `discoverMdxBodyAssetReferences`, `discoverMdxAssetReferences`, and `previewMarkdownToMdxConversion` - Asset-reference, byte-stream, document-graph, Markdown, and MDX helpers included in commit `76bc388ba9` - Tests: the corresponding `packages/content-engine/src/**/*.test.ts` files already on this branch ### Port with their existing tests | Area | Functions to move or adapt | Tests to move with them | | --- | --- | --- | | Source contract and integrity | `parseContentBlockSourceProp`, `getContentBlockSource`, `isEqualContentBlockSource`, `formatContentBlockSourceIntegrityIssue`, `getContentBlockSourceIntegrityIssues`, `allocateUniqueContentBlockTemplateName` | `packages/sdk/src/content-block.test.ts`, `packages/sdk/src/schema/content-block.test.ts` | | Existing instance/name utilities | `findChildReferenceIndex`, `findParentInstanceReference`, `getInstanceName` | `packages/sdk/src/instances-utils.test.tsx` | | Template naming in mutations | `assignUniqueBlockTemplateNamesMutable`, `findBlockTemplateNameCollision`, `getSourceBackedBlockTemplateContext`, `getBlockTemplateNameConfirmation`, `getBlockTemplateInsertionIndex` | `packages/project-build/src/runtime/block.test.ts` | | Content-mode property permissions | `getContentModePropEligibility`; preserve its distinction between editable, design-only, incompatible, and unknown props | `packages/project-build/src/runtime/content-mode-permissions.test.ts` | | Template resolution | `resolveMdxTemplates` | `packages/project-build/src/runtime/mdx-template-resolution.test.ts` | | Template materialization | `createMdxScopeIdGenerator`, `materializeMdxTemplates` | `packages/project-build/src/runtime/mdx-materialization.test.ts` | | MDX → instances and instances → MDX | `isMaterializedMdxAuthoredContentRoot`, `preserveMdxAuthoredContentIds`, `materializeMdxAuthoredContent`, `reconcileMdxAuthoredContent`, `omitTransientEmptyParagraphs`, `serializeMdxTemplateInsertion` | `packages/project-build/src/runtime/mdx-authored-content.test.ts`; retain the pure serialization cases from `mdx-storage-adapter.test.ts` | | Revisioned Asset I/O | `createAssetContentRevision`, `readAssetContentBytes`, the package-owned `createHttpAssetContentRepository`, the thin Builder `createBuilderHttpAssetContentRepository`, `createAssetContentBridge`, `initAssetContentBridge`, `getAssetContentBridge`; keep one repository implementation rather than copying the old Builder duplicate | `packages/asset-uploader/src/content-repository.test.ts`, `apps/builder/app/builder/shared/assets/mdx-content-repository.test.ts`, `apps/builder/app/shared/asset-content-bridge.client.test.ts` | | Source lifecycle planning | `createContentBlockBodyRemoval`, `prepareContentBlockConnect`, `prepareContentBlockDisconnect`, `prepareContentBlockSwitch`, `ContentBlockSourceRevisionConflictError` | `packages/project-build/src/runtime/content-block-source-lifecycle.test.ts` | | Per-Asset revision/session behavior | `getContentBlockSessionSource`, `isContentBlockSessionSourceCommitted`; reuse the loading, authorization, revision, queue, conflict, retry, cancellation, and source-replacement internals of `createMdxAssetEditingSession`, but key the rebuilt session by Asset rather than Content Block/render root | Port the applicable cases from `packages/project-build/src/runtime/mdx-asset-session.test.ts`; replace root-projection setup with in-memory snapshot setup | | Template migrations | `validateMdxTemplateMigrationConfirmation`, `planMdxTemplateMigration` | `packages/project-build/src/runtime/mdx-template-migration.test.ts` | | Publication | `createPublishedMdxMaterializationCache`, `materializePublishedMdx`, `getPublishedMdxContentDatabaseMaxBytes`, `getDynamicPublishedMdxSourceBlockIds`, `hasDynamicPublishedMdxSources`, `createPublishedBuildContentCompilationPlan`, `resolvePublishedMdxDependencyClosure`, `resolvePublishedMdxAssetCandidates` | `packages/project-build/src/runtime/published-mdx.test.ts`, `packages/project-build/src/content-database.test.ts` | | Build/import integrity | `assertContentBlockSourceIntegrity` and the Content Block cases in `getBuildIntegrityIssues`/`formatBuildIntegrityIssue` | `packages/project-build/src/build-integrity.test.ts` | | MDX paste/MCP insertion | `insertMdxText` and its input/result schemas; retain parsing, confirmation, template-resolution, and insertion-index logic, then invoke the normal mutation path | `packages/project-build/src/runtime/mdx-paste.test.ts` | | Builder source control | `createContentBlockSourceController`, `createBuilderContentBlockSourceController` and the existing source-control components | `content-block-source-controller.test.ts`, `content-block-source.test.tsx` | | Diagnostics | `formatContentBlockDiagnostic`, `deduplicateContentBlockDiagnostics`, `takeNewContentBlockDiagnostics` | Port only their focused cases from `apps/builder/app/shared/content-block-content.test.ts` | ### Explicitly do not port - `projectContentStorageChanges`, `createContentStorageProjection`, `createContentStorageSelectorProjection`, `executeContentStorageMutation`, `executeContentStorageTextReplacement`, `executeContentStoragePropMutation`, and `executeContentStorageStructuralMutation` - `applyMdxContentStorageChanges` and `prepareMdxContentStorageWrites`; replace them with one post-mutation snapshot serializer built from the reusable pure `reconcileMdxAuthoredContent` converter - `createContentBlockApplicationOperations` and the persistence-plan wrappers that route individual editing commands - Builder `$runtime*` projected stores, materialized mutation functions, alternate selector/tree helpers, pending projected-change state, and MDX-specific mutation sequences from `content-block-content.ts` - Canvas loading/saving/open-file presentation state, MDX history/undo, and atomic rollback machinery ## Implementation checklist ### 1. Preserve the `main` editing architecture - [x] Keep normal synchronous insert, text, Enter, slash menu, property, delete, move, wrap, copy/paste, selection, and focus mutations unchanged - [x] Materialize connected content through the same runtime instance interfaces used by ordinary Content Blocks - [x] Keep connected authored children in memory and exclude them from persisted build data - [x] Route persistence only after a synchronous mutation has updated the canvas - [x] Never wait for Asset loading, conversion, or saving before rendering an interaction - [x] Add no MDX-specific implementation to individual editing commands ### 2. Shared Markdown and MDX conversion - [x] Parse Markdown and the Markdown portion of MDX through one shared AST and plugin configuration - [x] Implement one shared MDX-to-instances converter and one shared instances-to-MDX converter - [x] Serialize headings, paragraphs, links, images, lists, tables, task lists, blockquotes, thematic breaks, code, emphasis, strong, strikethrough, and other supported GFM structures as Markdown - [x] Serialize only non-lossless/custom structures as Webstudio JSX - [x] Preserve nested Markdown in MDX elements, frontmatter, comments, tight/loose list semantics, code whitespace, reference links, and Asset references - [x] Omit transient empty draft paragraphs instead of emitting `<ws.element ws:tag="p" />` - [x] Preserve unsupported source text and authored metadata whenever it cannot be converted safely - [x] Add deterministic round-trip and canonical-output coverage for every supported Markdown structure ### 3. Invalid and unsafe MDX - [x] Preserve file source exactly; never silently repair or rewrite invalid source - [x] Ignore imports, exports, expressions, executable JavaScript, invalid properties, and other unsupported nodes when they can be isolated - [x] Continue rendering valid siblings and report every skipped subtree once with filename, line, and column - [x] Use the same recovery pipeline in Builder and publication so both render the same authored result for the current file revision - [x] When structural syntax prevents recovery, render no authored file content in Builder or publication instead of retaining a stale materialization - [x] Skip an unrecoverable Content Block on published sites and continue the build with a warning - [x] Keep invalid and unsafe files editable and reconnectable - [x] Enforce UTF-8, byte-size, depth, node-count, property-count, and generated-output limits with source-located diagnostics where possible ### 4. File formats and Asset creation - [x] Keep `.md` parsing and raw-HTML behavior unchanged - [x] Support creating, opening, editing, deleting, and recreating `.mdx` files through the normal text-file UI - [x] Open `.mdx` in the same editor experience as `.md` - [x] Always create an independent empty MDX Asset even when identical empty files exist - [x] Allow renaming `.md` to `.mdx` without converting or rewriting its contents; after rename, handle invalid contents through the normal MDX diagnostics and recovery pipeline - [x] Update an already-open MDX editor immediately after canvas persistence - [x] Bound read/write buffering and verify the returned Asset identity and revision ### 5. Source contracts and lifecycle - [x] Support direct Asset bindings and dynamic expression bindings - [x] Keep source mutation controls available only in Design mode; show the connected filename and **Open** action read-only in Content mode - [x] Reject duplicate source props, unsupported prop types, missing Assets, non-MDX Assets, and unauthorized Assets - [x] Analyze selected file diagnostics before confirmation without modifying the block or file; MDX syntax diagnostics do not block connection - [x] For a non-empty block, show **Abort** and **Connect** and explain that ordinary children will be replaced - [x] On connect, remove every non-template child, preserve Templates, set `src`, and materialize file content - [x] Replace disconnect handoff with normal `src` reset; discard derived content and leave the block empty - [x] Remove disconnect confirmation and conversion; resetting a repeated binding leaves every occurrence empty - [x] Switching sources validates/loads the target before replacing the usable binding and leaves both files unchanged - [x] Revalidate Asset revision and project version when confirmation is submitted - [x] Make repeated connect/disconnect operations idempotent - [x] Prevent disposed, stale, or out-of-order loads from replacing the newest source - [x] Keep loading, saving, conflict, empty-file, and open-file actions off canvas ### 6. Asynchronous persistence and conflicts - [x] Snapshot the current in-memory authored root after normal mutations - [x] Serialize/coalesce rapid edits per resolved Asset and always prepare against the latest local snapshot - [x] Allow independent Assets to save concurrently without scope leakage - [x] Use one revisioned document session and one ordered save queue per Asset, even when multiple Content Blocks or render scopes reference it - [x] After a shared Asset changes, rematerialize every connected occurrence without persisting their generated instances to project data - [x] Pin the resolved Asset and exact loaded revision for each write - [x] Recheck authorization immediately before storage - [x] Ensure local queued edits never produce stale-revision conflicts with each other - [x] On a genuine external revision conflict, preserve local content, stop writes, and enter the standard reload-required state - [x] Keep failed non-conflicting writes retryable without discarding local content - [x] Finish already accepted persistence writes after a Content Block unmounts or switches source - [x] Cancel obsolete loads, materializations, and UI updates after unmount, reload, or source switch - [x] Dispose project-scoped roots, queues, parse caches, failure state, generations, and session subscriptions when the Asset session is replaced - [x] Invalidate Asset resources and synchronize open file editors after successful writes - [x] Keep multi-root persistence ordered, serial, and non-atomic; report partial completion without rollback - [x] Do not add Asset-history undo/redo; native text-editor undo remains local to the open editor ### 7. Content mode parity - [x] Empty connected blocks remain selectable and expose the normal Content-mode `+` affordance - [x] First insertion renders immediately - [x] Enter creates and focuses the next paragraph immediately - [x] Slash commands work in newly created empty paragraphs - [x] Deleting one selected instance deletes exactly that instance - [x] Mouse and keyboard moves use the same operation and permissions - [x] Preserve sibling reorder, nested reparenting, and moves between editable Content Blocks where `main` permits them - [x] Preserve content-editable props such as Link `href` - [x] Preserve copy, paste, cut, multi-cut, wrapping, unwrapping, grid insertion, selection, Navigator order, and canvas order - [x] Cover mutation routing centrally, retain the full unconnected Builder suite, and exercise representative connected interactions in real Builder E2E ### 8. Template names and permissions - [x] Resolve only exact user-facing names from the direct flat Templates list - [x] Enforce unique names within each Templates list - [x] Reject duplicate names during rename with inline validation - [x] Allocate unique names during insert, paste, duplicate, move, wrap, and unwrap - [x] Apply valid template properties using the same permissions as Content mode - [x] Preserve and warn about ignored design-only, invalid, or stale properties - [x] Preserve template shell provenance without serializing expanded template internals - [x] Generate deterministic materialized IDs from source identity, render scope, and authored usage path so focus/selection survives saves - [x] Warn before renaming or deleting referenced templates; files remain unchanged - [x] Provide shared AST-based rename/remove migration operations for MCP-assisted multi-file cleanup ### 9. Scope, frontmatter, and diagnostics - [x] Scope materialized content by Content Block ID plus exact Collection-item/render context - [x] Prevent repeated instances and dynamic candidates from leaking generated IDs, template resolution, or diagnostics across scopes while sharing the underlying Asset source and revision where applicable - [x] Remove frontmatter editing from Settings; edit the exact YAML through the MDX Asset file editor - [x] Preserve the exact frontmatter source byte-for-byte during canvas body saves - [x] Route canvas body changes and complete-file editor saves through one revisioned per-Asset coordinator - [x] Show a source warning indicator in Settings and precise source-located diagnostics in the Asset editor - [x] Deduplicate repeated diagnostics without losing distinct locations - [x] Keep unresolved-template placeholders selectable and accessible only in Builder ### 10. Asset references, transfer, and integrity - [x] Discover and rewrite Asset references in frontmatter, Markdown links/images, and MDX properties - [x] Preserve Asset references while editing unrelated content - [x] Include referenced Asset and template revisions in dependency/cache identity - [x] Route referenced Assets through published URL rewriting - [x] Validate Content Block source integrity during project import, fragment/page transfer, copy/paste, build integrity, and pre-publish audit - [x] Allow multiple Content Blocks and dynamic render scopes to reference the same MDX Asset without treating the connection as an integrity error - [x] Preserve Templates when copying a connected Content Block - [x] Preserve direct and dynamic Content Block source bindings when copying or pasting within the same project - [x] When copying only one rendered Collection item, resolve that item’s current MDX Asset and replace the copied Content Block’s dynamic source with a direct binding to that same Asset - [x] When copying an entire Collection, preserve the Collection, its instances, and its dynamic source bindings - [x] Preserve the complete MDX source and frontmatter by reference; copying a connected block must not rewrite the shared file - [x] During cross-project transfer, copy every discoverable distinct bound MDX Asset and rewrite transferred bindings to the destination Assets - [x] If some cross-project dynamic candidates cannot be discovered or copied, continue with successful copies, preserve what can be represented safely, and warn about skipped candidates - [x] Revalidate source revisions and the destination project before applying a cross-project transfer - [x] If cross-project Asset creation or project insertion fails, leave the source project unchanged and clean up any newly reserved destination Asset - [x] Use the same transfer operation through Builder and MCP ### 11. Dynamic sources and publication - [x] Resolve dynamic bindings once per render/save scope - [x] Require finite, discoverable candidate sets for publication and load no unrelated MDX - [x] Warn and skip unbounded, missing, unreadable, malformed, or unsafe sources without failing the complete build - [x] Materialize only rendered Content Blocks; ignore Templates definitions and `src` on non-Content-Block instances - [x] Skip unresolved template subtrees while keeping valid siblings - [x] Include selected template styles, fonts, resources, unresolved names, and referenced Asset revisions in generated dependency output - [x] Keep draft-only MDX Assets out of publication retention ### 12. Builder, MCP, CLI, and shared operations - [x] Use the same converters, source lifecycle, revision checks, persistence service, diagnostics, and template migrations in Builder and MCP - [x] Keep frontmatter editing file-based; retain whole-frontmatter runtime operations only where compatibility requires them - [x] Avoid Builder-only or MCP-only mutation/conversion implementations - [x] Regenerate runtime/API contracts from their source definitions - [x] Update CLI Content Engine reference and generated MCP documentation ### 13. Verification and delivery - [x] Add regression tests that reproduce failures from the previous implementation, including double deletion and blocked rendering - [x] Cover two Content Blocks sharing one Asset: edits from either block serialize once, update both canvases, preserve scope-specific Templates, and cannot overwrite a newer local snapshot - [x] Verify new tests fail against the broken implementation and pass with the corrected behavior - [x] Run SDK, Content Engine, Asset Uploader, Project Build, Builder, React SDK, CLI, Protocol, and HTTP Client affected suites and typechecks - [x] Run real Builder E2E for source lifecycle, live file refresh, empty insertion, Enter/slash focus, Link props, delete, keyboard reorder, switching, disconnect, conflicts, unresolved templates, dynamic sources, and repeated scopes - [x] Regenerate shared project fixture bundles and verify a second generation leaves the tree clean - [x] Run lint, package boundaries, `pnpm oxfmt --check`, and `git diff --check` - [x] Review and update authoritative `docs/` content and navigation - [ ] Run the complete agent evaluation suite — not run because it requires separate explicit approval ### 14. Content-mode parity and disconnect handoff - [x] Allow keyboard reordering in Content mode through the normal move command while keeping the editable Content Block boundary - [x] Let already accepted Asset writes drain independently before resetting the source; disconnect does not load, flush, or convert the file - [x] Remove MDX-to-instance copying from disconnect - [x] Reset `src`, remove the derived tree, and leave the ordinary Content Block empty - [x] Preserve automatic text-editor focus after inserting editable content from the template menu - [x] Remove Builder frontmatter-save UI and report complete-file save failures through the Asset editor - [x] Add regression coverage for keyboard movement, disconnect teardown, source switching, insertion focus, and paragraph deletion focus restoration - [x] Run the complete six-scenario MDX Content Block Builder E2E suite ### 15. Final architecture review fixes - [x] Keep one canonical ordered pending document per Asset so shared Content Block occurrences cannot conflict with one another - [x] Merge ordered local edits from different occurrences while preserving text changes, insertions, deletions, and sibling reordering; keep external revisions on the strict reload-required path - [x] Centralize typed render-scope parsing, repeated-occurrence detection, scoped variable lookup, and dynamic Asset resolution across Canvas, Settings, source lifecycle, and copy/paste - [x] Capture the owning Asset session when a save is accepted so normal unmount/source-switch teardown cannot redirect or lose it - [x] Dispose project-owned roots, queues, caches, generations, failure state, and subscriptions when replacing the project Asset session - [x] Add regression coverage for same-Asset edits from two occurrences, same-node last-write ordering, structural reorder, insertion preservation, scoped source resolution, and project cleanup - [x] Re-run affected full unit suites, typechecks, production build, static checks, and all six MDX Content Block Builder E2E scenarios ### 16. Final persistence and transfer hardening - [x] Drain every accepted per-Asset edit before project-session teardown while retaining the owning authorization context - [x] Drain canvas serialization queues and revisioned Asset writes before pre-publish audit and publication - [x] Block publication on a queued serialization failure or Asset revision conflict - [x] Copy actual Asset bytes during cross-project paste, preserve MDX reference names, rewrite direct and Collection-backed Asset IDs, and clean up newly created Assets when insertion fails - [x] Include every resolved dynamic MDX source when copying an entire Collection and warn while preserving successful copies when individual Assets cannot be transferred - [x] Keep MDX editor save failures visible with retry and route conflicts through the standard reload-required flow - [x] Give keyboard nested reparenting the same editable Content Block boundary used by pointer movement - [x] Add regression coverage and rerun the complete Content Engine and Builder unit suites plus type, lint, formatting, package-boundary, and diff checks ## Current rebuild status - The connected Content Block now owns one complete MDX document: frontmatter feeds designed-shell bindings and the explicit Body outlet owns the authored MDX body. - Builder and published output resolve the same frontmatter document graph, including nested document references and Asset references. - Direct frontmatter text/property edits use the existing ordered per-Asset session; referenced-document values remain file-owned. - Repeated Collection occurrences keep isolated runtime shell/Body identities and occurrence-specific document values. - The local blog article layout has been upgraded to the document/Body model and its bindings validate cleanly. - Full affected package suites and typechecks pass; every new regression was demonstrated fail/pass. - `pnpm lint`, package boundaries, `pnpm oxfmt --check`, generated API/docs checks, fixture builds, and `git diff --check` pass. - Full fixture synchronization is blocked by the development API rejecting this local CLI version before fixture generation; existing fixture bundles build successfully. - Complete agent evaluations remain pending explicit approval. ### 17. Treat Content Block source as a property - [x] Declare Content Block `src` in component metadata as a Design-mode initial property labeled **Source** - [x] Add the short property description: “Create an .mdx file in Assets, then select or bind it here.” - [x] Accept only a direct Asset or an expression resolving to an Asset ID; reject URL/string sources, missing Assets, and non-MDX Assets - [x] Extract and reuse the Image-style Asset picker and binding presentation without duplicating source-control UI - [x] Render Source through the normal properties section and remove the separate source-section placement and special `src` filtering - [x] Route Source changes through Content Block connect, switch, and disconnect lifecycle operations instead of generic prop updates - [x] Keep connect confirmation, Open, diagnostics, and persistence state attached to Source; remove frontmatter and disconnect UI - [x] Add regression coverage for direct Assets, dynamic Asset bindings, invalid resolved values, lifecycle operations, and Design/Content mode visibility - [x] Re-review authoritative `docs/` content for the Source-property UI change and update it if required ### 18. Source property UX - [x] Show one full-width bindable Connect .mdx file button while disconnected - [x] Show equal-width filename and Open buttons while connected - [x] Switch files from the filename button through the source lifecycle - [x] Override PropertyLabel reset and Alt-click with the disconnect lifecycle - [x] Preserve confirmation, pending, failure, repeated-scope, and dynamic-source behavior - [x] Add regression coverage for connect, switch, binding, reset, and disconnect - [x] Update Content Block documentation - [x] Run affected tests, typecheck, lint, formatting, package boundaries, and diff checks ### 19. Reduced file-owned frontmatter follow-up This checklist supersedes the earlier structured-frontmatter and disconnect-copy behavior. Frontmatter remains file-owned and is edited only in the MDX Asset editor. #### A. Remove structured frontmatter UI - [x] Remove the JSON frontmatter editor and unused Builder controller/state - [x] Keep source mutation Design-mode-only; show connected Source read-only in Content mode and allow authorized editors to open the complete file - [x] Retain runtime frontmatter APIs only where compatibility requires them #### B. Preserve source and separate validation - [x] Preserve the exact frontmatter envelope during every canvas body save without parsing/stringifying YAML - [x] Preserve comments, ordering, quoting, whitespace, BOM, newline style, delimiters, and invalid but closed YAML - [x] Keep the existing unsupported-node/property MDX recovery; do not add an ad hoc partial parser - [x] Parse body and frontmatter independently so either valid half remains usable - [x] Keep bounded Asset-query frontmatter extraction unchanged - [x] Treat structurally malformed MDX as having no safe body tree #### C. Share Asset state with the raw editor - [x] Route canvas and complete-file changes through one ordered, non-blocking per-Asset coordinator - [x] Update canvas and clean raw editors immediately from shared in-memory source - [x] Reject stale dirty-editor saves, preserve unsaved text, and require reload; never force-overwrite - [x] Drain accepted writes after source replacement/disconnect and report background failures globally - [x] Dispose sessions only after they have no consumers and no queued work #### D. Connect, replace, and disconnect - [x] Load and diagnose before connect; block only unreadable/unauthorized/non-MDX Assets - [x] Show **Abort / Connect** for non-empty blocks, including diagnostics and the recoverable-content warning - [x] Set `src` and delete persisted children through one runtime mutation - [x] Switch to a loaded source without waiting for the previous Asset's network save - [x] Remove disconnect confirmation and conversion - [x] Reset `src` immediately, discard the derived tree, leave the block empty, and preserve the Asset - [x] Resetting a bound source leaves every repeated occurrence empty #### E. Dynamic and shared sources - [x] Keep one source/revision state per Asset and a separate derived instances tree per occurrence - [x] Resolve Templates independently per Content Block and preserve Builder-only missing-template placeholders - [x] Resolve bound `src` from the selected Collection occurrence - [x] Make **Open** use the occurrence's resolved Asset; update the filename and disable Open when unresolved - [x] Add direct/bound tests including Collection item A opening `a.mdx` and item B opening `b.mdx` #### F. Diagnostics, MCP, and publication - [x] Show an accessible warning icon beside Source with the first diagnostic, line/column, and remaining count - [x] Show precise YAML/MDX diagnostics in the Asset editor without repetitive canvas/toast UI - [x] Reuse parser/diagnostic/revision contracts in Builder, build, publication, and MCP - [x] Keep MCP frontmatter editing file-based and return structured diagnostics for complete-source writes - [x] Emit build warnings; published sites skip invalid/unsupported content without diagnostic UI #### G. Verification and documentation - [x] Add fail/pass regressions for source preservation, independent validation, stale raw saves, lifecycle, dynamic Open, shared Assets, and cleanup - [x] Run affected tests, typechecks, lint, formatting, package boundaries, fixtures, and diff checks - [x] Open Markdown and MDX editors at twice the default dialog size with the right-side preview visible initially - [x] Document canvas body editing, file-owned frontmatter, Content-mode Asset access, and cloning an MDX Asset for a new post - [x] Perform a final correctness, data-loss, DRY, simplification, accessibility, and performance review - [x] Keep this checklist synchronized with implementation status ### 20. Content-mode component properties - [x] Show a connected Content Block Source read-only in Content mode with an active **Open** action - [x] Keep source switching, binding, reset, and disconnect available only in Design mode - [x] Expose HTML Embed code and preserve Code Text's code-editor metadata on authored Text Content - [x] Audit every registered component and expose authored payload while keeping design, structure, and behavior controls in Design mode - [x] Keep media dimensions and Code Text theme Design-only while preserving adapter-owned intrinsic Image metadata during MDX round trips - [x] Add exact component-permission and regression coverage - [x] Update authoritative component documentation - [x] Run affected full suites, typechecks, formatting, lint, package boundaries, and generated contract/documentation checks ### 21. Designed document shell and Body outlet - [x] Add one structural **Content Block Body** outlet and keep legacy Content Blocks without it compatible - [x] Materialize and persist only `document.body` children through the Body outlet while preserving the surrounding designed shell and Templates - [x] Expose the connected document frontmatter through a Content Block `document` parameter for shell text and property bindings - [x] Write direct frontmatter-bound text and properties back through the same ordered per-Asset session without blocking canvas mutations - [x] Resolve document `$ref` dependencies and Asset references consistently in Builder and published output - [x] Keep referenced-document values read-only on canvas and direct editors to the referenced file - [x] Isolate repeated Collection occurrences while retaining occurrence-specific source, frontmatter, shell, and Body content - [x] Render the same designed shell, frontmatter values, and Body substitution in generated/published output - [x] Upgrade the local blog article layout to bind title, excerpt, dates, author data, and images from frontmatter and render MDX through Body - [x] Add fail/pass regressions for nested Body discovery, lifecycle, permissions, repeated values, document references, and generated shell rendering - [x] Update authoritative Content Block documentation - [x] Run affected tests and typechecks, formatting, lint, package boundaries, generated API/docs checks, and fixture builds ### 22. Final nested references and frontmatter editing - [x] Keep direct frontmatter-bound text and property controls editable in Content mode without exposing or replacing their bindings - [x] Keep referenced-document values read-only and preserve legacy instance-property name normalization - [x] Render direct frontmatter text bindings through the plain-text editing path - [x] Resolve nested legacy and structured Asset references from referenced Markdown/MDX frontmatter - [x] Preserve missing Asset references, emit structured `ASSET_NOT_FOUND` diagnostics, and keep them out of the document graph - [x] Add fail/pass regressions for the real Settings control and root/nested missing Asset-reference cases - [x] Fix exact Content Block selection after the Body node appears in Navigator - [x] Run full affected suites/typechecks, all six MDX Content Block E2E scenarios, deterministic fixture builds, generated checks, lint, formatting, package boundaries, and diff checks ### Before merge - [x] Resolve #6261: Nested Asset references in referenced Markdown frontmatter work consistently in Assets queries, previews, publication, and static export ### 23. Explicit writable expression bindings - [x] Add an explicit `readwrite` mode to expression bindings; existing expressions remain read-only by default - [x] Accept `readwrite` only for direct static member paths and centralize their parsing and validation - [x] Require the expression root to provide a writable data source; initially support the connected Content Block document frontmatter - [x] Update Content Block permissions, canvas text editing, and property controls to require a valid writable binding - [x] Persist direct frontmatter edits through the existing ordered per-Asset session - [x] Create new direct frontmatter bindings with `readwrite` mode and migrate existing compatible bindings - [x] Preserve writable mode through schemas, serialization, loading, copy/paste, transfer, and export - [x] Add fail/pass regressions for valid paths, derived expressions, unsupported sources, permissions, persistence, and compatibility - [x] Review authoritative `docs/` content and update it if the persisted binding contract affects user-facing documentation - [x] Reject unsafe frontmatter paths and reject `readwrite` bindings outside a connected Content Block document - [x] Preserve expression binding mode through the public HTTP client and document the MCP authoring contract - [x] Limit frontmatter value write-back to Content mode so Design mode retains normal binding replacement/reset behavior - [x] Keep bindings inside Templates and across nested Content Block ownership boundaries read-only - [x] Upgrade compatible legacy bindings even when the Content Block is disconnected during migration - [x] Run affected tests, typechecks, formatting, lint, package boundaries, fixture checks, and `git diff --check` Verification: full SDK (567), Project Build (2,277), and Builder (2,695) suites pass, along with affected Expression, Migration, Template, HTTP Client, and MCP suites; typechecks, lint, formatting, package boundaries, and diff checks pass. Generated contracts are refreshed. Fixture synchronization remains blocked before generation because the development API rejects this local CLI version as incompatible. The checked-in bundles pass the current Protocol suite and every fixture build with the current bundle version. |
||
|
Oleg Isonen
|
db2175f333 |
fix: transfer pasted assets across deployments (#6228)
Closes #6227 ## What changed Cross-deployment copy/paste now transfers referenced assets into the destination project before inserting instances, pages, templates, or folders. - Add optional source deployment metadata to the existing v0.1 clipboard formats. - Keep old v0.1 payloads readable and keep new payloads readable by older builders, which ignore the additive field. - Reuse the existing asset upload-by-URL flow for images, fonts, videos, and files. - Preserve source video dimensions through the existing upload route. - Replace source asset IDs and font-family references with destination values before runtime insertion. - Include and remap page social images and marketplace thumbnails. - Reject current clipboard payloads whose page metadata references omitted assets. - Leave the public `pageTransfer.insert` runtime input unchanged. - Abort insertion with a paste error when an asset cannot be imported. No clipboard version increase, custom network fetcher, address blocklist, rollback framework, or global paste wrapper is introduced. If a multi-asset import partially uploads before a later upload fails, the completed uploads may remain unused. ## Checklist - [x] Rebased onto `main` - [x] Preserve the existing v0.1 clipboard format - [x] Transfer and remap instance assets - [x] Transfer and remap page, template, and folder assets - [x] Preserve video metadata - [x] Keep runtime page-transfer API compatible - [x] Add regression coverage - [x] Review documentation impact and update page-copy documentation - [x] Run Builder tests and typecheck - [x] Run project-build tests and typecheck - [x] Run lint, package-boundary, and generated-API checks ## Verification - Builder full suite: 219 files, 2,552 tests passed - Project-build full suite: 78 files, 2,129 tests passed - Final affected suite: 84 tests passed - New regressions were confirmed failing before the fixes and passing afterward - Builder and project-build typechecks passed - `pnpm lint` passed - `pnpm check:package-boundaries` passed - `pnpm check:generated-api` passed - Fixture inputs and generated fixture bundles are unaffected |
||
|
Oleg Isonen
|
ca08919b13 |
fix: Confirm conflicting root styles on page paste (#5971)
## Summary Prevent page, template, folder, and marketplace page imports from silently replacing local declarations on the shared global root. When incoming root styles differ at the same property, breakpoint, and state, Builder now asks whether to preserve the project values or use the incoming values. Cancellation happens before the runtime mutation. ## Technical choices - Compare CSS values by their serialized meaning so equivalent keyword and unparsed values do not create false conflicts. - Plan breakpoint remapping with the same insertion path used by fragment paste, including ID collisions and breakpoint limits. - Keep named-token conflict resolution separate because token conflicts support suffixing and merging, while a shared root-local declaration has only an existing or incoming value. - Default the root-style dialog to **Keep existing** because global root styles affect every page, and show the breakpoint plus current and incoming values for each conflict. - Re-read current project data after the asynchronous token dialog before detecting root conflicts. - Leave no-conflict resolution undefined so the runtime performs a final conflict check instead of treating it as an explicit **Use incoming** choice. - Resolve root conflicts at the page-copy boundary before using the generic fragment/style insertion path. - Reuse one root-conflict UI resolver for clipboard paste and marketplace imports, one page-transfer traversal helper in the UI and runtime, and one presentational shell for token and root-style conflict dialogs. - Require an explicit `rootStyleConflictResolution` for conflicting `pageTransfer.insert` and `pages.copy` runtime operations so non-UI callers cannot overwrite global values accidentally. - Document that `ours` keeps target-project values and `theirs` uses incoming values in both generated runtime contracts. ## Todo - [x] Detect conflicting root-local declarations by property, breakpoint, and state - [x] Add keep-existing, use-incoming, and cancel behavior - [x] Cover page, template, nested-folder, marketplace, and direct runtime transfer paths - [x] Preserve non-conflicting incoming root declarations with either resolution - [x] Avoid false conflicts for equivalent CSS values and colliding breakpoint IDs - [x] Revalidate root styles after asynchronous token resolution - [x] Preserve existing named-token conflict behavior - [x] Remove duplicated transfer traversal, conflict resolution mapping, and dialog layout - [x] Regenerate the runtime operation contract - [x] Verify regression tests fail when their corresponding behavior is disabled - [x] Run targeted tests, typechecks, formatting, and lint ## Verification - `pnpm --filter @webstudio-is/project-build test -- src/runtime/style-copy.test.ts src/runtime/fragment.test.tsx src/runtime/page-copy.test.tsx` — 80 passed - `pnpm --filter @webstudio-is/builder test -- app/shared/copy-paste/plugin-page.test.ts app/shared/root-style-conflict-dialog.test.tsx app/shared/token-conflict-dialog.test.tsx` — 17 passed - Retrospective fail/pass audit covered semantic CSS equality, breakpoint ID collisions, both runtime guards, both resolution directions, folder/template paths, conflict detail rendering, and stale target data after token resolution - `pnpm --filter @webstudio-is/project-build typecheck` — passed - `pnpm --filter @webstudio-is/builder typecheck` — passed - `pnpm lint` — passed - Runtime contract regeneration was repeated and produced identical output - Fixture pipeline not run locally: no fixture inputs or outputs are affected; GitHub fixture checks passed on the previous revision - Agent evaluations not run: repository guidance requires separate explicit approval before running them ## Compatibility and limitations The new runtime inputs are optional for non-conflicting imports. Existing imports continue unchanged; only imports that would overwrite a conflicting shared root-local declaration require an explicit resolution. No migrations or persistence changes are required. Closes #5970 |
||
|
Oleg Isonen
|
d440e02990 |
feat: Add safer MCP authoring and draft-page verification (#5905)
- **Restore points for agent work**: Create named restore points before major edits, review changes with a dry run, and safely revert project content if needed. - **Shared Slot authoring**: Agents can turn existing sections into shared Slots and insert linked copies across pages, including valid nested Slots. - **DTCG and Figma token imports**: Import design tokens and CSS variables through CLI, MCP, or direct paste, with aliases, modes, composite values, and mappings supported. - **Consistent token conflict handling**: Every paste workflow now offers the same preserve, rename, or merge choices instead of silently overwriting existing design tokens. - **Draft-page previews**: Agents can generate and inspect previews of draft pages without publishing them or exposing drafts in production builds. - **Safer MCP mutations**: Destructive operations require explicit confirmation, invalid raw patches and invented record IDs are rejected, and restore operations commit atomically. - **More dependable agent contracts**: New tools have complete runtime schemas, clearer validation errors, and consistent behavior across CLI, MCP, Builder, and server APIs. |
||
|
Oleg Isonen
|
dde30ffb54 |
feat: Improve MCP authoring, compact output, and agent onboarding (#5883)
## Summary This improves how agents discover, author, inspect, and verify Webstudio projects while reducing unnecessary MCP context usage. ## What changed - Adds an **Agents** section to Project Settings with ready-to-run setup commands and client configuration for Claude Code, Codex, Cursor, and VS Code. - Makes large list and schema responses compact and paginated by default, with **verbose** available when complete records or diagnostics are needed. - Applies consistent compact output to assets, instances, styles, design tokens, CLI schemas, and project-session metadata. - Returns actionable expression warnings for props, resources, and Collection bindings, including unknown variables and risky render-time mutation resources. - Improves Collection expression guidance so agents can bind array items with the correct scoped variables. - Reports rendered audit outcomes explicitly as complete, partial, confirmation-required, or failed. - Expands runtime output schemas so MCP clients can validate responses reliably. - Strengthens the packaged CLI smoke workflow with a realistic seven-page stdio MCP scenario. ## Architecture - Consolidates `project-build` from 65 file-oriented entrypoints to 11 architecture-owned surfaces. - Separates the core model, contracts, runtime authoring, transfer, state, project sessions, MCP, visual tooling, persistence codecs, and authenticated server services. - Isolates Node-only JSX evaluation from browser-safe transfer utilities and authenticated server modules. - Keeps persistence codecs independent from authenticated database services, preventing unrelated module initialization. - Defines shared instance-path types once and reuses them across runtime modules. - Preserves Immer initialization through explicit package side-effect metadata. - Adds package tests that prevent a return to source-file entrypoints or generic exposure of Node-only surfaces. ## Validation - Full repository checks, tests, typechecks, generated-file checks, and fixtures - All 163 Builder test files and 2,141 tests - All 71 project-build test files and 1,794 tests - Lint and package-boundary validation - Full packed CLI release smoke test, including install, preview, agent setup, seven-page stdio MCP workflow, rendered audit, and structured commands |
||
|
Oleg Isonen
|
6e10ae3809 |
fix: Fix Builder paste transfer data handling and publish sync edge cases (#5861)
- Fix publish sync for empty JSON data variables - Retry transient asset downloads once with shared sync-client backoff - Fix Builder paste handling for Webstudio transfer data - Add transfer/data-envelope terminology and colocated runtime data-format modules - Rename page clipboard runtime operation to page transfer semantics - Report malformed Webstudio/Webflow paste data through paste results instead of inserting it as text - Add browser-level E2E coverage for instance, page, folder, template, generic plugin, malformed data, native input, and content-mode paste flows ## Details - Allows empty JSON data variables to sync/publish correctly. - Adds asset download retry coverage and uses shared backoff logic. - Introduces runtime `data-formats` modules for data envelopes, instance transfer data, and page transfer data. - Removes stale clipboard aliases and `validateTemplateStructure`. - Keeps clipboard as a Builder UI transport concern while runtime/protocol use transfer-data terminology. - Changes paste plugins to return expressive success/error results so callers can show toast errors and stop fallback insertion. - Prevents malformed Webstudio transfer payloads from falling through to text/HTML/markdown insertion. - Handles malformed Webflow-owned paste data without throwing. - Adds shared copy-paste fragment insertion helpers to reduce duplicated breakpoint-warning/fragment-data logic. |
||
|
Oleg Isonen
|
d291e4b671 |
feat: Builder api runtime contracts guardrails for MCP (#5857)
This PR expands Webstudio MCP and CLI into a more complete agent authoring interface. Agents can discover project capabilities, edit real Webstudio content, use data/resources, verify visual output, and report useful issues when workflows fail. ## MCP User Capabilities - Create, update, duplicate, reorder, and delete pages. - Create pages from templates and manage reusable page templates. - Inspect project pages, instances, props, styles, assets, variables, resources, and metadata. - Insert Webstudio components and authored JSX fragments into pages. - Edit text, props, styles, breakpoints, design tokens, and CSS variables. - Update page metadata, SEO fields, redirects, marketplace metadata, and project settings. - Create and use static data variables. - Create and use dynamic resources fetched over HTTP. - Use REST, GraphQL, and built-in system resources in authored pages. - Bind data variables and resources to visible page content. - Create resource-backed pages such as sitemap XML, structured SEO data, and API-driven content sections. - Preview current project state without requiring a publish step. - Capture screenshots for visual verification. - Verify responsive designs across desktop, tablet, and mobile breakpoints. - Use screenshot diff and OCR-style checks to inspect rendered output. - Reset and refresh local MCP project state when needed. - Discover available CLI/MCP commands from generated manuals. - Use clearer examples for resources, data variables, JSX authoring, previewing, and visual checks. - Read marketplace product metadata without requiring a full project snapshot. - Get safer guidance when using low-level patch tools. - Get a ready-to-copy Discord report template when MCP/CLI behavior is confusing or broken. |
||
|
Oleg Isonen
|
1ddf6b02c9 |
feat: API and CLI for everything (#5838)
Closes #5287 |
||
|
Oleg Isonen
|
82c3d83fd1 |
fix: optimize CLI import for large data amount (#5831)
## Summary - Add CLI project import support from synced `.webstudio/data.json` into another project via a destination share link. - Move project bundle import/export contract into `@webstudio-is/protocol`, including bundle versioning, staged upload constants, schema validation, and fixture coverage. - Upload project bundle data through staged TUS upload before calling the import RPC to avoid request body limits. - Upload referenced assets as binary files through the existing asset upload route before importing asset rows. - Add `--skip-assets` and `--ignore-version-check` import options. - Add retry handling for transient missing imported asset files by re-uploading only the reported missing assets. - Share missing-asset import error formatting/parsing through protocol to avoid duplicated string contracts. - Preserve browser asset upload behavior while adding API upload support. - Normalize schema naming across the branch to lower-case names without `Schema` suffix. |
||
|
Oleg Isonen
|
ac29cf7d1d |
feat: Keyboard shorthands for instance positioning and slot bug fixes (#5821)
Closes https://github.com/webstudio-is/webstudio/issues/4586 Closes https://github.com/webstudio-is/webstudio/issues/4619 ## Summary Fixes shared Slot editing so linked Slot occurrences stay consistent across structural edits and labels. ## Changes - Fixed moving a direct Slot child out of a Slot so the child is removed from all linked Slot occurrences. - Fixed unwrapping a direct Slot child so the child is removed from all linked Slot occurrences. - Added linked Slot label sync when renaming from Navigator or Settings panel. - Added/updated tests for canonical Slots, legacy Slots, nested Slot children, keyboard moves, unwrap, reparent, and label sync. - Simplified stale Slot detach logic that was incorrectly cloning only the selected Slot occurrence. ## Testing - `pnpm --filter @webstudio-is/builder test -- app/shared/instance-utils/...` - `pnpm --filter @webstudio-is/builder test -- app/builder/shared/commands.test.ts` - `pnpm --filter @webstudio-is/builder typecheck` - `pnpm lint` - `git diff --check` |
||
|
Oleg Isonen
|
d49582bcf5 |
feat: Copy-paste pages, folders and page templates (#5798)
Closes https://github.com/webstudio-is/webstudio/issues/5797 ## Summary - Add page, page template, and folder copy/paste support, including cross-project paste. - Reuse a shared page item actions menu for page headers and context menus with Copy, Duplicate, Delete, and shortcut hints. - Add Paste to the pages context menu. - Preserve template semantics when copying/pasting templates. - Support folder copy with nested folders/pages. - Add Backspace/Delete page deletion shortcuts and focus destructive dialog actions for Enter confirmation. - Rename `init-copy-paste.ts` to `copy-paste.ts`. ## Testing - `pnpm --filter @webstudio-is/builder typecheck` - `pnpm --filter @webstudio-is/builder test app/shared/copy-paste/copy-paste.test.ts app/shared/copy-paste/plugin-page.test.ts app/shared/page-utils.test.tsx app/builder/features/pages/page-utils.test.ts` - `git diff --check` - E2E copy/delete flows were added in `apps/builder/e2e/tests/pages-actions.e2e.ts` |