12 Commits

Include renames

Author	SHA1	Message	Date
Oleg Isonen	e69a3d2f8e	feat: Create structured MDX entries from collection folders (#6317) ... ## Paragraph editing follow-up — 42a16db02c (September 9) Nested formatted paragraphs now retain their text, links, styling identity, and existing labels during editing and saving. No database migration. - [x] Review all four changed text-editor files. - [x] Preserve text before and after nested links inside emphasis, strong text, and spans. - [x] Preserve existing instance labels on save and keep inherited formatting from overwriting a link's styling identifier. - [x] Demonstrate regression failures without each fix, then pass all 41 focused text-editor and MDX synchronization tests. - [x] Pass Builder typecheck, lint, formatting, and diff checks. - [x] Verify the reported paragraph in Design and Content modes: edits reach MDX and survive reload; original text restored. Verify matching link color and underline inside and outside editing. - [x] Review documentation and fixture impact: this restores existing documented behavior; no fixture inputs, generated output, or documentation changes are needed. - [x] Commit and push the reviewed fixes. - [ ] Confirm CI on this commit before merging. Full E2E and agent evaluations were not rerun for this focused follow-up. Earlier CI and suite results below apply to their recorded commits, not this new head. --- ## Final verification — 50093ed2b7 (September 9) Article edits now retain inline formatting and save newly inserted custom components reliably. Saving a file no longer sends a second project-sync write that could restore an older revision. No database migration. - [x] Fix iframe-blur saves, inherited template defaults, unused resolved assets, and stale template tag metadata. - [x] Remove duplicate server writes for already saved file revisions; share the result with the canvas using the existing external-content sync channel. - [x] Demonstrate new regression tests fail without their fixes and pass with them. Keep E2E file-write, content-order, and reload assertions; remove the obsolete duplicate metadata-write wait. - [x] Pass all 2,955 Builder tests, all 2,409 Project Build tests, and all 8 focused MDX E2E checks. - [x] Pass Builder and Project Build typechecks, lint, formatting, and diff checks. - [x] Run the complete fixture pipeline with no generated changes. The later Builder-only save-acknowledgment changes affect no fixture inputs or outputs. - [x] Verify live article fields, hero replacement/save/reload/restore, Accordion text edits, and YouTube URL edits. Verify UI edits reach MDX and MDX edits persist after reload. Restore original article content. - [x] Review documentation impact: these fixes restore the documented Content Block editing behavior; no further documentation change is needed. - [x] Apply `visual-change-approved` after the user's explicit approval of the 24 reviewed intended visual changes. - [x] Commit and push the reviewed changes; worktree is clean. - [x] Confirm CI and mergeability on this final commit: all 19 Main workflow jobs passed, including all six E2E shards; PR lint and submodule checks passed. Visual approval run 34383497017 passed. Its predecessor reported the 24 intended differences; all 24 current images were byte-identical to the reviewed report, with zero capture errors. GitHub reports no merge conflict. Agent evaluations were explicitly skipped for these final runtime fixes, as requested. Earlier evaluation results below apply to their recorded commits, not this final commit. This summary supersedes historical pending-check notes below. The PR has not been merged. --- ## Final merge check — 9bc31ca679 (September 9) - [x] Merge main without rewriting branch history. Preserve main's image-picker fix and description-list validation. - [x] Reset inherited HTML parent context at template boundaries. All four new regressions fail without the fix and pass with it. - [x] Fix shared navigation style composition and alignment; verify the browser regression fails before and passes after. Review focused screenshots. - [x] Pass 2,408 Project Build tests, 251 design-system tests, and 3 image-picker tests; Builder, Project Build, and design-system typechecks; lint, formatting, and diff checks. - [x] Run the complete fixture pipeline with no generated changes. - [x] Rerun all six agent evaluation scenarios on the merged code with GPT-5.5. All functional checks pass; incompatible/missing token-baseline comparison remains explicitly disabled. - [x] Review documentation impact: this merge preserves documented behavior; the image-editing guidance was already updated in f90e39cfcc. - [x] Commit and push; worktree is clean and GitHub reports no merge conflict. - [x] Confirm Main CI on 9bc31ca679: all 19 jobs passed, including all six E2E shards. - [x] Review final visual artifact 10110845203 from run 34367911826: 24 added/changed stories, zero capture errors. Reviewed collection dialogs and icons, light/dark login gradients, dashboard and project-settings navigation, panel spacing, button variants, inset lists, and resettable labels. Fourteen captures are byte-identical to the previously inspected CI images. Updated navigation and collection-settings captures were inspected; the formerly blank story renders normally. Pixel measurements confirm the columns story retains its intended column positions (second column x=261, third around x=388). - [x] Apply visual approval after renewed explicit user approval. Final-head CI is tracked above. The live hero replacement and article/custom-component checks below are now complete. The PR has not been merged. --- ## Merge-readiness follow-up — September 9 - [x] Remove contradictory image guidance and add an executable writable-source/read-only-alt binding recipe. Regenerate CLI/MCP docs. - [x] Verify the recipe regression fails with the old guidance and passes with the new guidance. All 10 focused documentation tests, 2 content-reference tests, CLI/docs typechecks, lint, formatting, and generated-documentation checks passed. - [x] Rerun all six agent evaluation scenarios with GPT-5.5: all functional checks passed. The incompatible/missing token-baseline gate was explicitly disabled; existing baselines are unchanged. - [x] Commit the documentation follow-up as f90e39cfcc. - [x] Review existing CI screenshots and identify unintended shared-list alignment changes. Implement a local shared-style composition/alignment fix with a regression test demonstrated failing before and passing after; all 251 design-system tests and its typecheck passed. - [x] Visually verify and push the shared list fix in 9bc31ca679. The focused run compared 28 stories; navigation is left aligned, column layouts remain intact, and the previously blank collection-settings story rendered normally. No product fix was inferred from that one blank capture. - [x] Review and approve the intended visual differences. See the current final-head CI status above. - [x] Verify these changes affect no fixture inputs or generated fixture output. No database migration. The previously pending live hero-swap and article/custom-component checks are now complete. See the current verification summary above. --- ## Article editing follow-up — d307c8ba3d (September 9) The target blog's hero source was incorrectly marked read-only. It now uses the existing writable image picker. Reading time and the author URL also use direct writable values. Time properties now have native date/date-time controls. No database migration. - [x] Fix the target's hero source, reading-time structure, and author URL bindings while preserving the current content and link destination. - [x] Verify Content-mode category, title, excerpt, date, reading-time, shared image Description, author name/URL, MDX heading, and list-item edits reach their source files. Check reloads and restore test values. - [x] Add date-only, local timestamp, UTC timestamp, empty-value, and legacy-value regression coverage; demonstrate failure without the date control and passing results with it. - [x] Fix the macOS Bash runner's empty-argument failure. Demonstrate default invocation and argument-forwarding regressions fail without their behavior and pass afterward. - [x] Update authoritative Content Block/Time docs and generated CLI/MCP guidance: inventory every article field, use direct writable bindings and appropriate controls, check saved source and reload, and never infer full editability from one heading. - [x] Pass 291 date/text/Time/design-system tests (one existing skip), 2 runner tests, and 9 documentation tests; relevant typechecks, lint, package boundaries, formatting, and diff checks. - [x] Run the complete fixture pipeline: no generated fixture changes. The later runner/docs-only edits do not affect fixture inputs or outputs. - [x] Run the full Builder E2E suite: 82 passed. This used an explicit reporter; the default invocation correction was then covered by the runner tests. - [x] Run all six agent evaluation scenarios on gpt-5.5 after the final image guidance correction. All functional checks passed. The older-model token-baseline gate was disabled explicitly; accepted baselines were not changed. - [x] Review, commit, and push code and generated documentation. - [x] With explicit approval, replace the live hero image through Content mode, verify the MDX reference and reload, then restore and verify the original image. - [x] Complete the article-field checks listed above. Insert Accordion and YouTube from templates; verify UI edits reach MDX and MDX edits persist after reload. Restore the original article content. - [x] Inspect CI for 7834d90a61: formatting, lint, typechecks, generated docs, package/Builder tests, build checks, and five E2E shards passed. The long MDX lifecycle exhausted its total 120-second budget at its final assertion; the trace showed completed edits and reloads, not a stalled save. - [x] Give only that lifecycle a 180-second total budget; leave individual save waits and assertions unchanged. Both local repeats passed (plus setup). Push d307c8ba3d; this test-timeout-only follow-up affects no fixture inputs or generated output. - [x] Verify Main CI for d307c8ba3d: all 19 jobs passed, including all six E2E shards. Visual approval remains separate. - [x] Finish visual review and approval after correcting navigation alignment and verifying the collection-settings story renders. The user approved the reviewed 24 intended differences. - [x] Recheck target publication preparation: seven documents included, zero omitted, zero missing-template omissions. No deployment. - Deferred: manual Storybook/screenshots; not authorized for this pass. These results supersede the earlier pending image-product decisions and live-check notes. Evaluation status for the latest fixes is recorded above. --- ## Blog authoring fixes — 6359f979a3 (September 9) This review fixes failed UI-to-MDX saves, missing source Undo/Redo, editor recovery and focus, and publication preparation for the migrated blog. No database migration. - [x] Stop treating template-record ordering as a style edit; still reject actual style changes that cannot be saved in MDX. - [x] Share save-preparation errors with property controls across the canvas frame. - [x] Keep malformed MDX as an unsaved draft, with repair and discard paths; focus the source editor when opened from the Navigator. - [x] Enable supported Content-mode Move actions using the existing central permission rules. - [x] Keep source Undo/Redo separate from transient rendering history. Serialize rapid commands, wait for pending canvas edits, reject stale-source conflicts, and retain the last 20 edits per project session. - [x] Fix publication candidate discovery for projected documents and unrelated author references; keep rejecting unsupported references used as the actual MDX source. - [x] Demonstrate regression tests failing without their fixes and passing afterward. - [x] Run the complete Builder (2,938), Content Engine (559), and Project Build (2,395) suites: 5,892 tests passed. - [x] Pass relevant package typechecks, root lint, formatting, package boundaries, and diff checks. - [x] Run the full fixture pipeline; no generated fixture changes. - [x] Review documentation impact and document draft recovery and source Undo/Redo. - [x] Verify live heading edits, author-file edits, malformed-draft recovery, and immediate delete/Undo/Redo; restore test content and confirm persisted source. - [x] Verify target publication preparation: seven documents included, zero omitted, no missing-template omissions. This did not deploy the site. - [x] Review, commit, and push. ### Remaining verification and decisions - [x] Correct the target's read-only Image source binding. The existing Content-mode picker replaces the article reference; shared Description edits use Asset settings. No new image-editing API or data model is needed. See the latest verification limits above. - [x] Repeat the full E2E and agent evaluation suites after these changes: 82 E2E tests and all six evaluation scenarios passed. - [x] Repeat the live Accordion/YouTube insertion and editing flow after the save fixes, including UI-to-MDX saves and MDX edits surviving reload. - Historical note: this round did not run CI or Storybook checks; later results are recorded above. Unknown JSX already has source-range lint diagnostics; it remains an editable missing-template placeholder, not an unconditionally blocked save. The target author's binding was also repaired through the UI so Content mode can edit its referenced author file. Shared resolved URLs remain read-only text; the image picker changes the article reference, and Asset settings edits the shared Description. This section supersedes earlier claims of complete verification for the latest code; historical results below remain records of earlier runs. --- ## Sync and import verification — a2a567551c (September 9) Ordinary `link` → `sync` → `import` now exports the current saved Builder project with its current assets. It no longer mixes an older published build with current files. Import uploads assets before assigning their source folders, then restores those assignments when importing the bundle. - [x] Fix foldered asset imports without a database migration. - [x] Preserve file formats, folder assignments, build metadata, and stable repeat-sync output. - [x] Keep explicit `--buildId` export behavior unchanged; default sync does not require publishing. - [x] Review implementation and demonstrate regression tests failing before the fixes and passing afterward. - [x] Update protocol test servers for the current Builder data endpoint. - [x] Review documentation impact and update CLI documentation and README. - [x] Run all 1,121 CLI tests and 43 HTTP-client tests, both package typechecks, root lint, package boundaries, formatting, and diff checks. - [x] Run the full fixture pipeline and include expected current-build fixture changes. - [x] Verify the real source through the local CLI: repeated sync gives identical output; import into the supplied target succeeds. Blog listing and article rendering were also checked during the import investigation. - [x] Commit and push the fix. - Deferred at user request: agent evaluations for this round. Later CI results are recorded above. Sync still downloads the project and missing asset files and builds the content index locally. This is not a transfer-free operation. Default sync exports editable-build metadata rather than published deployment metadata; use the explicit build option when selecting a particular build. --- ## Final review and verification — ab3a6b7c55 (September 9) Existing entries now use the same field controls as New entry. The author-name permission fix, field-level validation, publishing warnings, and final cache fixes are included. No database migration. ### Completed - [x] Review all 38 local files, including entry autosave, revision conflicts, permissions, shared fields, collection discovery, and publishing diagnostics. - [x] Allow existing-folder setup without rewriting entries or rejecting unconfigured frontmatter. - [x] Add Entry settings with autosave, repairable invalid fields, immutable filename slugs, and preserved MDX body/unknown properties. - [x] Warn before publication when missing or ambiguous custom templates omit content, including discovered dynamic sources. - [x] Allow authorized frontmatter writes to referenced Markdown author files and preserve error messages across the canvas-frame boundary. - [x] Reject impossible integer limits, slugless generated-filename patterns, and slug maximum length zero. Existing raw configurations remain readable for repairs. - [x] Share collection discovery across Assets, file editors, and property controls. Builder first synchronizes project asset metadata into the browser. The browser then groups that metadata once per update and shares configuration/template reads, instead of repeating them for each control. This does not remove initial synchronization or entry-frontmatter reads. Preserve project/access isolation, refresh retries, stale-response protection, and duplicate-filename invalidation. - [x] Demonstrate new regressions fail before fixes and pass afterward. - [x] Correct the folder-drag E2E's obsolete root-folder assumption and verify remembered-folder persistence. - [x] Review and update authoritative Content Engine and Content Block documentation. - [x] Merge current main and regenerate the conflicting CLI documentation from its source. - [x] Fix generated previews with shared development dependencies: invoke React Router binaries through Node, isolate each preview's Vite cache, and forward script options correctly through npm and pnpm. - [x] Fix a false MDX conflict when a new edit arrives immediately after the previous save is acknowledged. Advance the local revision before asynchronous parsing and ignore stale refresh results. - [x] Run the full fixture link/sync/build pipeline. Generated changes contain only the expected React Router scripts and Vite cache location. No database migration. - [x] Review the final diff and scan added lines for credential patterns. ### Verification results - Builder: **2,934 tests passed**. CLI: **1,120 tests passed**. - Content Engine: **559 passed**; Asset Uploader: **291 passed**; Project Build: **2,394 passed**; Design system: **250 passed**. - The clean full workspace run passed all 35 non-CLI packages. Its two CLI failures exposed pnpm argument forwarding; the complete CLI rerun passed after that fix. Earlier load-related test timeouts did not recur in the clean run. - All workspace typechecks passed. Builder and CLI typechecks and repository lint were repeated after their final changes. Package boundaries, generated API checks, and the complete generated-documentation gate passed. Documentation was regenerated from source; the post-commit clean-file check passed. - Full Builder E2E: **82 passed** from a fresh build, including generated previews, collection creation/reload, MDX editing, stale revisions, and permission checks. The isolated disposable test database was cleaned up afterward; development data was untouched. - Full agent evaluations: **all six scenarios passed on gpt-5.5**, including every functional check for Markdown blogs, reference discovery, MDX article editing, authentication, fonts, and design input. The account cannot use the old **gpt-5.4-mini** baseline model. Five baseline comparisons are incompatible and the MDX scenario has no accepted baseline; token-baseline gating was disabled explicitly. Accepted baselines are unchanged. - New preview and MDX regressions were demonstrated failing before their fixes and passing afterward. - Working-tree/diff checks passed. Main was merged without rewriting branch history. CI was not checked, as requested. ### Final delivery checklist - [x] Finish the full Builder E2E rerun. - [x] Finish the full agent evaluation rerun with the authorized accessible model. - [x] Commit, verify the clean generated-documentation gate, and push. GitHub reports the branch mergeable with current main. No known code or functional-test blocker remains from this review. CI was not checked at the user's request. Manual Storybook/screen-reader checks remain deferred; the automated E2E and agent visual checks above passed. The historical review sections below retain their original run results. This section supersedes earlier claims about strict entry validation, entry editing, browsing reads, and evaluation authorization. ## Outcome Content editors can create structured MDX entries from a collection folder without writing frontmatter. Designers configure flat fields, limits, optional slug behavior, and the Markdown template in the Builder. ## High-level goals - [x] Keep folders ordinary in the database and derive collection behavior from a direct **collection.json**. - [x] Add **Use as content collection** to folder creation. - [x] Give designers a visual configurator for the JSON Schema and MDX template. - [x] Show a collection badge and a generic **New entry** action. - [x] Generate independent MDX entries from the configured schema and template. - [x] Keep the current page unchanged after creating an entry; no automatic preview navigation. - [x] Select direct MDX entries with URLPattern filename rules; keep ignored files as ordinary supporting assets. - [x] Keep reserved collection files out of collection query results and generated content indexes. - [x] Preserve broken collections as visible, repairable collection folders. - [x] Restore normal folder behavior when **collection.json** is removed. ## Implementation todos - [x] Add the Content Engine collection contract, default schema/template, slug normalization, validation, serialization, and entry generation. - [x] Discover collections from Assets files without a database folder type or migration. - [x] Create **template.mdx** and **collection.json** when the folder checkbox is selected. - [x] Add basic field controls for text, long text, slug, number, integer, and Boolean values. - [x] Support required fields, string length limits, numeric limits, template defaults, and generated slugs. - [x] Add designer collection settings and template editing. - [x] Remove dynamic preview-page configuration and ignore its legacy metadata. - [x] Add the schema-generated **New entry** form and creation endpoint. - [x] Preserve reserved-file portability in authenticated export/import while excluding it from content query and generated-index results. - [x] Document the feature in the authoritative Content Engine guide. ## Deep review checklist A checked item means the area was inspected and evidence was recorded. It does not mean the behavior passed. Unresolved defects and product decisions remain listed in **Review findings**. ### 1. Product contract - [x] Decide whether the collection checkbox creates defaults immediately or opens the configurator first. - [x] Decide whether existing folders can be converted through the UI. - [x] Define collection activation when **collection.json** exists but is invalid. - [x] Define an entry: every valid direct MDX child or only files created through **New entry**. - [x] Confirm collection membership is direct-child only and nested folders can define their own collections. - [x] Define designer, content editor, viewer, service, CLI/MCP, old-client, and direct-API authority. - [x] Decide whether content mode is simplified UI or a server authorization boundary. - [x] Decide whether slugs are permanently immutable. - [x] Decide whether schema changes may invalidate existing entries or must migrate them. - [x] Decide whether invalid unused collections block unrelated queries or whole-project publication. - [x] Decide what happens to the preserved template after collection removal. - [x] Define the promised JSON Schema dialect and supported subset. - [x] Decide whether temporary invalid collection states are admitted for repair. - [x] Define duplicate-request semantics: no duplicates, exactly one success, and retry behavior. - [x] Decide whether entries require valid frontmatter only or valid complete MDX. ### 2. State machine - [x] Review ordinary folders and root-level **collection.json**. - [x] Review empty-folder, template-only, config-uploading, and config-loading initialization states. - [x] Review ready collections. - [x] Review invalid JSON, UTF-8, size, schema, and duplicate-config states. - [x] Review missing, duplicate, wrong-type, oversized, invalid-frontmatter, and invalid-MDX template states. - [x] Review forbidden direct files, invalid entries, and slug/filename mismatches. - [x] Distinguish transient read/network failures from persistent invalid content. - [x] Review removing, removed, moved, renamed, restored, duplicated, and deleted collection states. - [x] For every state, inspect visibility, allowed actions, server admission, query/publish behavior, recovery, and concurrent-client behavior; record recovery behavior. ### 3. Activation and deactivation paths - [x] Review checkbox creation. - [x] Review create/upload/rename/move/paste/duplicate of **collection.json**. - [x] Review restore, import, project duplication, and sync behavior. - [x] Review delete/rename/move/replace of **collection.json**. - [x] Verify all activation paths produce the same state and validation. - [x] Verify all deactivation paths restore ordinary-folder behavior without accidental data loss. ### 4. Folder membership and nested collections - [x] Verify only direct children are collection entries or reserved files. - [x] Verify a nested folder with its own direct config becomes its own collection. - [x] Verify parent rules do not leak into nested folders. - [x] Define filename and extension case sensitivity. - [x] Define duplicate logical-name behavior over generated storage names. - [x] Define how uploading, deleted, and stale assets affect folder state. - [x] Review moving entries out and between collections with different schemas. - [x] Review ordinary subfolders with arbitrary files inside a collection. - [x] Review template behavior after collection removal. ### 5. Roles and authorization - [x] Review config/template listing and download for every role. - [x] Review config/template content edits for every role. - [x] Review config/template rename, move, restore, and deletion for every role. - [x] Review entry create/edit/rename/move/delete/restore for every role. - [x] Review generic upload and subfolder operations for every role. - [x] Review collection removal and repair for every role. - [x] Verify server enforcement separately from hidden or disabled controls. - [x] Recheck permission changes while operations are in flight. ### 6. Mutation surfaces - [x] Review REST upload-ticket creation and upload completion. - [x] Review asset-content replacement and asset metadata changes. - [x] Review runtime add/update/duplicate operations. - [x] Review sync patches and direct persistence helpers. - [x] Review clipboard, drag/drop, move, rename, and duplicate paths. - [x] Review import/export, restore, and project/folder duplication paths. - [x] Review delete-unused-assets and recursive folder deletion. - [x] Verify invariants at the final persistence boundary, not only in UI preflight. ### 7. Concurrent operations - [x] Review create entry against schema save, template save, raw config edit, and collection removal. - [x] Review edit/delete/restore entry against schema and template changes. - [x] Review two collection-settings editors and raw editor versus configurator. - [x] Review schema writer A interleaved with template writer B. - [x] Review upload/move/rename against collection activation and removal. - [x] Review template/config deletion against creation and publication. - [x] Review folder deletion/move and project switch against all async operations. - [x] Review final validity, conflict detection, cleanup, user outcome, cache invalidation, and publish behavior across the implemented intersections; record conflict and recovery behavior. - [x] Review retry after the server commits but the HTTP response is lost. ### 8. Collection initialization and recovery - [x] Review project switch, permission revocation, dialog close, folder rename/move/delete, and conflicting uploads during initialization. - [x] Verify retry is idempotent and does not duplicate reserved files. - [x] Verify partial setup has a visible finish, retry, or cleanup path. - [x] Verify generic actions cannot corrupt the initialization window. - [x] Verify project and folder identities are captured for the whole operation. - [x] Review asset-count, storage, filename, and folder limits with cleanup. ### 9. JSON Schema conformance - [x] Confirm the exact draft and validate configs against its meta-schema. - [x] Review unknown dialects, boolean schemas, malformed keyword types, and duplicate JSON keys. - [x] Review refs/defs, recursion, composition, conditionals, dependencies, and nullable/multiple types. - [x] Review enum, const, format, pattern, patternProperties, and additionalProperties. - [x] Review min/max properties, array constraints, contains, and uniqueItems. - [x] Review exclusive numeric bounds and all string/number/integer limits. - [x] Differential-test validation against an independent draft 2020-12 implementation. - [x] Verify unsupported keywords are rejected or explicitly documented, never silently ignored. ### 10. Schema resource limits and hostile input - [x] Bound schema bytes, depth, field count, enum size, and reference depth. - [x] Bound validator compilation and validation time. - [x] Review recursive references and excessive composition branches. - [x] Review catastrophic patterns and regex denial of service. - [x] Review huge labels, keys, defaults, and values. - [x] Review prototype-sensitive and structurally awkward field names. - [x] Review memory and CPU amplification across many collections. ### 11. Configurator ownership and round trips - [x] Define owned, preserved, read-only, and rejected schema paths. - [x] Verify parse/save/parse semantic equivalence and repeated-save idempotence. - [x] Verify a UI edit changes only explicitly owned schema paths. - [x] Review property/form order and required-array order. - [x] Review unknown root, property, and Webstudio extension metadata. - [x] Review editable primitive fields that also contain enum, const, format, composition, readOnly, or custom constraints. - [x] Review type changes for incompatible preserved or removed keywords. - [x] Review rename/removal collisions with unsupported properties and requirements. - [x] Surface hidden preserved rules that can cause unexplained validation errors. - [x] Review field ordering/reordering. - [x] Review choosing/changing slug field, slug source, and template filename. - [x] Review turning newly added fields into slug or slug-source fields. ### 12. Required/default/unset semantics - [x] Review absent, empty string, false, zero, null, default, and user-cleared values for every field type. - [x] Define required-property versus non-empty-value behavior. - [x] Verify required Boolean false remains valid and understandable. - [x] Review optional Boolean absence versus explicit false. - [x] Review explicitly clearing template-backed values. - [x] Review JSON Schema default as annotation versus persisted entry value. - [x] Verify precedence among template values, schema defaults, generated slug, and user values. - [x] Review hidden/non-renderable required fields and template requirements. - [x] Verify a collection cannot be ready when every entry form must fail. ### 13. Schema evolution and migration - [x] Review add/remove optional and required fields. - [x] Review field rename and label-only changes. - [x] Review type changes and tightened/relaxed limits. - [x] Review enum option, default, slug field/source, template name, and additionalProperties changes. - [x] Verify compatible save ordering and identify impossible two-write migrations. - [x] Decide whether entries are rewritten and whether migration is atomic. - [x] Review entry changes after validation but before schema commit. - [x] Ensure designers see all incompatible entries or an actionable summary. ### 14. Template, YAML, and MDX - [x] Review absent/empty frontmatter and empty YAML. - [x] Review BOM, CRLF, final-newline, and alternate YAML delimiter behavior. - [x] Review duplicate keys, anchors, aliases, tags, comments, quoting, and multiline strings. - [x] Review Unicode and byte limits. - [x] Verify frontmatter and starter-body preservation. - [x] Review invalid MDX bodies, imports, exports, expressions, and unsafe nodes. - [x] Review missing, renamed, duplicate, wrong-type, and oversized templates. - [x] Verify template changes affect future entries only. - [x] Review reserved template exposure through direct asset URLs and build copying. ### 15. New-entry form and creation - [x] Verify every required field is renderable or supplied safely. - [x] Verify deterministic field order and correct controls/constraints. - [x] Compare client and authoritative server validation. - [x] Prevent submitted values from overwriting protected hidden fields. - [x] Preserve values and focus after failed creation. - [x] Review per-field errors, error summary, and first-error focus. - [x] Prevent duplicate submission and define lost-response recovery. - [x] Verify local sync, reload persistence, and project-switch handling. - [x] Review no/stale preview page, query delay, and draft-filtered entries. - [x] Review large forms and narrow/mobile layouts. ### 16. Entry editing and lifecycle - [x] Review body-only, frontmatter-only, and combined edits. - [x] Review invalid intermediate frontmatter and invalid complete MDX. - [x] Review slug change, filename rename, and reference consequences. - [x] Review delete, restore, move out, move between collections, duplicate, paste, and undo. - [x] Review slug reuse after delete and restore collision. - [x] Review edits concurrent with schema/template changes. - [x] Review repair of legacy invalid entries and blocked intermediate saves. - [x] Decide whether existing entries receive a generated settings form or raw MDX editing. ### 17. Slug and filename identity - [x] Review slug source selection, same slug/source field, manual override, and visible final normalization. - [x] Enforce lowercase-dash identity independently of editable schema metadata. - [x] Review empty normalized slugs and template/schema slug defaults. - [x] Review diacritics, non-Latin scripts, and emoji-only sources. - [x] Use locale-independent case comparison and Unicode normalization. - [x] Review character/byte limits and case-insensitive filesystems. - [x] Review collisions with reserved config/template filenames. - [x] Review concurrent creation, stale reservations, deletion, reuse, restoration, and URL encoding. ### 18. Collection-root restrictions - [x] Review upload, external drop, internal drag, generic text creation, paste, duplicate, move, rename, and extension changes. - [x] Review runtime add, sync patch, import, and restore bypasses. - [x] Review upload tickets completed after activation. - [x] Review moving or renaming **collection.json** through metadata changes. - [x] Review dedicated entry creation, initialization, repair, body editing, and collection removal exceptions. - [x] Verify subfolder creation and arbitrary files inside subfolders remain allowed. ### 19. Storage, database, idempotency, and cleanup - [x] Review object-without-File, File-without-Asset, stale UPLOADING, and soft-deleted states. - [x] Review storage success/database failure and database success/storage failure. - [x] Review postcheck failure, object cleanup, row cleanup, and cleanup failure. - [x] Verify cleanup preserves the original error and cannot delete another request's asset. - [x] Review timeout/lost response after commit and safe retries. - [x] Review reservation lookup/cleanup failure and expiration. - [x] Review failed creation cleanup and distinguish incomplete uploads from existing entries with repairable field errors. Existing field errors do not hide entries or block publication. - [x] Verify orphan reclamation and project-scoped cleanup. ### 20. Query, build, and publication - [x] Verify config/template exclusion only for an identified collection. - [x] Verify ordinary root-level **collection.json** remains ordinary content. - [x] Review template queryability after collection removal. - [x] Validate actual entry frontmatter and decide complete-MDX requirements. - [x] Review invalid content versus transient read failure. - [x] Decide invalid-collection blast radius across queries, field catalog, build, and publish. - [x] Review direct asset IDs, document references, build copying, and public URLs for reserved-file leakage. - [x] Verify preview and publish use the same collection rules. ### 21. Cache and snapshot consistency - [x] Verify config, template, entries, filenames, folder movement, activation, and removal affect source revision. - [x] Verify immutable/revision-pinned reads under concurrent swaps. - [x] Verify config and template come from one logical version. - [x] Review revision consistency between entry reads and indexing. Queries do not require entry-schema validation. - [x] Review rejected-promise caching and retry after repair. - [x] Verify no cross-project cache collision. ### 22. Performance and availability - [x] Review sequential versus bounded parallel reads. - [x] Review request cancellation and repeated preview-query work. - [x] Review rate limits and resource-exhaustion protection. - [x] Review progress/cancellation and actionable limit errors. ### 23. Invalid-state repair - [x] Review repair for every config, template, forbidden-file, entry, and slug mismatch state. - [x] Show exact folder, file, field, and rule. - [x] Distinguish retryable transport errors from content errors. - [x] Provide missing-template creation despite normal root restrictions. - [x] Provide duplicate-file deletion and forbidden-file move recovery. - [x] Preserve raw repair when the configurator cannot parse the schema. - [x] Show collection health after repair. - [x] Prevent custom-template leakage to content editors when config cannot identify it. ### 24. Entry navigation - [x] Remove dynamic preview configuration and automatic navigation after entry creation. - [x] Verify entry creation leaves the selected page unchanged, with or without a slug field. ### 25. Accessibility and responsive behavior - [x] Review native form submission and required semantics beyond an asterisk. - [x] Review linked errors, error summary, live announcements, and busy state. - [x] Review focus after errors, add/remove field, key edits, confirmation transitions, and dialog close. - [x] Review dirty-close warning, explicit cancel/retry, and durable feedback beyond toasts. ### 26. Portability, compatibility, and rollback - [x] Review export/import, folder/project duplication, and restore ordering. - [x] Verify logical filenames, not storage names, define the portable contract. - [x] Review case-insensitive filesystem behavior. - [x] Review rollback after collection files already exist. - [x] Verify removal or rollback leaves content recoverable. - [x] Confirm database/migration assumptions. ### 27. Observability and supportability - [x] Review state-transition and validation diagnostics. - [x] Track cleanup failures and orphaned reservations. - [x] Make publish errors actionable without logging private content. - [x] Ensure client errors do not expose stack traces or sensitive internals. ### 28. Verification strategy - [x] Unit-test schema, slug, merge, serialization, and validators. - [x] Demonstrate each new regression test fails without its fix. - [x] Run affected unit suites, typechecks, lint, docs checks, E2E, fixture-impact review, and final diff review. - [x] Review authoritative documentation impact. ## Persistence and recovery No database migration is needed. Folders remain ordinary database rows; their direct **collection.json** activates collection behavior. This PR adds no table, column, index, RPC, folder type, or backfill. - Setup writes the template first and **collection.json** last. Interrupted setup leaves an ordinary folder and can be retried. - Existing immutable File revisions, conditional Asset updates, and upload reservations provide conflict detection and retry-safe entry creation. - Settings saves validate the intended template and schema together, then write only the files that changed. Existing entry field errors stay repairable and do not block unrelated saves, queries, or publication. New entry creation remains strict. - Invalid or interrupted collections remain visible and repairable. Configuration/template structure errors remain distinct from entry-field errors; entry-field errors do not invalidate the entire collection. - Public/service publication excludes configuration and template files. Authenticated CLI exports retain them for lossless import. ## Review findings The implementation review is complete. Confirmed defects were fixed and the following tradeoffs remain explicit. ### Fixed - Enforced collection structure and permissions across REST, runtime patches, import, query, and publication, while keeping existing entry-field errors non-blocking for queries and publication. - Preserved Undo/Redo, including restoring shared File revisions in cloned projects. - Rejected forged shared-file metadata while preserving metadata edits on legacy files with null uploader provenance. - Added retry and recovery behavior for setup, settings saves, lost responses, and transient reads. - Moved collection setup into one retry-safe server operation and kept the local Builder update as one Undoable transaction. - Centralized collection inspection and limited schemas to the flat fields the configurator can represent. Opening a collection still reads its entries in the background to report field errors. - Guarded stale dialogs and deferred saves against permission changes, protected files, project switches, and replaced revisions. - Kept Asset settings open while choosing nested folders; the existing browser regression fails before the fix and passes afterward. - Preserved required Boolean false values, field-level errors, dirty-close prompts, and inline Markdown editing. - Separated authenticated portable exports from public runtime assets. - Updated the authoritative Content Engine guide. ### Known tradeoffs - A failure after template content saves but before configuration saves can temporarily leave an invalid collection. Retrying the settings save completes it; otherwise the visible files allow repair. The template/config rename endpoint remains retry-safe and rolls back a rename when its config write fails. - Cloned-project Undo can retain an unused prior File revision. Separate read-then-delete cleanup could race with another restore, so this path retains the revision. - Build/asset persistence and import already use multiple calls. This PR validates before writes and provides retry/conflict handling; it does not provide a transaction spanning every resource or a global collection lock. - Opening a collection loads its configuration and template through shared Builder discovery, then checks entry frontmatter in the background for field errors. Unchanged configuration/template content is reused across controls and entry edits. Frontmatter checks still require entry reads; ordinary folder browsing does not discover every collection’s file content. ## Latest unstaged-change review — 2026-09-07 - [x] Review collection settings, validation, autosave/error recovery, entry creation, reserved-file actions, and conversion flows. - [x] Review shared labels, list keyboard handling, dialog focus/Escape, buttons, and panel spacing migrations. - [x] Fix Enter/Space list activation lost when arrow navigation replaced the existing press handler. - [x] Recognize historical generated ASCII/Unicode slug patterns, use the bundled Unicode slug schema, and save a locally resolved $ref without changing existing filenames. - [x] Prevent duplicate slugless entries after a lost response: New entry sends a stable UUID per attempt, reused by the repository's existing filename/content retry checks. No database migration. Callers omitting the optional requestId retain the previous behavior. - [x] Preserve Content Block template validation independently of page HTML placement, without accepting invalid rendered nesting. - [x] Demonstrate the new review regressions fail without their fixes and pass afterward. - [x] Review authoritative documentation and update legacy slug behavior. - [x] Run fixtures and inspect generated changes: none. - Deferred manual verification: Storybook screenshots were not authorized for this pass. - Historical evaluation status: not run in this pass; superseded by the final verification section above. ## Verification Local checks for commit **2b8195692e**: - Builder app: 1,739 tests; Builder core: 1,150 tests. After the final fixes, entry/settings tests and shared asset REST checks also passed, including the added stable-retry test. - Content Engine: 535 tests. - Asset Uploader: 279 tests. - Design system: 249 tests. - Project Build: 2,392 tests. - All five affected package typechecks, changed-file lint/formatting, package boundaries, and Content Engine documentation tests passed. - Complete fixture link/sync/build pipeline passed with no tracked fixture changes. - Staged diff and credential-pattern checks passed. No database schema, migration, or generated database type changes. The lockfile includes the collection validator and filename matcher dependencies. - CI for this commit is pending. Builder E2E, manual screen-reader/mobile checks, Storybook screenshots, and agent evaluations were not run in this pass. ## MDX editing review — commit c3702a7238 - [x] Review all 36 local changed files: MDX parsing/materialization, template enforcement, mutation commits, persistence, Assets refresh, Navigator, source binding, editor diagnostics, dialogs, and query menus. - [x] Validate new custom components at the shared Builder mutation commit before applying tree changes. Cover move, reparent, paste, nested paste, duplicate, and batched operations; preserve unchanged authored content. - [x] Match template names and component structure, including renamed templates, instead of accepting any component type found in Templates. - [x] Remove the registered-component fallback from MDX materialization and completion; report untemplated JSX in the editor. - [x] Preserve unresolved JSX until its placeholder is deleted, then persist its removal without clearing the Assets resource during refresh. - [x] Keep custom designer content outside the MDX body; omit the extra body for fresh unconnected Content Blocks. - [x] Resolve Navigator's MDX-file action from the active content root for dynamic sources. - [x] Fix the first dialog drag jump while honoring subsequent explicit size changes; contain long query menus and prevent editor Backspace from deleting instances. - [x] Demonstrate new review regression failures before the fixes and passing results afterward. - [x] Review and update authoritative Content Block documentation. - [x] Run the complete fixture pipeline; no generated fixture changes. - Historical evaluation status: deferred in this pass; superseded by the final verification section above. - Historical verification limits: no Storybook screenshots, Builder E2E, or manual accessibility checks in this pass. See the final verification section for the current E2E result; manual checks remain deferred. ### Latest verification - Builder affected suites: **624 tests passed**. - Project Build full suite: **2,394 tests passed**. - Design system full suite: **250 tests passed**. - Query Builder React full suite: **14 tests passed**. - Component Registry full suite: **7 tests passed**. - All five affected package typechecks, repository lint, changed-source formatting, staged diff checks, and credential-pattern scan passed. - Fixture link/sync/build passed without generated changes. No database migration. - CI for this commit has not yet been verified. The new mutation-commit check covers Builder operations affecting loaded external content roots. It is not a claim that every remote API mutation is now guarded by this client-side check. Closes #6316 ## Local-change review — 2026-09-08 - [x] Review all 47 changed files and trace collection setup, filename filters, public metadata/upload APIs, sync patches, Content-mode write targets, local folder persistence, and MCP downloads. - [x] Fix ignored MDX files being renamed into entries through the metadata API. Selected filenames require New entry. - [x] Fix supporting-asset moves/renames being blocked by unrelated invalid entries. Configuration/template checks still run. The later entry-repair work also removed entry-field validation as a blocker for unrelated writes and publication. - [x] Validate frontmatter directly with Ajv, resolve the bundled slug $ref without network fetching, preserve the flat schema contract, and avoid retaining old validator instances. - [x] Verify Unicode generation, legacy-pattern upgrades, sibling constraints, field error paths, optional binding paths, referenced-file permissions, and independent queued edits. - [x] Review authoritative documentation, correct contradictory referenced-write guidance, and regenerate CLI/MCP documentation. - [x] Demonstrate new review regression failures with the fixes disabled, then restore and pass them. - [x] Pass Content Engine (553), asset uploader (284), SDK (570), project-build (2,394), MCP command (61), and collection/Content-mode browser tests (279), plus expression tests and documentation checks. - [x] Pass seven affected package typechecks, repository lint, formatting, package boundaries, and fixture link/sync/build. No generated fixture changes or database migration. - Historical evaluation status: deferred in this pass; the user subsequently authorized the full suite. See the final verification section above. - Historical verification limits: no Builder E2E or manual visual/accessibility checks in this pass. See the final verification section for the current E2E result; manual checks remain deferred.	2026-09-09 21:31:04 +02:00
Oleg Isonen	f9681dc30f	feat: MDX-backed Content Blocks (#6234) ... ## Summary Supersede #6182 with a clean implementation of MDX-backed Content Blocks on top of the unchanged `main` Content Block editing model. A connected `.mdx` Asset changes where authored children are loaded and saved; it must not change how Content mode inserts, edits, selects, moves, copies, or deletes instances. The previous implementation remains available in `agent/mdx-content-block-6124` and `agent/mdx-content-block-6124-review`. Reuse its independently tested parsers, converters, schemas, Asset repository, template resolution, publication, MCP, and UI work where those pieces fit the corrected architecture. Do not port its per-operation storage routing, projected editing stores, MDX undo/history, or atomic multi-root machinery. Ref #6124 Ref #6182 Closes #6261 ## Locked behavior - `.md` keeps its existing Markdown behavior; `.mdx` enables MDX. - Standard Markdown structures are stored as Markdown. JSX is used only for structures Markdown cannot represent losslessly. - Generic elements use `<ws.element ws:tag="…">`; template instances use `<ws.element ws:name="…">`. - Invalid, unsafe, or unsupported MDX never prevents editing the file or publishing the project. Builder and publication use the same recovery result: recoverable nodes are skipped with source-located warnings, while an unrecoverable document renders no authored file content in either environment. - Connecting a source never serializes existing Content Block children into the file. **Connect** removes ordinary persisted children, preserves Templates, sets `src`, and uses file content. - Disconnecting resets `src`, discards derived instances, leaves the Content Block empty, and never changes the Asset. - While connected, normal Content Block mutations update in-memory instances synchronously. Asset persistence happens afterward and never blocks rendering. - Genuine external revision conflicts use the standard reload-required flow. Rapid local edits are ordered/coalesced and must never conflict with each other. - Repeated scopes use Content Block ID plus Collection item identity. - Multiple Content Blocks may intentionally share one MDX Asset. They share one revisioned document session and edits propagate to every connected occurrence. - Missing templates show warnings/placeholders only in Builder and are skipped on published sites. ## Reuse strategy - [x] Audit all 62 original implementation commits and 5 later review/fix commits - [x] Reuse the shared Content Engine Markdown/MDX AST, parser, serializer, diagnostics, reference discovery, and tests - [x] Reuse SDK source schemas, instance-name utilities, unique template naming, and source-integrity validation - [x] Reuse template resolution/materialization and authored-tree conversion after removing storage-operation coupling - [x] Reuse revisioned Asset content repository, authorization, limits, and browser/server transport - [x] Reuse publication, document-graph, CLI, MCP, source-control UI, frontmatter UI, and E2E work where contracts remain valid - [x] Reuse tests as behavioral specifications; rewrite tests that assert the rejected projection/routing architecture - [x] Do not port per-operation MDX routing, alternate projected mutation stores, semantic reconciliation after each command, MDX Builder history/undo, atomic multi-file rollback, or canvas loading/saving UI ## Old-branch porting map — do not rewrite Before implementing any checklist batch, inspect the named implementation and tests on `agent/mdx-content-block-6124-review`. Move the implementation and its tests first; change only the boundary that conflicts with the rebuilt architecture. Do not create a second implementation under a new name. When a test encodes the rejected projected-store architecture, retain its behavioral assertion and replace only its setup. ### Already ported - `parseMarkdownAst`, `createMdxDocumentFromMarkdown`, `parseMdxDocument`, `serializeMdxDocument`, `preferMarkdownSyntax`, `discoverMdxBodyAssetReferences`, `discoverMdxAssetReferences`, and `previewMarkdownToMdxConversion` - Asset-reference, byte-stream, document-graph, Markdown, and MDX helpers included in commit `76bc388ba9` - Tests: the corresponding `packages/content-engine/src/**/*.test.ts` files already on this branch ### Port with their existing tests | Area | Functions to move or adapt | Tests to move with them | | --- | --- | --- | | Source contract and integrity | `parseContentBlockSourceProp`, `getContentBlockSource`, `isEqualContentBlockSource`, `formatContentBlockSourceIntegrityIssue`, `getContentBlockSourceIntegrityIssues`, `allocateUniqueContentBlockTemplateName` | `packages/sdk/src/content-block.test.ts`, `packages/sdk/src/schema/content-block.test.ts` | | Existing instance/name utilities | `findChildReferenceIndex`, `findParentInstanceReference`, `getInstanceName` | `packages/sdk/src/instances-utils.test.tsx` | | Template naming in mutations | `assignUniqueBlockTemplateNamesMutable`, `findBlockTemplateNameCollision`, `getSourceBackedBlockTemplateContext`, `getBlockTemplateNameConfirmation`, `getBlockTemplateInsertionIndex` | `packages/project-build/src/runtime/block.test.ts` | | Content-mode property permissions | `getContentModePropEligibility`; preserve its distinction between editable, design-only, incompatible, and unknown props | `packages/project-build/src/runtime/content-mode-permissions.test.ts` | | Template resolution | `resolveMdxTemplates` | `packages/project-build/src/runtime/mdx-template-resolution.test.ts` | | Template materialization | `createMdxScopeIdGenerator`, `materializeMdxTemplates` | `packages/project-build/src/runtime/mdx-materialization.test.ts` | | MDX → instances and instances → MDX | `isMaterializedMdxAuthoredContentRoot`, `preserveMdxAuthoredContentIds`, `materializeMdxAuthoredContent`, `reconcileMdxAuthoredContent`, `omitTransientEmptyParagraphs`, `serializeMdxTemplateInsertion` | `packages/project-build/src/runtime/mdx-authored-content.test.ts`; retain the pure serialization cases from `mdx-storage-adapter.test.ts` | | Revisioned Asset I/O | `createAssetContentRevision`, `readAssetContentBytes`, the package-owned `createHttpAssetContentRepository`, the thin Builder `createBuilderHttpAssetContentRepository`, `createAssetContentBridge`, `initAssetContentBridge`, `getAssetContentBridge`; keep one repository implementation rather than copying the old Builder duplicate | `packages/asset-uploader/src/content-repository.test.ts`, `apps/builder/app/builder/shared/assets/mdx-content-repository.test.ts`, `apps/builder/app/shared/asset-content-bridge.client.test.ts` | | Source lifecycle planning | `createContentBlockBodyRemoval`, `prepareContentBlockConnect`, `prepareContentBlockDisconnect`, `prepareContentBlockSwitch`, `ContentBlockSourceRevisionConflictError` | `packages/project-build/src/runtime/content-block-source-lifecycle.test.ts` | | Per-Asset revision/session behavior | `getContentBlockSessionSource`, `isContentBlockSessionSourceCommitted`; reuse the loading, authorization, revision, queue, conflict, retry, cancellation, and source-replacement internals of `createMdxAssetEditingSession`, but key the rebuilt session by Asset rather than Content Block/render root | Port the applicable cases from `packages/project-build/src/runtime/mdx-asset-session.test.ts`; replace root-projection setup with in-memory snapshot setup | | Template migrations | `validateMdxTemplateMigrationConfirmation`, `planMdxTemplateMigration` | `packages/project-build/src/runtime/mdx-template-migration.test.ts` | | Publication | `createPublishedMdxMaterializationCache`, `materializePublishedMdx`, `getPublishedMdxContentDatabaseMaxBytes`, `getDynamicPublishedMdxSourceBlockIds`, `hasDynamicPublishedMdxSources`, `createPublishedBuildContentCompilationPlan`, `resolvePublishedMdxDependencyClosure`, `resolvePublishedMdxAssetCandidates` | `packages/project-build/src/runtime/published-mdx.test.ts`, `packages/project-build/src/content-database.test.ts` | | Build/import integrity | `assertContentBlockSourceIntegrity` and the Content Block cases in `getBuildIntegrityIssues`/`formatBuildIntegrityIssue` | `packages/project-build/src/build-integrity.test.ts` | | MDX paste/MCP insertion | `insertMdxText` and its input/result schemas; retain parsing, confirmation, template-resolution, and insertion-index logic, then invoke the normal mutation path | `packages/project-build/src/runtime/mdx-paste.test.ts` | | Builder source control | `createContentBlockSourceController`, `createBuilderContentBlockSourceController` and the existing source-control components | `content-block-source-controller.test.ts`, `content-block-source.test.tsx` | | Diagnostics | `formatContentBlockDiagnostic`, `deduplicateContentBlockDiagnostics`, `takeNewContentBlockDiagnostics` | Port only their focused cases from `apps/builder/app/shared/content-block-content.test.ts` | ### Explicitly do not port - `projectContentStorageChanges`, `createContentStorageProjection`, `createContentStorageSelectorProjection`, `executeContentStorageMutation`, `executeContentStorageTextReplacement`, `executeContentStoragePropMutation`, and `executeContentStorageStructuralMutation` - `applyMdxContentStorageChanges` and `prepareMdxContentStorageWrites`; replace them with one post-mutation snapshot serializer built from the reusable pure `reconcileMdxAuthoredContent` converter - `createContentBlockApplicationOperations` and the persistence-plan wrappers that route individual editing commands - Builder `$runtime*` projected stores, materialized mutation functions, alternate selector/tree helpers, pending projected-change state, and MDX-specific mutation sequences from `content-block-content.ts` - Canvas loading/saving/open-file presentation state, MDX history/undo, and atomic rollback machinery ## Implementation checklist ### 1. Preserve the `main` editing architecture - [x] Keep normal synchronous insert, text, Enter, slash menu, property, delete, move, wrap, copy/paste, selection, and focus mutations unchanged - [x] Materialize connected content through the same runtime instance interfaces used by ordinary Content Blocks - [x] Keep connected authored children in memory and exclude them from persisted build data - [x] Route persistence only after a synchronous mutation has updated the canvas - [x] Never wait for Asset loading, conversion, or saving before rendering an interaction - [x] Add no MDX-specific implementation to individual editing commands ### 2. Shared Markdown and MDX conversion - [x] Parse Markdown and the Markdown portion of MDX through one shared AST and plugin configuration - [x] Implement one shared MDX-to-instances converter and one shared instances-to-MDX converter - [x] Serialize headings, paragraphs, links, images, lists, tables, task lists, blockquotes, thematic breaks, code, emphasis, strong, strikethrough, and other supported GFM structures as Markdown - [x] Serialize only non-lossless/custom structures as Webstudio JSX - [x] Preserve nested Markdown in MDX elements, frontmatter, comments, tight/loose list semantics, code whitespace, reference links, and Asset references - [x] Omit transient empty draft paragraphs instead of emitting `<ws.element ws:tag="p" />` - [x] Preserve unsupported source text and authored metadata whenever it cannot be converted safely - [x] Add deterministic round-trip and canonical-output coverage for every supported Markdown structure ### 3. Invalid and unsafe MDX - [x] Preserve file source exactly; never silently repair or rewrite invalid source - [x] Ignore imports, exports, expressions, executable JavaScript, invalid properties, and other unsupported nodes when they can be isolated - [x] Continue rendering valid siblings and report every skipped subtree once with filename, line, and column - [x] Use the same recovery pipeline in Builder and publication so both render the same authored result for the current file revision - [x] When structural syntax prevents recovery, render no authored file content in Builder or publication instead of retaining a stale materialization - [x] Skip an unrecoverable Content Block on published sites and continue the build with a warning - [x] Keep invalid and unsafe files editable and reconnectable - [x] Enforce UTF-8, byte-size, depth, node-count, property-count, and generated-output limits with source-located diagnostics where possible ### 4. File formats and Asset creation - [x] Keep `.md` parsing and raw-HTML behavior unchanged - [x] Support creating, opening, editing, deleting, and recreating `.mdx` files through the normal text-file UI - [x] Open `.mdx` in the same editor experience as `.md` - [x] Always create an independent empty MDX Asset even when identical empty files exist - [x] Allow renaming `.md` to `.mdx` without converting or rewriting its contents; after rename, handle invalid contents through the normal MDX diagnostics and recovery pipeline - [x] Update an already-open MDX editor immediately after canvas persistence - [x] Bound read/write buffering and verify the returned Asset identity and revision ### 5. Source contracts and lifecycle - [x] Support direct Asset bindings and dynamic expression bindings - [x] Keep source mutation controls available only in Design mode; show the connected filename and **Open** action read-only in Content mode - [x] Reject duplicate source props, unsupported prop types, missing Assets, non-MDX Assets, and unauthorized Assets - [x] Analyze selected file diagnostics before confirmation without modifying the block or file; MDX syntax diagnostics do not block connection - [x] For a non-empty block, show **Abort** and **Connect** and explain that ordinary children will be replaced - [x] On connect, remove every non-template child, preserve Templates, set `src`, and materialize file content - [x] Replace disconnect handoff with normal `src` reset; discard derived content and leave the block empty - [x] Remove disconnect confirmation and conversion; resetting a repeated binding leaves every occurrence empty - [x] Switching sources validates/loads the target before replacing the usable binding and leaves both files unchanged - [x] Revalidate Asset revision and project version when confirmation is submitted - [x] Make repeated connect/disconnect operations idempotent - [x] Prevent disposed, stale, or out-of-order loads from replacing the newest source - [x] Keep loading, saving, conflict, empty-file, and open-file actions off canvas ### 6. Asynchronous persistence and conflicts - [x] Snapshot the current in-memory authored root after normal mutations - [x] Serialize/coalesce rapid edits per resolved Asset and always prepare against the latest local snapshot - [x] Allow independent Assets to save concurrently without scope leakage - [x] Use one revisioned document session and one ordered save queue per Asset, even when multiple Content Blocks or render scopes reference it - [x] After a shared Asset changes, rematerialize every connected occurrence without persisting their generated instances to project data - [x] Pin the resolved Asset and exact loaded revision for each write - [x] Recheck authorization immediately before storage - [x] Ensure local queued edits never produce stale-revision conflicts with each other - [x] On a genuine external revision conflict, preserve local content, stop writes, and enter the standard reload-required state - [x] Keep failed non-conflicting writes retryable without discarding local content - [x] Finish already accepted persistence writes after a Content Block unmounts or switches source - [x] Cancel obsolete loads, materializations, and UI updates after unmount, reload, or source switch - [x] Dispose project-scoped roots, queues, parse caches, failure state, generations, and session subscriptions when the Asset session is replaced - [x] Invalidate Asset resources and synchronize open file editors after successful writes - [x] Keep multi-root persistence ordered, serial, and non-atomic; report partial completion without rollback - [x] Do not add Asset-history undo/redo; native text-editor undo remains local to the open editor ### 7. Content mode parity - [x] Empty connected blocks remain selectable and expose the normal Content-mode `+` affordance - [x] First insertion renders immediately - [x] Enter creates and focuses the next paragraph immediately - [x] Slash commands work in newly created empty paragraphs - [x] Deleting one selected instance deletes exactly that instance - [x] Mouse and keyboard moves use the same operation and permissions - [x] Preserve sibling reorder, nested reparenting, and moves between editable Content Blocks where `main` permits them - [x] Preserve content-editable props such as Link `href` - [x] Preserve copy, paste, cut, multi-cut, wrapping, unwrapping, grid insertion, selection, Navigator order, and canvas order - [x] Cover mutation routing centrally, retain the full unconnected Builder suite, and exercise representative connected interactions in real Builder E2E ### 8. Template names and permissions - [x] Resolve only exact user-facing names from the direct flat Templates list - [x] Enforce unique names within each Templates list - [x] Reject duplicate names during rename with inline validation - [x] Allocate unique names during insert, paste, duplicate, move, wrap, and unwrap - [x] Apply valid template properties using the same permissions as Content mode - [x] Preserve and warn about ignored design-only, invalid, or stale properties - [x] Preserve template shell provenance without serializing expanded template internals - [x] Generate deterministic materialized IDs from source identity, render scope, and authored usage path so focus/selection survives saves - [x] Warn before renaming or deleting referenced templates; files remain unchanged - [x] Provide shared AST-based rename/remove migration operations for MCP-assisted multi-file cleanup ### 9. Scope, frontmatter, and diagnostics - [x] Scope materialized content by Content Block ID plus exact Collection-item/render context - [x] Prevent repeated instances and dynamic candidates from leaking generated IDs, template resolution, or diagnostics across scopes while sharing the underlying Asset source and revision where applicable - [x] Remove frontmatter editing from Settings; edit the exact YAML through the MDX Asset file editor - [x] Preserve the exact frontmatter source byte-for-byte during canvas body saves - [x] Route canvas body changes and complete-file editor saves through one revisioned per-Asset coordinator - [x] Show a source warning indicator in Settings and precise source-located diagnostics in the Asset editor - [x] Deduplicate repeated diagnostics without losing distinct locations - [x] Keep unresolved-template placeholders selectable and accessible only in Builder ### 10. Asset references, transfer, and integrity - [x] Discover and rewrite Asset references in frontmatter, Markdown links/images, and MDX properties - [x] Preserve Asset references while editing unrelated content - [x] Include referenced Asset and template revisions in dependency/cache identity - [x] Route referenced Assets through published URL rewriting - [x] Validate Content Block source integrity during project import, fragment/page transfer, copy/paste, build integrity, and pre-publish audit - [x] Allow multiple Content Blocks and dynamic render scopes to reference the same MDX Asset without treating the connection as an integrity error - [x] Preserve Templates when copying a connected Content Block - [x] Preserve direct and dynamic Content Block source bindings when copying or pasting within the same project - [x] When copying only one rendered Collection item, resolve that item’s current MDX Asset and replace the copied Content Block’s dynamic source with a direct binding to that same Asset - [x] When copying an entire Collection, preserve the Collection, its instances, and its dynamic source bindings - [x] Preserve the complete MDX source and frontmatter by reference; copying a connected block must not rewrite the shared file - [x] During cross-project transfer, copy every discoverable distinct bound MDX Asset and rewrite transferred bindings to the destination Assets - [x] If some cross-project dynamic candidates cannot be discovered or copied, continue with successful copies, preserve what can be represented safely, and warn about skipped candidates - [x] Revalidate source revisions and the destination project before applying a cross-project transfer - [x] If cross-project Asset creation or project insertion fails, leave the source project unchanged and clean up any newly reserved destination Asset - [x] Use the same transfer operation through Builder and MCP ### 11. Dynamic sources and publication - [x] Resolve dynamic bindings once per render/save scope - [x] Require finite, discoverable candidate sets for publication and load no unrelated MDX - [x] Warn and skip unbounded, missing, unreadable, malformed, or unsafe sources without failing the complete build - [x] Materialize only rendered Content Blocks; ignore Templates definitions and `src` on non-Content-Block instances - [x] Skip unresolved template subtrees while keeping valid siblings - [x] Include selected template styles, fonts, resources, unresolved names, and referenced Asset revisions in generated dependency output - [x] Keep draft-only MDX Assets out of publication retention ### 12. Builder, MCP, CLI, and shared operations - [x] Use the same converters, source lifecycle, revision checks, persistence service, diagnostics, and template migrations in Builder and MCP - [x] Keep frontmatter editing file-based; retain whole-frontmatter runtime operations only where compatibility requires them - [x] Avoid Builder-only or MCP-only mutation/conversion implementations - [x] Regenerate runtime/API contracts from their source definitions - [x] Update CLI Content Engine reference and generated MCP documentation ### 13. Verification and delivery - [x] Add regression tests that reproduce failures from the previous implementation, including double deletion and blocked rendering - [x] Cover two Content Blocks sharing one Asset: edits from either block serialize once, update both canvases, preserve scope-specific Templates, and cannot overwrite a newer local snapshot - [x] Verify new tests fail against the broken implementation and pass with the corrected behavior - [x] Run SDK, Content Engine, Asset Uploader, Project Build, Builder, React SDK, CLI, Protocol, and HTTP Client affected suites and typechecks - [x] Run real Builder E2E for source lifecycle, live file refresh, empty insertion, Enter/slash focus, Link props, delete, keyboard reorder, switching, disconnect, conflicts, unresolved templates, dynamic sources, and repeated scopes - [x] Regenerate shared project fixture bundles and verify a second generation leaves the tree clean - [x] Run lint, package boundaries, `pnpm oxfmt --check`, and `git diff --check` - [x] Review and update authoritative `docs/` content and navigation - [ ] Run the complete agent evaluation suite — not run because it requires separate explicit approval ### 14. Content-mode parity and disconnect handoff - [x] Allow keyboard reordering in Content mode through the normal move command while keeping the editable Content Block boundary - [x] Let already accepted Asset writes drain independently before resetting the source; disconnect does not load, flush, or convert the file - [x] Remove MDX-to-instance copying from disconnect - [x] Reset `src`, remove the derived tree, and leave the ordinary Content Block empty - [x] Preserve automatic text-editor focus after inserting editable content from the template menu - [x] Remove Builder frontmatter-save UI and report complete-file save failures through the Asset editor - [x] Add regression coverage for keyboard movement, disconnect teardown, source switching, insertion focus, and paragraph deletion focus restoration - [x] Run the complete six-scenario MDX Content Block Builder E2E suite ### 15. Final architecture review fixes - [x] Keep one canonical ordered pending document per Asset so shared Content Block occurrences cannot conflict with one another - [x] Merge ordered local edits from different occurrences while preserving text changes, insertions, deletions, and sibling reordering; keep external revisions on the strict reload-required path - [x] Centralize typed render-scope parsing, repeated-occurrence detection, scoped variable lookup, and dynamic Asset resolution across Canvas, Settings, source lifecycle, and copy/paste - [x] Capture the owning Asset session when a save is accepted so normal unmount/source-switch teardown cannot redirect or lose it - [x] Dispose project-owned roots, queues, caches, generations, failure state, and subscriptions when replacing the project Asset session - [x] Add regression coverage for same-Asset edits from two occurrences, same-node last-write ordering, structural reorder, insertion preservation, scoped source resolution, and project cleanup - [x] Re-run affected full unit suites, typechecks, production build, static checks, and all six MDX Content Block Builder E2E scenarios ### 16. Final persistence and transfer hardening - [x] Drain every accepted per-Asset edit before project-session teardown while retaining the owning authorization context - [x] Drain canvas serialization queues and revisioned Asset writes before pre-publish audit and publication - [x] Block publication on a queued serialization failure or Asset revision conflict - [x] Copy actual Asset bytes during cross-project paste, preserve MDX reference names, rewrite direct and Collection-backed Asset IDs, and clean up newly created Assets when insertion fails - [x] Include every resolved dynamic MDX source when copying an entire Collection and warn while preserving successful copies when individual Assets cannot be transferred - [x] Keep MDX editor save failures visible with retry and route conflicts through the standard reload-required flow - [x] Give keyboard nested reparenting the same editable Content Block boundary used by pointer movement - [x] Add regression coverage and rerun the complete Content Engine and Builder unit suites plus type, lint, formatting, package-boundary, and diff checks ## Current rebuild status - The connected Content Block now owns one complete MDX document: frontmatter feeds designed-shell bindings and the explicit Body outlet owns the authored MDX body. - Builder and published output resolve the same frontmatter document graph, including nested document references and Asset references. - Direct frontmatter text/property edits use the existing ordered per-Asset session; referenced-document values remain file-owned. - Repeated Collection occurrences keep isolated runtime shell/Body identities and occurrence-specific document values. - The local blog article layout has been upgraded to the document/Body model and its bindings validate cleanly. - Full affected package suites and typechecks pass; every new regression was demonstrated fail/pass. - `pnpm lint`, package boundaries, `pnpm oxfmt --check`, generated API/docs checks, fixture builds, and `git diff --check` pass. - Full fixture synchronization is blocked by the development API rejecting this local CLI version before fixture generation; existing fixture bundles build successfully. - Complete agent evaluations remain pending explicit approval. ### 17. Treat Content Block source as a property - [x] Declare Content Block `src` in component metadata as a Design-mode initial property labeled **Source** - [x] Add the short property description: “Create an .mdx file in Assets, then select or bind it here.” - [x] Accept only a direct Asset or an expression resolving to an Asset ID; reject URL/string sources, missing Assets, and non-MDX Assets - [x] Extract and reuse the Image-style Asset picker and binding presentation without duplicating source-control UI - [x] Render Source through the normal properties section and remove the separate source-section placement and special `src` filtering - [x] Route Source changes through Content Block connect, switch, and disconnect lifecycle operations instead of generic prop updates - [x] Keep connect confirmation, Open, diagnostics, and persistence state attached to Source; remove frontmatter and disconnect UI - [x] Add regression coverage for direct Assets, dynamic Asset bindings, invalid resolved values, lifecycle operations, and Design/Content mode visibility - [x] Re-review authoritative `docs/` content for the Source-property UI change and update it if required ### 18. Source property UX - [x] Show one full-width bindable Connect .mdx file button while disconnected - [x] Show equal-width filename and Open buttons while connected - [x] Switch files from the filename button through the source lifecycle - [x] Override PropertyLabel reset and Alt-click with the disconnect lifecycle - [x] Preserve confirmation, pending, failure, repeated-scope, and dynamic-source behavior - [x] Add regression coverage for connect, switch, binding, reset, and disconnect - [x] Update Content Block documentation - [x] Run affected tests, typecheck, lint, formatting, package boundaries, and diff checks ### 19. Reduced file-owned frontmatter follow-up This checklist supersedes the earlier structured-frontmatter and disconnect-copy behavior. Frontmatter remains file-owned and is edited only in the MDX Asset editor. #### A. Remove structured frontmatter UI - [x] Remove the JSON frontmatter editor and unused Builder controller/state - [x] Keep source mutation Design-mode-only; show connected Source read-only in Content mode and allow authorized editors to open the complete file - [x] Retain runtime frontmatter APIs only where compatibility requires them #### B. Preserve source and separate validation - [x] Preserve the exact frontmatter envelope during every canvas body save without parsing/stringifying YAML - [x] Preserve comments, ordering, quoting, whitespace, BOM, newline style, delimiters, and invalid but closed YAML - [x] Keep the existing unsupported-node/property MDX recovery; do not add an ad hoc partial parser - [x] Parse body and frontmatter independently so either valid half remains usable - [x] Keep bounded Asset-query frontmatter extraction unchanged - [x] Treat structurally malformed MDX as having no safe body tree #### C. Share Asset state with the raw editor - [x] Route canvas and complete-file changes through one ordered, non-blocking per-Asset coordinator - [x] Update canvas and clean raw editors immediately from shared in-memory source - [x] Reject stale dirty-editor saves, preserve unsaved text, and require reload; never force-overwrite - [x] Drain accepted writes after source replacement/disconnect and report background failures globally - [x] Dispose sessions only after they have no consumers and no queued work #### D. Connect, replace, and disconnect - [x] Load and diagnose before connect; block only unreadable/unauthorized/non-MDX Assets - [x] Show **Abort / Connect** for non-empty blocks, including diagnostics and the recoverable-content warning - [x] Set `src` and delete persisted children through one runtime mutation - [x] Switch to a loaded source without waiting for the previous Asset's network save - [x] Remove disconnect confirmation and conversion - [x] Reset `src` immediately, discard the derived tree, leave the block empty, and preserve the Asset - [x] Resetting a bound source leaves every repeated occurrence empty #### E. Dynamic and shared sources - [x] Keep one source/revision state per Asset and a separate derived instances tree per occurrence - [x] Resolve Templates independently per Content Block and preserve Builder-only missing-template placeholders - [x] Resolve bound `src` from the selected Collection occurrence - [x] Make **Open** use the occurrence's resolved Asset; update the filename and disable Open when unresolved - [x] Add direct/bound tests including Collection item A opening `a.mdx` and item B opening `b.mdx` #### F. Diagnostics, MCP, and publication - [x] Show an accessible warning icon beside Source with the first diagnostic, line/column, and remaining count - [x] Show precise YAML/MDX diagnostics in the Asset editor without repetitive canvas/toast UI - [x] Reuse parser/diagnostic/revision contracts in Builder, build, publication, and MCP - [x] Keep MCP frontmatter editing file-based and return structured diagnostics for complete-source writes - [x] Emit build warnings; published sites skip invalid/unsupported content without diagnostic UI #### G. Verification and documentation - [x] Add fail/pass regressions for source preservation, independent validation, stale raw saves, lifecycle, dynamic Open, shared Assets, and cleanup - [x] Run affected tests, typechecks, lint, formatting, package boundaries, fixtures, and diff checks - [x] Open Markdown and MDX editors at twice the default dialog size with the right-side preview visible initially - [x] Document canvas body editing, file-owned frontmatter, Content-mode Asset access, and cloning an MDX Asset for a new post - [x] Perform a final correctness, data-loss, DRY, simplification, accessibility, and performance review - [x] Keep this checklist synchronized with implementation status ### 20. Content-mode component properties - [x] Show a connected Content Block Source read-only in Content mode with an active **Open** action - [x] Keep source switching, binding, reset, and disconnect available only in Design mode - [x] Expose HTML Embed code and preserve Code Text's code-editor metadata on authored Text Content - [x] Audit every registered component and expose authored payload while keeping design, structure, and behavior controls in Design mode - [x] Keep media dimensions and Code Text theme Design-only while preserving adapter-owned intrinsic Image metadata during MDX round trips - [x] Add exact component-permission and regression coverage - [x] Update authoritative component documentation - [x] Run affected full suites, typechecks, formatting, lint, package boundaries, and generated contract/documentation checks ### 21. Designed document shell and Body outlet - [x] Add one structural **Content Block Body** outlet and keep legacy Content Blocks without it compatible - [x] Materialize and persist only `document.body` children through the Body outlet while preserving the surrounding designed shell and Templates - [x] Expose the connected document frontmatter through a Content Block `document` parameter for shell text and property bindings - [x] Write direct frontmatter-bound text and properties back through the same ordered per-Asset session without blocking canvas mutations - [x] Resolve document `$ref` dependencies and Asset references consistently in Builder and published output - [x] Keep referenced-document values read-only on canvas and direct editors to the referenced file - [x] Isolate repeated Collection occurrences while retaining occurrence-specific source, frontmatter, shell, and Body content - [x] Render the same designed shell, frontmatter values, and Body substitution in generated/published output - [x] Upgrade the local blog article layout to bind title, excerpt, dates, author data, and images from frontmatter and render MDX through Body - [x] Add fail/pass regressions for nested Body discovery, lifecycle, permissions, repeated values, document references, and generated shell rendering - [x] Update authoritative Content Block documentation - [x] Run affected tests and typechecks, formatting, lint, package boundaries, generated API/docs checks, and fixture builds ### 22. Final nested references and frontmatter editing - [x] Keep direct frontmatter-bound text and property controls editable in Content mode without exposing or replacing their bindings - [x] Keep referenced-document values read-only and preserve legacy instance-property name normalization - [x] Render direct frontmatter text bindings through the plain-text editing path - [x] Resolve nested legacy and structured Asset references from referenced Markdown/MDX frontmatter - [x] Preserve missing Asset references, emit structured `ASSET_NOT_FOUND` diagnostics, and keep them out of the document graph - [x] Add fail/pass regressions for the real Settings control and root/nested missing Asset-reference cases - [x] Fix exact Content Block selection after the Body node appears in Navigator - [x] Run full affected suites/typechecks, all six MDX Content Block E2E scenarios, deterministic fixture builds, generated checks, lint, formatting, package boundaries, and diff checks ### Before merge - [x] Resolve #6261: Nested Asset references in referenced Markdown frontmatter work consistently in Assets queries, previews, publication, and static export ### 23. Explicit writable expression bindings - [x] Add an explicit `readwrite` mode to expression bindings; existing expressions remain read-only by default - [x] Accept `readwrite` only for direct static member paths and centralize their parsing and validation - [x] Require the expression root to provide a writable data source; initially support the connected Content Block document frontmatter - [x] Update Content Block permissions, canvas text editing, and property controls to require a valid writable binding - [x] Persist direct frontmatter edits through the existing ordered per-Asset session - [x] Create new direct frontmatter bindings with `readwrite` mode and migrate existing compatible bindings - [x] Preserve writable mode through schemas, serialization, loading, copy/paste, transfer, and export - [x] Add fail/pass regressions for valid paths, derived expressions, unsupported sources, permissions, persistence, and compatibility - [x] Review authoritative `docs/` content and update it if the persisted binding contract affects user-facing documentation - [x] Reject unsafe frontmatter paths and reject `readwrite` bindings outside a connected Content Block document - [x] Preserve expression binding mode through the public HTTP client and document the MCP authoring contract - [x] Limit frontmatter value write-back to Content mode so Design mode retains normal binding replacement/reset behavior - [x] Keep bindings inside Templates and across nested Content Block ownership boundaries read-only - [x] Upgrade compatible legacy bindings even when the Content Block is disconnected during migration - [x] Run affected tests, typechecks, formatting, lint, package boundaries, fixture checks, and `git diff --check` Verification: full SDK (567), Project Build (2,277), and Builder (2,695) suites pass, along with affected Expression, Migration, Template, HTTP Client, and MCP suites; typechecks, lint, formatting, package boundaries, and diff checks pass. Generated contracts are refreshed. Fixture synchronization remains blocked before generation because the development API rejects this local CLI version as incompatible. The checked-in bundles pass the current Protocol suite and every fixture build with the current bundle version.	2026-08-29 20:15:20 +02:00
Oleg Isonen	5a57234c75	revert: Store Content Block content in MDX assets ... Revert the accidental merge of #6125 so the implementation can continue in draft PR #6181.	2026-08-18 13:23:32 +01:00
Oleg Isonen	86b6310fff	feat: Store Content Block content in MDX assets	2026-08-17 15:22:20 +01:00
Oleg Isonen	9fe63c7501	fix: Submit webhook forms only once (#5979) ... ## Summary Webhook Form action resources were being persisted both as prop-bound actions and render-time data sources. Generated sites could therefore execute the POST while rendering and again when the visitor submitted the form. This change keeps prop-bound resources action-only and hardens generation for existing builds that still contain a stale data-source binding. Ref #5806 ## Implementation - Keep `resources.upsertProp` resources out of render-time data and remove stale exposure when an action is edited. - Make the mutation's `dataSourceId` result optional and regenerate the runtime contract. - Exclude action-bound resources from the generated data-resource map, preserving them only in the action map. - Extend the Webhook Form E2E with a local request-counting endpoint. ## Todo - [x] Prevent new Form actions from creating render-time data sources. - [x] Protect existing persisted builds from loading action resources during rendering. - [x] Regenerate the runtime operation contract. - [x] Add mutation, generation, Builder-store, and generated-site regression coverage. - [x] Verify zero calls on page load and one populated POST on submit. ## Verification - [x] Focused Builder E2E: `Webhook Form action submits once and persists after reload` - [x] Project Build test suite: 2,062 tests - [x] SDK test suite: 505 tests - [x] Builder resource mutation test file: 25 tests - [x] SDK, Project Build, and Builder typechecks - [x] Changed-file lint, formatting, and diff checks - [x] Generated API cleanliness check - [ ] Agent evaluation suite (not run; requires explicit approval) ## Notes No fixture inputs or generated fixture bundles are affected.	2026-08-02 22:56:08 +02:00
Oleg Isonen	12ad74584a	feat: Improve MCP authoring, auditing, and verification (#5886) ... ## Summary This improves how coding agents discover, author, verify, and audit Webstudio projects through the CLI and MCP. The work focuses on making agents more reliable with less context, while keeping edits semantic and Builder-editable. ## What changes for users - Agents can verify dynamic bindings after authoring and receive actionable diagnostics when expressions, resources, scopes, or component props do not line up. - Agents can create complete resource-backed runtime UI through a semantic operation instead of coordinating fragile low-level edits. - MCP discovery and command output are compact by default, with pagination and focused verbose detail when an agent needs more context. - Asset, style, token, font, page, and component inspection provides stronger usage context so agents can preserve the existing design system. - Rendered audits provide better evidence for responsive behavior, geometry, contrast, generated-build size, and large-project runs. - Cross-project MCP workflows can run from a manifest with explicit mutation approval, isolated failures, resumable progress, and persistent checkpoints. - Project-specific agent onboarding is clearer and does not add telemetry. - Packaged CLI releases gain stronger smoke verification for install, MCP startup, authoring, preview, screenshots, and audits. ## Reliability and architecture - Runtime operation output schemas are generated from the runtime registry and remain client-compatible. - Project-build exports are consolidated around architectural boundaries instead of feature-by-feature convenience entry points. - HTTP, preview, screenshot, and project-session paths are hardened for longer autonomous workflows. - Craft compatibility work is read-only and provenance-aware; it does not silently opt ordinary projects into Craft behavior. - A minimal-context evaluation harness exercises key workflows with a weaker model and stores bounded, privacy-safe evidence. ## Validation - Rebased on the latest `origin/main`. - `pnpm lint` - Repository-wide `oxfmt --check` - CLI typecheck and full test suite: 773 tests passed. - Project-build typecheck and full test suite: 1,866 tests passed. - Protocol typecheck and full test suite: 83 tests passed. - HTTP client tests: 23 passed. - CSS data tests: 3,620 passed, 1 skipped. - Focused Builder tests: 67 passed. - Package-boundary, generated API, and generated documentation checks passed. - Minimal-context agent evaluations passed for authenticated resource-backed UI and design-system-aware responsive authoring.	2026-07-16 17:53:16 +02:00
Oleg Isonen	dde30ffb54	feat: Improve MCP authoring, compact output, and agent onboarding (#5883) ... ## Summary This improves how agents discover, author, inspect, and verify Webstudio projects while reducing unnecessary MCP context usage. ## What changed - Adds an **Agents** section to Project Settings with ready-to-run setup commands and client configuration for Claude Code, Codex, Cursor, and VS Code. - Makes large list and schema responses compact and paginated by default, with **verbose** available when complete records or diagnostics are needed. - Applies consistent compact output to assets, instances, styles, design tokens, CLI schemas, and project-session metadata. - Returns actionable expression warnings for props, resources, and Collection bindings, including unknown variables and risky render-time mutation resources. - Improves Collection expression guidance so agents can bind array items with the correct scoped variables. - Reports rendered audit outcomes explicitly as complete, partial, confirmation-required, or failed. - Expands runtime output schemas so MCP clients can validate responses reliably. - Strengthens the packaged CLI smoke workflow with a realistic seven-page stdio MCP scenario. ## Architecture - Consolidates `project-build` from 65 file-oriented entrypoints to 11 architecture-owned surfaces. - Separates the core model, contracts, runtime authoring, transfer, state, project sessions, MCP, visual tooling, persistence codecs, and authenticated server services. - Isolates Node-only JSX evaluation from browser-safe transfer utilities and authenticated server modules. - Keeps persistence codecs independent from authenticated database services, preventing unrelated module initialization. - Defines shared instance-path types once and reuses them across runtime modules. - Preserves Immer initialization through explicit package side-effect metadata. - Adds package tests that prevent a return to source-file entrypoints or generic exposure of Node-only surfaces. ## Validation - Full repository checks, tests, typechecks, generated-file checks, and fixtures - All 163 Builder test files and 2,141 tests - All 71 project-build test files and 1,794 tests - Lint and package-boundary validation - Full packed CLI release smoke test, including install, preview, agent setup, seven-page stdio MCP workflow, rendered audit, and structured commands	2026-07-15 18:11:28 +02:00
Oleg IsonenandClaude Fable 5	071a4d9a85	feat: MCP update 3 (#5864) ... ## Summary This update adds new MCP and CLI capabilities for project auditing, structured data, project-specific agent instructions, focused project search, safer bulk editing, image descriptions, font discovery, visual verification, and external registry inspection. ## New Capabilities ### Audit projects - Run a single `audit` command across accessibility, SEO, security, assets, styles, and performance. - Limit an audit to selected scopes, pages, or severity levels. - Receive stable findings with locations and suggested fixes. - Run rendered audits across desktop, mobile, and Builder breakpoint widths. - Detect horizontal overflow, broken images, oversized images, eager below-fold images, render-blocking resources, and legacy font formats. - Identify unused assets, tokens, CSS variables, local styles, and breakpoints. - Identify duplicate token declarations. - Detect accessibility issues involving names, labels, headings, landmarks, IDs, keyboard focus, ARIA, iframes, and autoplaying media. - Detect SEO issues involving titles, descriptions, language, social images, and structured data. - Detect unsafe new-tab links and write resources exposed during page rendering. ### Search an entire project - Search project text, labels, props, resources, assets, and styles through `search-project`. - Find relevant records without downloading and reparsing the full project snapshot. - Scope searches to the relevant project-data areas. ### Add validated JSON-LD - Add structured data with a dedicated JSON-LD component. - Place JSON-LD inside the page Head Slot. - Edit JSON in a purpose-built code editor. - Prevent structurally invalid JSON-LD from being saved. - Validate Schema.org types, properties, and values. - Find invalid, missing, unknown, deprecated, or incompatible structured data through the SEO audit. - Let agents discover the correct JSON-LD workflow without using page custom metadata or a generic HTML embed. ### Configure project-specific agent instructions - Add AI-agent instructions from the new **Agents** section in Project Settings. - Synchronize those instructions into a managed root `AGENTS.md`. - Update or remove the managed file automatically during project sync. - Protect existing user-owned `AGENTS.md` files from being overwritten. - Share the instructions with authenticated CLI and MCP agents. - Keep agent instructions out of publicly readable production bundles. ### Generate and manage image descriptions - Audit image assets for missing descriptions. - Let a vision-capable agent inspect each image in its rendered context. - Save generated descriptions to multiple image assets in one operation. - Mark images as intentionally decorative. - Reuse the saved asset description wherever the image is used. - Re-run the audit to verify that missing-description findings are resolved. ### Perform bounded bulk replacements - Replace matching literal text across selected pages with `replace-text`. - Replace matching fixed prop text with `replace-prop-text`. - Replace matching resource names or URLs with `replace-resource-text`. - Restrict replacements by page, instance, prop, resource, field, match mode, and maximum change count. - Leave dynamic expressions unchanged. ### Inspect element ancestry - Return an element’s direct parent in instance listings. - Request the complete ancestor chain when inspecting an element. - Understand tree placement without rebuilding the instance hierarchy from raw project data. ### Discover available fonts - List uploaded font families through `list-fonts`. - Discover available system font stacks. - Inspect font family, style, weight, format, and associated asset information before applying typography. ### Inspect external component registries - Inspect a local or remote shadcn-style registry item. - Read its description, files, dependencies, targets, and documentation. - Receive an explicit Webstudio compatibility report. - Inspect registry metadata without installing or executing external code. ### Capture generated project routes directly - Run `webstudio screenshot --path /route` against the configured project. - Generate a temporary production-style preview automatically. - Capture the requested route. - Stop the temporary preview after capture. - Avoid manually building and starting a preview for a single screenshot. ### Assert visual changes - Add expected text to screenshot comparisons. - Set maximum pixel mismatch limits. - Set minimum or maximum changed-region counts. - Assert whether overall brightness or a color channel moved in the expected direction. - Use these assertions as pass/fail evidence for visual changes. ### Require compatible CLI versions - Send the project-bundle contract version with CLI synchronization requests. - Stop incompatible CLI versions before they receive incomplete project data. - Tell users exactly how to update the CLI or run the latest version once. - Prevent older clients from silently dropping project settings. ## Upgrade Note This branch introduces a new synchronized project-bundle contract. Older CLI versions must update before synchronizing against the new API. ```sh npm install -g webstudio@latest ``` Or run the latest version once: ```sh npx webstudio@latest <command> ``` --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-07-12 11:27:58 +02:00
Oleg Isonen	d291e4b671	feat: Builder api runtime contracts guardrails for MCP (#5857) ... This PR expands Webstudio MCP and CLI into a more complete agent authoring interface. Agents can discover project capabilities, edit real Webstudio content, use data/resources, verify visual output, and report useful issues when workflows fail. ## MCP User Capabilities - Create, update, duplicate, reorder, and delete pages. - Create pages from templates and manage reusable page templates. - Inspect project pages, instances, props, styles, assets, variables, resources, and metadata. - Insert Webstudio components and authored JSX fragments into pages. - Edit text, props, styles, breakpoints, design tokens, and CSS variables. - Update page metadata, SEO fields, redirects, marketplace metadata, and project settings. - Create and use static data variables. - Create and use dynamic resources fetched over HTTP. - Use REST, GraphQL, and built-in system resources in authored pages. - Bind data variables and resources to visible page content. - Create resource-backed pages such as sitemap XML, structured SEO data, and API-driven content sections. - Preview current project state without requiring a publish step. - Capture screenshots for visual verification. - Verify responsive designs across desktop, tablet, and mobile breakpoints. - Use screenshot diff and OCR-style checks to inspect rendered output. - Reset and refresh local MCP project state when needed. - Discover available CLI/MCP commands from generated manuals. - Use clearer examples for resources, data variables, JSX authoring, previewing, and visual checks. - Read marketplace product metadata without requiring a full project snapshot. - Get safer guidance when using low-level patch tools. - Get a ready-to-copy Discord report template when MCP/CLI behavior is confusing or broken.	2026-07-10 02:13:22 +02:00
Oleg Isonen	1ddf6b02c9	feat: API and CLI for everything (#5838) ... Closes #5287	2026-07-01 01:20:50 +02:00
Oleg Isonen	e7b7faeca5	fix: performance optimizations (#5827)	2026-06-19 12:23:44 +02:00
Oleg Isonen	ac29cf7d1d	feat: Keyboard shorthands for instance positioning and slot bug fixes (#5821) ... Closes https://github.com/webstudio-is/webstudio/issues/4586 Closes https://github.com/webstudio-is/webstudio/issues/4619 ## Summary Fixes shared Slot editing so linked Slot occurrences stay consistent across structural edits and labels. ## Changes - Fixed moving a direct Slot child out of a Slot so the child is removed from all linked Slot occurrences. - Fixed unwrapping a direct Slot child so the child is removed from all linked Slot occurrences. - Added linked Slot label sync when renaming from Navigator or Settings panel. - Added/updated tests for canonical Slots, legacy Slots, nested Slot children, keyboard moves, unwrap, reparent, and label sync. - Simplified stale Slot detach logic that was incorrectly cloning only the selected Slot occurrence. ## Testing - `pnpm --filter @webstudio-is/builder test -- app/shared/instance-utils/...` - `pnpm --filter @webstudio-is/builder test -- app/builder/shared/commands.test.ts` - `pnpm --filter @webstudio-is/builder typecheck` - `pnpm lint` - `git diff --check`	2026-06-16 02:25:28 +02:00