mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-11-03 02:27:57 +09:00
agent/validate-css-before-publish
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oleg Isonen
|
db2d092268 |
ci: replace Lost Pixel with self-hosted visual regression (#6086)
## Summary Replace the defunct Lost Pixel service and the interim Chromatic workflow with a fast, self-hosted visual comparison pipeline. The workflow indexes the existing CSF stories without building Storybook, builds and serves a lightweight browser app with Vite, renders the merge base and current revision through Webstudio's shared browser tooling, and compares screenshots with Webstudio's pixel-diff and OCR engine. It uses GitHub Actions cache and artifacts only—no R2 bucket or external visual-testing service. Closes #6064 ## Implementation - [x] Remove the Lost Pixel workflow, configuration, acknowledgement, and global two-second wait. - [x] Remove the interim Chromatic workflow and settings. - [x] Extract source-agnostic capture, diff, OCR, audit, cache, and report tooling into `@webstudio-is/vision`; rename the Webstudio-specific preview boundary to `@webstudio-is/project-build/preview`. - [x] Expose the reusable browser capture session through the shared vision boundary. - [x] Make OCR and diff-artifact generation conditional on detected pixel changes. - [x] Render existing CSF stories through a typed, lightweight browser harness built with Vite. - [x] Replace the embedded HTML and handwritten HTTP/MIME server with a real HTML entry, Vite production build, and Vite preview server; isolate declarative build configuration in `story-build.ts`. - [x] Apply Storybook preview annotations, decorators, global CSS, backgrounds, and story metadata. - [x] Store story scopes as shared data and load each revision's own configuration. - [x] Discover changed, added, and removed stories from both revisions. - [x] Populate private story sources symmetrically in the baseline worktree. - [x] Isolate every story with a fresh React root and deterministic time, randomness, intervals, focus, animation, and layout readiness. - [x] Preserve story-owned visual options for masking, settling, and interval control. - [x] Reuse Vite preview servers, one Chromium process, grouped browser pages, production bundles, and a bounded CPU worker pool for image comparison. - [x] Recheck initial pixel differences with serial captures, then recapture persistent baseline/current pairs on the same browser page so renderer variance cannot create false failures. - [x] Report individual story render failures without discarding successful captures. - [x] Enforce a 0.1 per-pixel threshold and 0.001% total mismatch tolerance. - [x] Run OCR only for changed comparisons and include text-change evidence. - [x] Move the generic report model, writer, static viewer, and opener into the shared Vision package for reuse by CI and future CLI comparisons. - [x] Bundle report HTML, styling, and browser behavior as static Vision assets without runtime source-file lookups. - [x] Copy external screenshot inputs into self-contained report artifacts for CLI reuse. - [x] Handle cached removed-only comparisons without rendering the removed story on the current server. - [x] Support focused local reproduction and opening the report with one command. - [x] Pin CI to the lockfile's Playwright Chromium, cache complete revision screenshots with matching restore/save paths and automatically derived runtime invalidation, and render both revisions on a miss. - [x] Keep cache archives bounded to one revision instead of accumulating the cache history. - [x] Run the complete visual suite in one CI job and upload one report as a 14-day artifact. - [x] Fail PR checks on visual differences, empty runs, missing screenshots, missing or empty report artifacts, and infrastructure failures. - [x] Rerun fork checks when `safe-to-deploy` is added. - [x] Allow reviewed visual-only differences through `visual-change-approved` and invalidate approval after new commits. - [x] Keep capture, pixel diff, OCR, and report generation source-agnostic for future draft-versus-published project comparisons. - [x] Use Vite for the complete story HTML, module, asset, production-build, and preview-server lifecycle; inline the selected stories and skip ineffective tree-shaking to keep the cached run under one minute. - [x] Review Builder product documentation impact: no user-facing Builder documentation is affected; contributor usage is documented in `scripts/visual-regression/README.md`. - [x] Review fixture impact: no fixture inputs or generated fixture outputs are affected. ## Verification - [x] Project-build Vision tests, including the reusable report test, and entrypoint contract tests pass. - [x] CLI screenshot, preview, and browser-session tests pass. - [x] Project-build, CLI, Builder, design-system, and visual-runner typechecks pass. - [x] Visual runner tests pass (10), and the complete Vision package passes (79), including demonstrated fail-before/pass-after coverage. - [x] Full Storybook production build passes. - [x] Full local Vite run completes all 238 stories in 69.9s cold and 50.3s with a cached baseline plus two same-page serial verifications, with zero differences; identical-image comparison remains sub-second. - [x] The report writer passes a bundled-output smoke test and materializes external screenshot assets. - [x] The generated report validates all 238 entries and referenced images. - [x] Hosted cached-baseline comparison completes all 238 stories in 52.9s with zero differences; the cache-miss fallback completes in 101.3s. - [x] Cache-miss fallback renders both Git revisions on demand. - [x] Visual differences produce baseline, current, diff, OCR, JSON, and HTML evidence and fail without approval. - [x] Hosted single-job visual workflow restores both the pinned browser and baseline cache, validates all 238 public and private stories with zero differences, and uploads the report. - [x] The final Vite production build renders all 238 stories without dependency invalidation or duplicate React runtimes, and restores `NODE_ENV` so cached serial fallback installs the baseline's dev dependencies. - [x] Root lint, package-boundary validation, and relevant formatting checks pass. - [x] Workflow YAML parses successfully. - [x] `git diff --check`, staged-diff review, and final worktree review pass. ## Usage Run the complete local comparison and open the visual report: ```sh pnpm visual-regression:ui ``` Focus one component while iterating: ```sh pnpm visual-regression:ui -- --grep "Design system/Button" ``` Reopen the latest report without rerunning: ```sh pnpm visual-regression:report ``` The CI report is available as the `visual-regression-report` artifact. ## Tradeoffs - Baseline screenshots are a disposable acceleration cache, not source of truth. A missing or invalid cache renders the merge base directly from Git. - GitHub artifact storage provides downloadable one-click reports; permanently hosted browser links would require another hosting surface. - OCR runs after pixel triage because invoking Tesseract for every unchanged story would make the common path unnecessarily slow. - Vite production builds add Rollup's module graph and HTML/asset pipeline. The runner emits one bundle because every selected story is rendered, and skips tree-shaking because it removed little while increasing build time. - The verified local path is 50.3s cached and 69.9s cold. The hosted path is 52.9s cached and 101.3s cold. A new browser/runtime cache must render its baseline once; the pinned browser keeps later runners on the reusable path. - The complete suite stays in one Actions job; browser-page concurrency is internal and bounded. |
||
|
Oleg Isonen
|
5c6ffd3148 |
docs: document Content Engine workflows (#6097)
## Summary - add a step-by-step Content Engine guide for building a Markdown article overview and dynamic article page - clarify that the example folder structure is optional and link to the Markdown Blog marketplace template - explain that an AI agent can complete the entire workflow through Webstudio MCP and include a ready-to-use prompt - document frontmatter, JSON files, relative and published asset paths, document references, draft filtering, and single-result resources - document Markdown-to-Markdown, Markdown-to-JSON, JSON-to-Markdown, and JSON-to-JSON document references, including fragments and JSON Pointer escaping - add a schema-aware `webstudio man content-engine` reference for fields, operators, saved and preview inputs, result/output/content modes, diagnostics, limits, and reference syntax - generate the GitBook Content Engine reference and the matching MCP manual section from that shared CLI source - fail documentation generation when result modes, output modes, content modes, diagnostic fields, or Content Engine limits drift from the implementation - add four focused visual-editor screenshots - keep the Assets page focused on asset management and link it to the dedicated Content Engine guide - update MCP Assets examples and the Markdown blog recipe to use Markdown files, `result: "one"`, direct detail bindings, page metadata, and visual verification ## Impact Users get one workflow for treating files as the source of truth while designing article listings and detail pages in Webstudio. The generated reference documents the complete Assets query contract and document-reference system. Agents receive the same reference through the MCP manual, so the product and agent documentation no longer need separate factual copies. This does not change product runtime behavior. ## Checklist - [x] Documentation impact reviewed - [x] Added the Content Engine guide and generated reference to `docs/SUMMARY.md` - [x] Updated related Assets, Data variables, and MCP documentation - [x] Verified internal documentation and image targets - [x] Verified the Markdown Blog marketplace URL returns HTTP 200 - [x] Confirmed the MCP recipe test failed before the recipe correction and passed afterward - [x] Confirmed the schema-coverage test fails when an implemented result mode is omitted and passes after restoration - [x] Confirmed both Content Engine generator tests fail without their corresponding generator behavior and pass after restoration - [x] Ran `pnpm --filter @webstudio-is/project-build test -- src/mcp.test.ts` - [x] Ran `pnpm --filter webstudio test -- src/commands/api-command-docs.test.ts` - [x] Ran the Content Engine and MCP generator tests - [x] Ran project-build and CLI typechecks - [x] Ran `pnpm check:package-boundaries` - [x] Ran targeted lint and formatting checks - [x] Ran `pnpm check:generated-docs` - [x] Verified the final diff and working tree - [ ] Agent evaluation suite: not run because it requires explicit approval Ref #5973 |
||
|
Oleg Isonen
|
1f4095cf9a |
fix: Improve generated MCP documentation (#5991)
## Summary
- render the generated MCP version as visible body content immediately
below the page heading
- keep the GitBook navigation label stable as `MCP`
- stamp only the CLI version placeholder during release publication,
leaving Markdown content owned by the template
- remove empty `{}` blocks from the rendered MCP argument examples while
preserving meaningful examples and complete structured JSON data
## Root cause
GitBook uses the `SUMMARY.md` link label as the published page heading
and suppresses the Markdown H1 from the document body. The generated
version was therefore present in source but absent from the rendered
page.
The release workflow also coupled version stamping to the complete
rendered Markdown line. It now replaces and verifies only the standard
`0.0.0-webstudio-version` placeholder.
## Verification
- live page structure inspected: published H1 is `MCP`
- MCP manual regression test demonstrated failing before empty argument
blocks were filtered, then passing after the fix
- release placeholder stamping simulated against the generated page
- `pnpm --filter webstudio exec vitest run src/commands/man.test.ts`
- `pnpm --filter @webstudio-is/project-build typecheck`
- `pnpm --filter webstudio typecheck`
- `pnpm check:generated-docs`
- `pnpm lint`
- `pnpm oxfmt --check packages/project-build/src/mcp.ts
packages/cli/src/commands/man.ts packages/cli/src/commands/man.test.ts`
- `git diff --check`
Fixture inputs and outputs are unaffected.
## Todo
- [x] Generate the release version as visible GitBook body content.
- [x] Decouple release stamping from the rendered Markdown content.
- [x] Remove empty MCP argument example blocks.
- [x] Validate generated output and regression coverage.
- [ ] Merge this pull request.
- [ ] Publish the latest released revision to the `docs` branch.
- [ ] Verify the version on the live GitBook page.
Ref #5986
|
||
|
Oleg Isonen
|
e6a29b5d36 |
fix: Show released CLI version in MCP docs (#5990)
## Summary - make the MCP page generator render the CLI package version directly in the page title - publish `Webstudio MCP v<release>` from the exact generated page in each successful release - merge each successful release tag into the publication-only `docs` branch - serialize publication and ignore reruns or older release tags already contained in `docs` ## Root cause The first publication design advanced `docs` to the release commit without putting the CLI package version into the generated page. The repository uses `0.0.0-webstudio-version` until release time, so the release workflow now replaces that existing version placeholder with the successful tag version before publishing. ## Todo - [x] Generate the CLI version in the MCP page title. - [x] Publish the generated page from the exact release revision. - [x] Make publication idempotent and prevent version downgrades. - [x] Validate generation, formatting, linting, and release-branch behavior. - [ ] Merge this pull request. - [ ] Publish `0.285.0` as the initial version on the existing `docs` branch. ## Verification - `pnpm docs:mcp:test` - `pnpm docs:mcp:typecheck` - `pnpm check:generated-docs` - `pnpm lint` - `pnpm oxfmt --check` - `git diff --check` - isolated Git simulation for release merge, exact generated-page selection, version replacement, and rerun detection - regression tests demonstrated failing without version rendering and validation, then passing after restoration Ref #5986 |
||
|
Oleg Isonen
|
99371187dc |
docs: Move product documentation into Builder (#5987)
## Summary - move the complete GitBook documentation tree and assets into Builder under `/docs` - generate the exhaustive MCP page from the local CLI source in the same change - publish GitBook from a release-gated `docs` branch containing the exact successfully released revision - run MCP documentation tests, typechecking, and stale-output checks with the existing generated-docs check - remove links to the Community Discussions repository ## Technical choices - Import the current documentation as a linear snapshot so the branch satisfies Builder's required-linear-history protection. - Keep the Markdown wrapper separate from the TypeScript generator at `scripts/docs/mcp-page.md`. - Generate through `packages/cli/local.js` so CLI guidance and its documentation are reviewed together. - Let `check:generated-docs` reject CLI changes that leave the committed MCP page stale. - After both GitHub release creation and package publication succeed, fast-forward the read-only `docs` branch to the immutable release tag commit. A divergent update fails instead of publishing a different revision. - Configure GitBook to publish `/docs` from the `docs` branch, not directly from `main`. - Exclude `/docs` from the repository-wide `oxfmt` scan so imported GitBook Markdown retains its established formatting; generator tests and stale-output checks still validate the MCP page. - Exclude `transcripts/` and GitHub Discussions from the migration. ## Todo - [x] Import documentation, navigation, includes, and assets under `/docs`. - [x] Generate the MCP page from the local CLI source. - [x] Move MCP generation and validation into Builder. - [x] Require MCP documentation updates alongside CLI changes. - [x] Publish released revisions through a read-only `docs` branch. - [x] Remove Community repository links from Builder and the migrated docs. - [x] Preserve and verify a final Community Git bundle locally. - [ ] Merge this pull request. - [ ] Let the next successful release create or advance the `docs` branch. - [ ] Repoint GitBook Git Sync to `webstudio-is/webstudio`, branch `docs`, root directory `/docs`. - [ ] Verify `docs.webstudio.is`, navigation, assets, anchors, and the MCP page in production. - [ ] Delete `webstudio-is/webstudio-community`. ## Verification - `pnpm check:generated-docs` - `pnpm lint` - `pnpm oxfmt --check` - `git diff --check origin/main...HEAD` - generator regression test demonstrated failing when the GitBook wrapper behavior was removed, then passing after restoration - no fixture inputs or outputs are affected ## Known limitation All documentation merged after the previous release waits for the next successful release before GitBook publishes it. This deliberately keeps the public MCP reference aligned with an installable CLI revision. Ref #5986 |
||
|
Oleg Isonen
|
d291e4b671 |
feat: Builder api runtime contracts guardrails for MCP (#5857)
This PR expands Webstudio MCP and CLI into a more complete agent authoring interface. Agents can discover project capabilities, edit real Webstudio content, use data/resources, verify visual output, and report useful issues when workflows fail. ## MCP User Capabilities - Create, update, duplicate, reorder, and delete pages. - Create pages from templates and manage reusable page templates. - Inspect project pages, instances, props, styles, assets, variables, resources, and metadata. - Insert Webstudio components and authored JSX fragments into pages. - Edit text, props, styles, breakpoints, design tokens, and CSS variables. - Update page metadata, SEO fields, redirects, marketplace metadata, and project settings. - Create and use static data variables. - Create and use dynamic resources fetched over HTTP. - Use REST, GraphQL, and built-in system resources in authored pages. - Bind data variables and resources to visible page content. - Create resource-backed pages such as sitemap XML, structured SEO data, and API-driven content sections. - Preview current project state without requiring a publish step. - Capture screenshots for visual verification. - Verify responsive designs across desktop, tablet, and mobile breakpoints. - Use screenshot diff and OCR-style checks to inspect rendered output. - Reset and refresh local MCP project state when needed. - Discover available CLI/MCP commands from generated manuals. - Use clearer examples for resources, data variables, JSX authoring, previewing, and visual checks. - Read marketplace product metadata without requiring a full project snapshot. - Get safer guidance when using low-level patch tools. - Get a ready-to-copy Discord report template when MCP/CLI behavior is confusing or broken. |
||
|
Bogdan Chadkin
|
b6f87bac06 |
build: stop building in checks (#3831)
-1 minute in checks |
||
|
Andras Bacsai
|
9fb3c54258 |
Release 0.42.0 (#1165)
## Description 1. What is this PR about (link the issue and add a short description) ## Steps for reproduction 1. click button 2. expect xyz ## Code Review - [ ] hi @kof, I need you to do - conceptual review (architecture, feature-correctness) - detailed review (read every line) - test it on preview ## Before requesting a review - [ ] made a self-review - [ ] added inline comments where things may be not obvious (the "why", not "what") ## Before merging - [ ] tested locally and on preview environment (preview dev login: 5de6) - [ ] updated [test cases](https://github.com/webstudio-is/webstudio-builder/blob/main/apps/builder/docs/test-cases.md) document - [ ] added tests - [ ] if any new env variables are added, added them to `.env.example` and the `builder/env-check.js` if mandatory |