Oleg Isonen
8e7a8a21aa
Closes #6020 ## Summary Adds the first end-to-end slice for anonymous, LLM-authored reliability reports from the CLI and MCP. The shared report contract captures the generalized user story, attempted workflow, expectation, actual result, recovery attempts, user impact, technical context, acceptance criteria, and agent runtime (client, provider, model, version, and reasoning effort). Guidance requires removing any user- or project-identifying information before submission. ## Architecture - Adds a project-authenticated `api.reports.issue` tRPC mutation as the reusable product boundary. - Maps that mutation to the public `report-issue` operation for CLI and MCP callers. - Keeps the endpoint directly callable from a future Builder UI through the existing tRPC client. - Publishes through a server-side GitHub App installation, so end users do not need GitHub accounts or tokens. - Discovers the App installation for the fixed Webstudio reporting repository instead of requiring deployment-specific installation or repository configuration. - Searches for a hidden anonymous technical marker before creation to avoid obvious duplicates. ## Todo 1. [x] Define a shared anonymous LLM-authored report contract. 2. [x] Include the user story, attempted workflow, expected and actual behavior, recovery attempts, impact, technical context, and acceptance criteria. 3. [x] Include the LLM client, provider, model, client version when known, and reasoning effort. 4. [x] Attach sanitized CLI, Node.js, operating-system, architecture, execution-mode, and API-contract metadata automatically. 5. [x] Add the Builder-reusable project tRPC mutation. 6. [x] Expose the mutation to CLI and MCP as `report-issue`. 7. [x] Add GitHub App authentication and issue publication. 8. [x] Reduce deployment configuration to the App ID and private key. 9. [x] Add marker-based duplicate detection. 10. [x] Add guidance for explicit user-requested reports and qualifying repeated-friction reports. 11. [x] Review and simplify repeated GitHub request handling and public protocol exports. 12. [ ] Add server-side throttling and harden concurrent duplicate creation. 13. [x] Configure and install the reporting GitHub App in deployed environments. 14. [x] Exercise the deployed endpoint with a controlled test report. 15. [ ] Run the complete agent evaluation suite after explicit approval. 16. [x] Review documentation impact and update the generated MCP documentation. ## Verification - Protocol issue-report contract tests: passed (3) - Builder GitHub reporting tests: passed (4) - Builder API router tests: passed (30) - CLI schema, command-doc, MCP, and project-session tests: passed - Protocol, HTTP client, project-build, CLI, and Builder typechecks: passed - Root lint: passed - Generated API and documentation checks: passed - New and changed regression tests were demonstrated failing against the previous behavior, then passing after implementation - No fixture inputs or outputs are affected - Agent evaluations were not run because they require explicit approval ## Deployment notes The Builder only needs `GITHUB_ISSUE_REPORT_APP_ID` and `GITHUB_ISSUE_REPORT_PRIVATE_KEY`. The server discovers the App installation for `webstudio-is/webstudio` and creates a short-lived installation token. The App needs repository metadata read access and Issues write access. ## Known limitations Duplicate detection currently uses GitHub issue search and can race when identical reports arrive concurrently. The endpoint is project-authenticated, but dedicated server-side throttling remains a follow-up before production rollout. No Builder UI is included yet; the tRPC boundary is intentionally ready for one.
Webstudio CLI
The Webstudio CLI helps you to link, sync and build projects from Webstudio Cloud. This README will guide you through the process of setting up a Webstudio project on your local machine and continually sync it with the cloud.
Prerequisites: Installing Node.js
You need Node.js to use the Webstudio CLI. If Node.js is already installed in your system, you can skip ahead to the section on installing the Webstudio CLI.
To install Node.js using NVM, first install NVM by running:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash
Once NVM is installed, you can install Node.js version 22 by running:
nvm install 22
Verify your Node.js installation by checking its version:
node --version
Installing the Webstudio CLI
To get started with the Webstudio CLI:
-
Download and install the CLI using the following command:
npm install -g webstudio@latest -
Confirm the installation by checking the CLI version:
webstudio --version -
To keep your CLI updated, use the same command used for installation whenever a new release is available.
Initiating a Webstudio Project
Now, you can run a Webstudio project on your local machine using this command:
webstudio
This will initiate the flow to connect your Webstudio Cloud project and build it in your local computer. The default flow will guide you through the steps. You can also perform all the operations individually using independent commands.
Commands
Here is the list of independent commands:
- version
- help
- link
- sync
- build
link
The link command stores access to a Webstudio Cloud project. MCP clients use that connection to read and edit the latest Builder build directly; the project does not need to be published or synchronized first.
Shell-capable agents can call MCP tools directly as CLI subcommands immediately after linking. This does not require MCP client configuration or a client restart:
webstudio meta.index
webstudio components.search '{"brief":"button"}'
webstudio insert-fragment --input-file .temp/insert-fragment.json --dry-run
.temp/insert-fragment.json:
{
"parentInstanceId": "<parent-id>",
"fragment": "<ws.element ws:tag='section' />"
}
The explicit equivalent is webstudio mcp single-op-call <tool> '<json>'.
Use webstudio connect <client> only when you want persistent native MCP tools
inside Claude Code, Codex, Cursor, or VS Code. A persistent client may need to
reconnect after its configuration or cached tool catalog changes.
See the Webstudio MCP documentation for setup, discovery, editing, verification, and safety guidance.
You can link a project from Webstudio Cloud with the following command:
webstudio link
This command will prompt you to paste a link which you can create using the Share option in your project.
Make sure to provide Build access when generating the link in Webstudio Cloud.
sync
Once the project is linked, use the sync command to sync it with the cloud:
webstudio sync
Make sure to publish the project in Webstudio Cloud before running the sync command in your local Webstudio project.
sync is only for downloading a published project export. It is not part of the MCP editing workflow.
build
Now, you can build your project with the build command:
webstudio build
During this phase, the CLI establishes the necessary routes and pages, scaffolding the entire application using the default Remix template. Additionally, all assets, such as images and fonts are downloaded to the assets folder inside the public directory.
Once the project is scaffolded, you can run npm install and then npm run dev to run your app in development mode.
If you want to build a production version of the app, you can run npm run build.
Deployment
Vercel
Once you've built the project locally, you can use the Vercel CLI to deploy your project directly to Vercel:
vercel deploy
Follow the instructions here to install the vercel CLI. We plan to add more deployment targets in future.
Netlify
If you want to deploy to netlify, you can use Netlify CLI to deploy your project directly to Netlify:
netlify deploy
You can configure the project to support netlify serverless/edge-functions respectively, as deployment target at the time of initially setting up your project. Please check the initiating-a-webstudio-project section.
You can manually change it using the build command. For serverless functions:
webstudio build --template netlify
Important Notes
If you use vercel build before vercel deploy, make sure to clean your app folder in the project afterward.
Vercel injects a few files to support and deploy Remix using their CLI, but these files are not necessary for your project when you use it locally.