Giuseppe
b3dd7f18c4
Implements an operation-based system to accomplish a number of edit tasks on a JSX+CSS representation of a selected Webstudio tree. Every JSX element has the instance id as data-ws-id and therefore the LLM knows what to edit. The idea is to create a number of operations (aiOperations) specs/schemas with Zod describing unique edit tasks, make a discriminated union, convert them to JSON Schema. We then inject this information and the current JSX+CSS in the LLM prompt asking the model to generate an array of operations. The LLM responds with an array of operations (aiOperations) to fulfill the user request. We then process these doing the necessary transformations to convert these to a Webstudio-friendly format (wsOperations) and send them to the client where we apply them (see applyOperations). Some of these operations are going to be "intermediate" operations for now. Generating quality user interfaces might be challenging with LLMs. Having to do so within a generic operations framework like the one described above makes it even more challenging. For this reason when it comes to generating user interfaces, the Operation chain instead offers an operation called generateTemplatePrompt that provides information about insertion point in the existing UI and a prompt, possibly enhanced. Another chain can use these information and a more sophisticated prompt and model to then generate the user interface and return a generateInsertTemplate wsOperation. Webstudio does exactly this in a rest endpoint, using the template-generator chain with gpt4 instead of gpt-3.5-turbo.
1.1 KiB
Webstudio AI
The Webstudio AI package offers two main features:
LLM Abstraction
A minimal abstraction for a generic, vendor-agnostic messages format and LLM client, allowing for building features without using model-specific APIs.
See details.
An implementation of the above that uses the OpenAI chat completion endpoint is available here.
Chains
A common interface for chains. Chains are regular async functions that can execute an arbitrary number of steps, including calling a LLM via a model client.
See details.
Usage
Install the package
pnpm i @webstudio-is/ai
For an example usage check out the base example and the copywriter chain that generates copy for Webstudio.
Prompt templates
This package comes with a small CLI that allows to turn prompt templates from markdown files to TypeScript modules, which is what you would use in your chain.
pnpm run build:prompts