0 Fork 0

mirror of https://github.com/webstudio-is/webstudio.git synced 2026-10-28 23:57:17 +09:00

Files

T

Oleg Isonen 4273b7001f refactor: Organize Builder development and test tooling (#6285 )

## Outcome

Builder development, backend, E2E, and CI tooling now share one
ownership model: Builder defines named commands and backend modes, while
CI only schedules those commands.

## Technical choices

- Group shared Compose files, PostgreSQL initialization, and backend
helpers under `apps/builder/backend`.
- Resolve `compose.<mode>.yaml` by convention instead of maintaining
Compose filename lists in scripts and workflows.
- Keep persistent development and disposable test Compose projects
separate by default, while allowing the standard `COMPOSE_PROJECT_NAME`
override.
- Make schema snapshots invalidate from discovered migration sources and
backend files.
- Move E2E behavior recipes into named Builder commands; retain concise
root proxies for developer entry points.
- Let Vitest allocate available browser ports instead of maintaining a
cross-package port registry.
- Keep test-suite scheduling explicit in `main.yml`, but move executable
test commands to root package scripts.
- Move fixture-size enforcement from inline GitHub JavaScript into a
locally runnable repository script.

## Todo

- [x] Consolidate Builder backend and Compose ownership.
- [x] Add named development, E2E, schema-cache, Supabase, and CI-shard
commands.
- [x] Remove Compose topology and behavior flags from CI.
- [x] Remove the static Vitest browser-port registry.
- [x] Extract fixture-size checking from the workflow.
- [x] Verify migration bootstrap, snapshot-backed E2E, tests, builds,
types, formatting, lint, generated contracts, and workflow syntax.
- [x] Review documentation impact: updated the E2E developer guide; no
user-facing `docs/` content is affected.

## Verification

- Full package, CLI, Builder core, and Builder app test suites
- Full workspace typecheck
- Builder production build
- Schema-cache generation through the complete migration history
- Supabase migration compatibility
- Snapshot-backed Playwright animation workflow
- Backend regression tests, including demonstrated fail/pass coverage
- All four Compose mode configurations
- `oxfmt`, `oxlint`, package boundaries, generated API, and generated
docs checks
- `actionlint` and Bash syntax checks

The fixture-size checker preserves the existing Linux `du` thresholds.
Its three inputs build locally; macOS allocation sizes are not directly
comparable to the Linux CI thresholds, so CI remains the authoritative
size result.

2026-08-31 21:09:54 +02:00

4.3 KiB

Raw Permalink Blame History

Builder E2E tests

Builder E2E workflows run with Playwright Test. The Playwright setup project waits for PostgREST, resets the disposable database, and warms the login route before Chromium runs the selected tests.

Run the suite

Use the full command for CI-like validation:

pnpm e2e:builder

The first run after migrations change applies the regular migration pipeline and writes an ignored schema cache under .cache/builder-e2e. Refresh it explicitly with:

pnpm e2e:builder:refresh-schema-cache

The regular E2E suite uses lightweight PostgreSQL containers. Verify that the complete migration chain also applies to the pinned Supabase PostgreSQL image with:

pnpm e2e:builder:check-supabase-migrations

Pass normal Playwright Test arguments after --. For example:

pnpm e2e:builder -- --grep "Builder can copy, duplicate, and delete a page"
pnpm e2e:builder -- 'apps/builder/e2e/tests/pages-actions.[shard-5].e2e.ts'
pnpm e2e:builder -- --debug --grep "Builder can copy"

A lightweight Playwright discovery pass validates the selection before Docker, database migrations, browser installation, or the Builder build starts. A filter that matches no tests therefore fails without starting the test environment.

Fast local reruns

Keep the E2E backend and Builder development server running to avoid rebuilding Builder and restarting Docker for each edit.

Terminal 1:

pnpm e2e:builder:dev:backend

Terminal 2:

pnpm dev

Terminal 3:

pnpm e2e:builder:dev -- --grep "Builder can copy, duplicate, and delete a page"

pnpm e2e:builder:dev defaults to E2E_BUILDER_URL=https://127.0.0.1:3000. Override it when the development server uses another port. Chromium maps wstd.dev and its project subdomains to loopback for local Builder URLs, so host-file changes are unnecessary.

The development backend uses E2E_DB_BOOTSTRAP=if-empty and keeps Docker containers running. The Playwright setup project still resets test data before each invocation.

Test organization

Put workflows in e2e/tests/*.e2e.ts and import test from e2e/test.ts.
Use Playwright's page fixture for the workflow under test. Playwright gives each test an isolated context and closes it automatically.
Use Playwright's context fixture when setup should share authentication with the workflow page.
Use withBrowserContext(browser, callback) for isolated setup or setup that runs in beforeAll. If cookies must survive from beforeAll into the tests, create an explicit context with newBrowserContext(browser) and close it in afterAll.
Use newIsolatedPage(browser) for identities that must not share cookies.
Keep tests in a file independent even though Playwright runs them in declaration order by default.

CI discovers isolated Playwright shards from filename tags. Each shard uses four workers, and files are assigned by measured CI duration to keep jobs near three to four minutes:

pages-actions.[shard-2].[shard-5].[shard-6].e2e.ts

Every E2E filename must contain at least one unique shard tag. CI derives its matrix from those tags, so adding or removing a shard does not require editing the workflow. Files with multiple tags are partitioned across those shards. All files selected by a shard must use the same set of tags. Rebalance by changing filename tags when measured job durations drift. Suites may opt into parallel mode only when every worker creates uniquely named setup data and the tests do not depend on each other's mutations.

Backend organization

Development, E2E, schema-cache, and Supabase compatibility commands share the backend definition in apps/builder/backend. compose.yaml defines the common services, and compose.<mode>.yaml defines only the differences for a named mode. Backend commands resolve these files by convention; callers do not pass Compose filenames.

The schema-cache fingerprint discovers migration sources and non-test backend files automatically. Adding a database image, initialization script, Compose override, or migration therefore invalidates the cache without updating a filename registry.

Failures

Failed shard jobs upload the Playwright HTML report, traces, screenshots, videos, and backend service logs.

When you need a completely clean backend locally:

E2E_SKIP_CLEANUP=false pnpm e2e:builder

4.3 KiB Raw Permalink Blame History