mirror of
https://github.com/webstudio-is/webstudio.git
synced 2026-11-10 06:24:35 +09:00
Oleg Isonen
8718feb465
## Summary Unifies frontend testing around three intentional boundaries: - Vitest Node projects for pure and server tests - Vitest Browser Mode with Playwright/Chromium for DOM and component tests - Playwright Test for Builder end-to-end workflows The Builder no longer uses jsdom. Ordinary E2E workflows use an automatically managed isolated `page` fixture; explicit contexts remain only for simultaneous identities and suite-scoped setup. ## Technical choices - Split Builder tests into one Node project and independent browser projects so each Chromium/Vite process has a bounded workload. - Added shared Browser Mode configuration for the affected SDK React packages. - Preserve the Builder's production top-window bridge ownership; Browser Mode uses explicit test-only installation seams. - Centralized Playwright context options and kept action/navigation timeouts bounded. - Validate Playwright filters before Docker, migrations, browser installation, or Builder builds start. - Preserve the first failed E2E attempt's report, traces, screenshots, and videos when CI retries a shard. - Split static checks, typechecks, CLI tests, core package tests, browser package tests, and Builder projects into parallel CI jobs with five-minute hard ceilings. - Distribute Builder E2E tests across ten isolated shards with two workers; parallel suites create unique per-worker setup data. - Reuse color contrast DOM/canvas samples so the exhaustive theme sweep takes about four seconds instead of timing out at 30 seconds. - The removed cost is jsdom behavior drift, oversized serial check jobs, and unnecessary E2E startup for empty selections. Real browser startup and E2E environment setup remain. ## Todo - [x] Split Builder pure/server tests from browser/component tests. - [x] Migrate Builder DOM tests from jsdom to Vitest Browser Mode. - [x] Migrate affected SDK React DOM tests to the shared Browser Mode setup. - [x] Remove jsdom annotations, dependencies, and compatibility setup. - [x] Replace ordinary manual E2E contexts with the isolated `page` fixture. - [x] Keep explicit contexts only for concurrent identities and `beforeAll` setup. - [x] Preserve setup authentication isolation from workflow pages. - [x] Add bounded Playwright action and navigation timeouts. - [x] Preserve first-attempt E2E failure evidence across CI retry. - [x] Reject empty E2E selections before expensive environment setup. - [x] Parallelize repository checks into bounded CI shards. - [x] Isolate parallel E2E setup data across workers. - [x] Remove the color contrast sweep bottleneck. - [x] Update E2E contributor documentation. - [x] Review product documentation impact; no user-facing product docs are affected. - [x] Run the full repository tests and typechecks. - [x] Run formatting, lint, package-boundary, generated-file, and fixture checks. - [x] Complete final correctness and simplification review. ## Verification - `pnpm --filter @webstudio-is/builder test` - `pnpm -r typecheck` - `pnpm lint` - `pnpm check:package-boundaries` - `pnpm check:generated-api` - `pnpm check:generated-docs` - `pnpm fixtures` - Design-system: 222 tests pass locally. - Builder: all seven Vitest projects pass locally. - The formerly slow E2E shard passes locally in 1m55s. - Full database-backed Builder E2E execution passes in CI. - All CI jobs pass; the slowest completed in 4m58s. - No fixture inputs/outputs or generated files changed. Ref #6270
50 lines
1.6 KiB
TypeScript
50 lines
1.6 KiB
TypeScript
import { expect, test } from "vitest";
|
|
import {
|
|
getE2eFileShards,
|
|
getE2eShardMatrix,
|
|
getE2eTestMatch,
|
|
} from "./test-shards";
|
|
|
|
test("discovers the shard matrix from e2e filenames", () => {
|
|
expect(
|
|
getE2eShardMatrix([
|
|
"second.[shard-2].[shard-5].e2e.ts",
|
|
"first.[shard-1].e2e.ts",
|
|
"another.[shard-2].[shard-5].e2e.ts",
|
|
"helper.ts",
|
|
])
|
|
).toEqual([
|
|
{ shard: "shard-1", partition: "" },
|
|
{ shard: "shard-2", partition: "1/2" },
|
|
{ shard: "shard-5", partition: "2/2" },
|
|
]);
|
|
});
|
|
|
|
test("requires shard tags and consistent multi-shard groups", () => {
|
|
expect(() => getE2eFileShards("unassigned.e2e.ts")).toThrow(
|
|
"Every e2e file must have a shard tag"
|
|
);
|
|
expect(() =>
|
|
getE2eFileShards("duplicated.[shard-2].[shard-2].e2e.ts")
|
|
).toThrow("Every shard tag in an e2e filename must be unique");
|
|
expect(() =>
|
|
getE2eShardMatrix([
|
|
"first.[shard-2].[shard-5].e2e.ts",
|
|
"second.[shard-2].[shard-6].e2e.ts",
|
|
])
|
|
).toThrow("Every file selected by shard-2 must use the same shard tags");
|
|
});
|
|
|
|
test("matches all e2e files locally and only the selected shard in CI", () => {
|
|
const allTests = getE2eTestMatch(undefined);
|
|
expect(allTests.test("tests/pages.[shard-1].e2e.ts")).toBe(true);
|
|
|
|
const shardTests = getE2eTestMatch("shard-2");
|
|
expect(shardTests.test("tests/pages.[shard-2].e2e.ts")).toBe(true);
|
|
expect(
|
|
shardTests.test("tests/pages.[shard-2].[shard-5].[shard-6].e2e.ts")
|
|
).toBe(true);
|
|
expect(shardTests.test("tests/pages.[shard-1].e2e.ts")).toBe(false);
|
|
expect(() => getE2eTestMatch("../../other")).toThrow("Invalid e2e shard");
|
|
});
|