admin/webstudio
1
0 Fork 0
mirror of https://github.com/webstudio-is/webstudio.git synced 2026-10-30 00:27:43 +09:00
Code Issues Packages Projects Releases Wiki Activity
6,844 Commits 122 Branches 310 Tags
docs
Commit Graph
17 Commits
Author SHA1 Message Date
Oleg Isonen 55920c57c4 fix: Show all Markdown and Assets query diagnostics (#6308) 
Ref #6305

## Outcome

Show actionable Markdown, MDX, and Assets query problems in the editor,
query Diagnostics, API, and MCP.

The current implementation substantially improves source diagnostics,
query error aggregation, query scoping, exact paths, MCP feedback, and
loading states. The exhaustive audit below also records what is not
complete yet.

## Completion rule

An error state is complete only when all applicable checks pass:

- [x] The editor uses the production validator and shows every issue for
the open file.
- [x] Query execution uses the same validation and returns every issue
for every file that matches the query, not only the returned page.
- [ ] MCP can reproduce the state through a real tool call and returns
the same code, severity, message, file, and location.
- [x] Diagnostics shows fatal errors and nonfatal warnings in the
correct place.
- [x] Exact file is always present; line and column are present whenever
the parser can determine them.
- [x] Multiple issues in the same file, multiple files, and multiple
validation phases are returned together.
- [ ] A regression test proves failure without the behavior and success
with it.

## Cross-surface contract

### Required validator parity

- [x] Editor, query execution, and MCP file operations invoke the same
exported production file-validation pipeline for byte decoding,
source/frontmatter, Markdown/MDX, Content Block semantics, and
references. Copied schemas, adapter-only checks, or reimplementations do
not count as parity.
- [x] Query builder, query execution, `validate-asset-query`, and
`preview-asset-query` invoke the same exported structural and
catalog-aware semantic query validator with the same project snapshot
and context.
- [x] Every surface preserves the same complete, deterministically
ordered diagnostic array: severity, code, message, phase/scope, asset
ID, file path, line/column/range, query path, and
fatal-versus-warning/result behavior. UI, API, and MCP serializers must
not drop or remap fields.
- [ ] A table-driven contract suite calls the real editor, query, API,
and MCP entry points—including MCP upload, update, download, validate,
and preview—and compares full normalized arrays for multiple errors,
multiple warnings, multiple files, and pre-pagination matches.

- [x] Markdown and MDX source syntax/frontmatter use
validateTextAssetSource in editor, query compilation, and MCP file
tools.
- [x] Query request execution and MCP local parsing use
assetQueryRequest.
- [x] Fatal query setup errors retain all detected semantic errors and
accompanying warnings.
- [x] Nonfatal query warnings retain a stable code and exact query path.
- [x] Source diagnostics retain severity, phase, code, asset ID, path,
and available source position.
- [x] Query Diagnostics excludes diagnostics produced only by unrelated
published queries.
- [x] Query, Preview, and Diagnostics tabs show loading states.
- [x] Authenticated diagnostics can show a sanitized internal message;
public/pre-auth errors remain redacted.
- [x] Share byte-level UTF-8 validation instead of decoding separately
in query and MCP.
- [x] Make the query editor and MCP call one literal shared
query-validation function before execution.
- [x] Share cross-file reference validation across editor, query, and
MCP file operations.
- [x] Share contextual Content Block/template validation across all
applicable surfaces.
- [x] Preserve an invalid stored query in the editor and explain it
instead of replacing it with defaults.
- [x] Define and enforce whether “matches the query” includes every
pre-pagination match; current source validation follows
returned/paginated IDs.
- [x] Continue through source, reference, and runtime phases where safe
so one phase does not hide later issues.
- [x] Map every user-triggerable graph/storage/runtime failure to a
structured public error instead of INTERNAL_ERROR.
- [ ] Label candidate-file warnings separately from returned-file
diagnostics.

## File transport and encoding

- [x] Empty Markdown and MDX are accepted.
- [x] UTF-8 BOM and CRLF are accepted.
- [x] Whole Markdown/MDX file over 1 MiB is detected.
- [x] Missing or unavailable selected content is reported.
- [x] Invalid UTF-8 frontmatter returns the same structured diagnostic
in query and MCP.
- [x] Invalid UTF-8 body returns the same structured diagnostic in query
and MCP.
- [x] MCP download must use fatal UTF-8 decoding rather than replacement
characters.
- [x] Validate bytes before committing an upload, or return the
committed Asset ID and explicit cleanup state when post-commit feedback
fails.
- [x] Reject filename/format mismatches instead of allowing either hint
to bypass Markdown/MDX validation.
- [x] Storage status, missing body, invalid metadata, revision mismatch,
format mismatch, short read, and excess read each return structured file
context.
- [x] Content changing during validation has a clear retryable conflict
diagnostic.

## Frontmatter

- [x] No frontmatter.
- [x] Empty frontmatter.
- [x] Opening delimiter without a closing delimiter.
- [x] Frontmatter over 64 KiB.
- [x] Invalid YAML grammar.
- [x] Multiple YAML parser errors.
- [x] Duplicate keys.
- [x] YAML aliases.
- [x] Unsupported YAML tags.
- [x] Scalar or array root instead of an object.
- [x] Non-JSON values, NaN, Infinity, and unsafe integers.
- [x] Nesting deeper than 8.
- [x] More than 256 fields/array entries.
- [x] One string over 16 KiB.
- [x] Normalized indexed properties over 64 KiB.
- [x] Frontmatter warnings remain nonfatal and visible.
- [x] Collect all post-parse normalization/limit errors; this phase
currently stops at the first.
- [ ] Prove every state above through real MCP upload, update, download,
query, and Diagnostics paths.

## Markdown body

- [x] Treat ordinary CommonMark/GFM as permissive; malformed-looking
Markdown and raw HTML are not automatically syntax errors.
- [x] Preserve and validate the body when frontmatter is invalid.
- [x] Detect the 1 MiB body/file limit.
- [x] Handle body extraction and bounded excerpts.
- [x] Return a shared structured invalid-UTF-8 body error from every
byte-reading surface.
- [ ] Distinguish author errors from internal
excerpt-limit/configuration errors.
- [ ] Prove body-size and decoding states through real MCP and
Diagnostics.

## MDX grammar and safety

- [x] Malformed MDX/JSX grammar.
- [x] Imports.
- [x] Exports.
- [x] Executable flow expressions.
- [x] Executable text expressions.
- [x] JSX expression attributes.
- [x] JSX spread attributes.
- [x] Arbitrary components.
- [x] HTML-looking JSX outside ws.element.
- [x] Unsupported/raw MDX node types.
- [x] Duplicate attributes and normalized alias collisions.
- [x] Event-handler attributes.
- [x] srcdoc, srcset, and dangerouslySetInnerHTML.
- [x] Unsafe URL protocols in link/source/action-like props.
- [x] Forbidden tags: base, link, meta, noscript, script, style,
template, title.
- [x] Conflicting, empty, or non-static ws:name and ws:tag.
- [x] More than 20,000 nodes.
- [x] Nesting deeper than 100.
- [x] More than 4,000 props.
- [x] Multiple recoverable unsafe-mdx warnings are collected.
- [ ] Collect multiple fatal grammar/structural errors when parser
recovery can do so; unrecoverable parsing currently stops at the first
fatal error.
- [x] Preserve sibling expression warnings when unsupported JSX is also
present; {first()}, <Card />, {second()} currently reports only the Card
warning.
- [ ] Prove every state above through real MCP and Diagnostics, not only
validator unit tests.

## Content Block semantics

- [x] Unresolved template warning exists.
- [x] Unknown template prop warning exists.
- [x] Incompatible template prop warning exists.
- [x] Design-only template prop warning exists.
- [x] Stale template prop warning exists.
- [x] Invalid materialized HTML content model error exists.
- [x] Run these checks for an MDX file even when no active editor
Content Block root has materialized it.
- [x] Run applicable semantic checks in query Diagnostics.
- [x] Run applicable semantic checks for MCP upload as well as
contextual update/download.
- [x] Return all Content Block frontmatter diagnostics; one path still
throws only diagnostics[0].
- [ ] Add real MCP and UI coverage for every semantic state.

## Cross-file references

- [x] Non-string $ref marker.
- [x] Invalid reference URL.
- [x] Invalid fragment encoding.
- [x] Invalid fragment.
- [x] Invalid JSON Pointer.
- [x] Missing referenced document.
- [x] Missing referenced JSON path.
- [x] Missing referenced local Asset warning.
- [x] Unsupported target representation.
- [x] Reference cycle.
- [x] Duplicate document ID or URL.
- [x] Missing graph source or target.
- [x] Root not found.
- [x] Duplicate/unused/unresolved reference.
- [x] Invalid resolved value.
- [x] Content identity conflict.
- [x] Target revision mismatch.
- [x] Representation selector required/invalid.
- [ ] Diagnose {$ref: value, extraKey: value} as an invalid marker or
explicitly document it as ordinary data.
- [x] Collect all bad references and targets instead of stopping at the
first graph failure.
- [x] Validate references from the editor and MCP file tools, not only
query compilation.
- [x] Map graph compilation/resolution, Markdown-document, and
JSON-document errors to exact structured Diagnostics rows.
- [x] Prevent an unrelated file's graph error or duplicate canonical URL
from blocking an exact-file query.
- [ ] Prove missing targets, cycles, bad representations, and bad JSON
paths through real MCP.

## Query request and structure

- [x] Invalid JSON, invalid request UTF-8, empty/non-object request, and
request over 512 KiB.
- [x] Missing query and unknown request/query keys.
- [x] Invalid result mode.
- [x] Malformed where/all/any/filter shape.
- [x] More than 32 filters or children.
- [x] Filter nesting deeper than 8.
- [x] Empty field path or segment.
- [x] Field path deeper than 9.
- [x] Bare properties path.
- [x] Unknown or nested standard field.
- [x] Missing or unknown operator.
- [x] Non-JSON operator value.
- [x] Invalid in value or more than 1,000 in values.
- [x] Non-boolean exists/isEmpty value.
- [x] More than 8 sort entries.
- [x] Invalid sort field or direction.
- [x] first/last without a sort.
- [x] Invalid output mode or includeMetadata.
- [x] Duplicate output fields or more than 256 fields.
- [x] No selected output.
- [x] Negative, fractional, or over-1,000 limit.
- [x] Negative, fractional, or over-1,000 offset.
- [x] Invalid content mode.
- [x] Invalid full/markdown-body-ref maxBytes outside 1..1 MiB.
- [x] Invalid range offset/length or range over 256 KiB.
- [x] Empty or over-255-character index revision.
- [x] Multiple simultaneous schema issues retain exact query paths.
- [x] Make content option objects strict; unknown keys are currently
stripped.
- [x] Show an invalid persisted resource body rather than silently
loading the default query.
- [x] Aggregate unknown-key errors with every other structural issue;
MCP currently stops at the first unsupported key.
- [x] Add field-value type validation, for example reject size gt
"large" and id contains {}.
- [x] Warn about unobserved dynamic fields selected only as output.
- [ ] Prove the full structural matrix through the MCP server/API end to
end; the current broad matrix tests the local adapter.

## Query semantic states

- [x] INCOMPATIBLE_OPERATOR is fatal for standard fields.
- [x] UNOBSERVED_FIELD is a nonfatal warning.
- [x] INCOMPATIBLE_OBSERVED_TYPES is a nonfatal warning.
- [x] STRUCTURED_SORT_ORDER is a nonfatal warning.
- [x] Every incompatible standard-field operator is collected.
- [x] Fatal semantic errors retain nonfatal warnings.
- [x] Successful warnings appear with exact query paths.
- [ ] Reproduce every catalog-dependent semantic state through real MCP.
- [x] Test repeated warnings at distinct query paths without accidental
deduplication.

## Query execution and hydration

- [x] Stale index revision.
- [x] result one matching multiple files, including matchedCount.
- [x] Candidate document limit of 1,000.
- [x] Result payload limit of 16 MiB.
- [x] Hydrated file count limit of 20.
- [x] Per-file hydration limit of 1 MiB.
- [x] Total hydration limit of 2 MiB.
- [x] Range limit of 256 KiB.
- [x] Binary content requested as text.
- [x] Invalid UTF-8 during hydration.
- [x] Missing, stale, or conflicting content identity.
- [x] Canonical metadata/byte-count mismatch.
- [x] Storage returning more bytes than requested.
- [x] Missing content reader.
- [x] Selected content not returned.
- [x] Runtime Asset URL unavailable.
- [x] Document graph document-count/per-file/total limits.
- [x] Classify candidate-document and result-byte limits as
CONTENT_LIMIT_EXCEEDED instead of INVALID_REQUEST.
- [x] Give unavailable URL, missing reader/content, storage, selection,
and assembly failures stable codes and exact asset/file details instead
of INTERNAL_ERROR.
- [x] Never silently turn an exact-file unavailable-content failure into
an empty successful result.
- [x] Add direct regression tests for duplicate identity, excess bytes,
and canonical byte mismatch.
- [ ] Reproduce every runtime/hydration state through real MCP.

## Scoping and aggregation

- [x] Errors and warnings from multiple loaded result files are returned
together.
- [x] Nonfatal errors do not suppress successful query results.
- [x] Unrelated published-query diagnostics do not replace current-query
diagnostics.
- [x] File path and available line/column are shown.
- [x] Exact nested query paths are shown.
- [x] Validate and report every pre-pagination matching file if that is
the product definition.
- [ ] Explain candidate metadata warnings when invalid metadata prevents
deciding whether a file matches.
- [ ] Stop including METADATA_PREPARATION_FAILED globally unless the UI
labels its candidate scope.
- [x] Aggregate reference errors instead of stopping at the first.
- [x] Aggregate across validation phases where safe.
- [x] Add a matrix for same-file multiple errors, multiple files,
outside-page errors, filter-only candidate files, and unrelated queries.

## Transport, authorization, cancellation, and internal failures

- [x] UNAUTHORIZED and FORBIDDEN.
- [x] REQUEST_CANCELLED in query HTTP paths.
- [x] Sanitized authenticated internal message.
- [x] Generic unauthenticated/public internal message.
- [x] Preserve structured causes for document-load HTTP
status/body/metadata failures.
- [x] Preserve structured causes for object-storage read failures.
- [x] Preserve structured causes for graph selection/assembly failures.
- [ ] Test MCP/API cancellation directly.
- [ ] Reproduce authorization, cancellation, storage, and
sanitized-internal states through real MCP where safe.

## MCP reproduction

- [x] Real MCP reproduction: multiple Markdown YAML errors.
- [x] Real MCP reproduction: multiple simultaneous structural query
errors.
- [x] MCP update returns Markdown and MDX source diagnostics.
- [x] MCP upload returns Markdown and MDX source diagnostics.
- [x] MCP download returns Markdown and MDX source diagnostics.
- [x] Temporary remote fixtures are removed after reproduction.
- [ ] Real MCP reproduction for every frontmatter code.
- [ ] Real MCP reproduction for every MDX fatal/warning class.
- [ ] Real MCP reproduction for every Content Block semantic diagnostic.
- [ ] Real MCP reproduction for every reference/graph error.
- [ ] Real MCP reproduction for every semantic query warning/error.
- [ ] Real MCP reproduction for every hydration/runtime/limit error.
- [ ] Real MCP reproduction for auth, cancellation, storage, and
internal errors where safe.
- [ ] A durable table-driven test proves code/severity/message/path
parity between editor, query, Diagnostics, and MCP.
- [x] Preserve stable top-level error identity and structured details
through MCP instead of flattening known errors to MCP_TOOL_FAILED.
- [x] Prove failed post-commit MCP feedback reports whether a mutation
committed and returns enough identity for cleanup.

## Real MCP recheck and follow-up fixes (2026-09-03)

Ran the branch CLI against the PR preview backend. The first run exposed
that the saved CLI origin pointed to a different backend, so those
results were discarded. Every MCP result below came from the PR preview.
All temporary remote and local fixtures were removed afterward. The
failures found in that run were then fixed and covered by regression
tests.

- [x] Exact broken Markdown query returns FRONTMATTER_INVALID with
severity, filename, message, line, and column.
- [x] Exact MDX query returns unsafe-mdx with severity, filename,
message, line, and column.
- [x] A no-extension query returns warnings from both matching Markdown
and MDX files.
- [x] A Markdown-only query excludes the unrelated MDX warning.
- [x] Two fatal MDX files inside the returned page are both reported;
matching warnings remain attached.
- [x] Structural query validation returns three independent errors with
exact paths in one call.
- [x] Semantic query validation returns three fatal operator errors and
two warnings with exact paths in one call.
- [x] MCP upload, update, download, and exact-file query return multiple
source diagnostics for the same MDX source.
- [x] Query preview reports a missing Asset reference as ASSET_NOT_FOUND
with the source file and reference.
- [x] All temporary MCP fixtures were deleted and the saved CLI
connection was restored.
- [x] Fixed after recheck: validate every matching file before
pagination, including broken files outside the returned page.
- [x] Fixed after recheck: unrelated graph failures no longer block an
exact-file query.
- [x] Fixed after recheck: update, download, editor, and query paths use
shared reference validation.
- [x] Fixed after recheck: graph failures map to structured diagnostics
with the exact file.
- [x] Fixed after recheck: shared fatal UTF-8 validation runs before
upload writes and preserves structured error identity.
- [x] Fixed after recheck: MCP rejects filename/format mismatches.
- [x] Fixed after recheck: MDX validation keeps sibling and nested
warnings around unsupported components.
- [x] Fixed after recheck: MCP preserves stable top-level error codes
and structured details.
- [x] Fixed after recheck: query and MCP serializers preserve full
normalized diagnostic fields.
- [x] Fixed after recheck: unknown keys are aggregated with all other
structural query errors.
- [x] Fixed after recheck: shared query validation covers literal types,
output-only fields, and strict content options.
- [x] Fixed after recheck: unavailable selected content returns an
exact-file diagnostic instead of a silent empty result.

## Tests and delivery

- [x] Full Builder suite: 2,707 tests.
- [x] Content Engine suite: 445 tests.
- [x] Asset uploader suite: 211 tests.
- [x] Project runtime/MCP targeted suite: 128 tests.
- [x] CLI/MCP suite: 1,051 tests.
- [x] Relevant package typechecks, lint, generated docs, and diff
checks.
- [x] Documentation impact reviewed and generated Content Engine/MCP
reference updated.
- [x] Add the missing cross-surface error matrix described above.
- [ ] Demonstrate red/green behavior for every new matrix case.
- [ ] Run the complete agent evaluation suite after explicit approval.
- [ ] Re-run all checks after the remaining checklist is implemented.

## Compatibility and risk

Normal query result and expression-binding shapes remain unchanged.
Diagnostics add optional fields. Existing public/pre-auth error
redaction remains intact. The remaining unchecked items are broader
exhaustive real-MCP coverage, parser-recovery limits, candidate-warning
labeling, cancellation coverage, and agent evaluations.
 2026-09-04 16:00:28 +02:00
Oleg Isonen 11d76eb43d perf: Diagnose and reduce Assets resource latency (#6095) 
## Summary

Adds Builder-only resource performance metrics to grouped Diagnostics
tables, removes duplicate object-store reads from the Assets query path,
scopes compiled-artifact reuse to one repository request, and compiles
unpinned Builder previews from their concrete query values instead of
hydrating the complete saved dynamic deployment plan. Nothing is
precomputed or retained in a warm serverless process after the request
finishes.

Ref #6092

## Technical choices

- Keep `__performance__` as internal response metadata and strip it
before resource values enter expression-visible caches.
- Record Builder round-trip, server duration, response size, Assets
phases, document work, completed storage reads, and cumulative
compiler/document-graph storage-read durations.
- Present Diagnostics as collapsible sections ordered Database and
sizes, Timing, Query work, then detailed database/result inspectors.
Open only the first available section by default. Every metric keeps an
explanatory tooltip, with info icons revealed only when the label cell
is hovered or keyboard-focused.
- Add a copy action to every section header. It copies that section's
raw structured data as formatted JSON without opening a collapsed
section.
- Keep Index preparation as an inclusive parent duration and explain
that relationship in Diagnostics.
- Compile ordinary unpinned Builder preview requests from a union of
their concrete query values. A detail route therefore hydrates only
documents that can satisfy that request.
- Continue using the saved artifact-producing plan for revision-pinned
requests.
- Keep publication and prepublication preparation unchanged.
- Load only `props`, `dataSources`, and `resources` when constructing
the single-query Builder preview plan, matching the already-narrow batch
path instead of downloading and parsing the complete dev Build row.
- Build the deployment-wide saved plan only when detailed Diagnostics
requests its Published database statistics. Report that work as the
separate inclusive `Published diagnostics` timing and exclude its child
phases and fetch counters from the normal query measurements.
- Reuse successfully read content bytes across compiler preparation,
document graph construction, and runtime resolution within one query or
batch. Graph-fetched bytes are size-validated before entering the
revision-keyed request map, and graph additions stop when the request
cache reaches the 2 MiB hydrated-content limit.
- Before repairing canonical metadata, load candidate metadata once and
prove its revision, base document, and prepared metadata tier match the
open source snapshot. Skip synchronization when it is already current;
retain recovery and repair for stale, missing, insufficient-tier, or
inconsistent rows.
- Key request-local bytes by content reference and revision so
source-change retries cannot reuse stale data.
- Instantiate compiled-artifact reuse per `PostgresAssetRepository`;
repeated preparations inside one request can reuse or coalesce work, but
later requests and other serverless instances cannot.
- Remove the module-level 32-entry compiled-artifact cache introduced in
#5946.
- Introduce no precomputed, persisted, process-local, or cross-request
content cache.

## Findings and measurements

The supplied production post resource was bimodal before removing the
process cache:

- Warm process: approximately 1.0–1.5 seconds.
- Cold process: approximately 5.7–6.7 seconds.
- Ordinary response: approximately 4.6 kB.

The earlier development capture was a process-cache hit: 542.6 ms server
time and 932.5 ms Builder round trip. A later hit measured 491.5 ms
server time and 739.1 ms round trip. Those results demonstrated the old
module cache's nondeterministic serverless behavior rather than
cold-path performance.

After request-scoped reuse replaced the process cache, the supplied cold
capture measured 3,848.5 ms server time. Index preparation was 3,259.7
ms. The saved dynamic plan caused 25 compiler content fetches and 301 kB
of compiler input even though the concrete post query resolved only two
documents. The published database contained 28 documents and used 45.4
kB bounded / 512 kB unbounded.

Tracing compilation also exposed duplicate request-time I/O: selected
Markdown was fetched for compiler input, document graph construction,
and runtime document resolution. Request-local byte reuse changes that
representative path from three object-store reads to one.

The remaining dominant issue was therefore query breadth, not response
serialization or runtime result resolution. The Builder was compiling
the deployment-wide dynamic plan to answer a concrete post request. The
new preview path compiles the concrete values; regression coverage
demonstrates that a saved two-post dynamic plan reads only `post-a` when
previewing `post-a`.

The next cold capture confirmed that change: Builder round trip fell
from 4,391.4 ms to 2,245.2 ms, server duration from 3,848.5 ms to
1,601.8 ms, and index preparation from 3,259.7 ms to 1,044.9 ms.
Compiler content fetches fell from 25 to 1 and document-graph content
fetches from 3 to 1. Canonical metadata remained 239.2 ms because an
unchanged request still performed a repair synchronization round and
then loaded the same stored metadata again. The new fast path reduces
that normal case to one recovery-safe candidate metadata read.

A subsequent capture confirmed the canonical metadata fast path: server
duration fell from 1,601.8 ms to 1,401.0 ms, index preparation from
1,044.9 ms to 684.8 ms, and canonical metadata from 239.2 ms to 74.7 ms.
The post query remained narrow at one compiler fetch and one
document-graph fetch. Builder round trip rose from 2,245.2 ms to 2,411.1
ms while the server improved, so that difference is outside the measured
server work and is consistent with
browser/network/development-environment variance.

At that point `buildPlan` was the largest remaining isolated phase at
242.8 ms. Single-query preview was still loading every dev Build column
and parsing pages, styles, instances, deployment data, and other fields
that are not inputs to the Assets compilation plan. The narrow loader
now requests and parses only the three content-engine fields used by
plan construction.

The next capture confirmed that optimization: build plan fell from 242.8
ms to 99.0 ms, server duration from 1,401.0 ms to 1,056.8 ms, and the
concrete post query remained at one compiler fetch and one
document-graph fetch. Index preparation stayed effectively flat at 679.7
ms, making compiler entries (142.3 ms), source snapshot (119.2 ms),
source validation (119.1 ms), and document graph (114.0 ms) the next
candidates. The new child timings separate cumulative storage waiting
from the remaining compiler/document-graph work before choosing another
behavioral optimization.

The storage timing capture then showed that I/O dominated the remaining
compiler and graph phases: compiler storage was 114.6 ms of 125.9 ms,
while document-graph storage was 114.0 ms of 147.6 ms. The graph path
consulted compiler-cached bytes but did not retain newly fetched graph
documents, so runtime reference resolution fetched the same two-document
chain again. Retaining validated graph bytes in the same request reduces
the regression fixture from four physical object-store reads to two
without changing its resolved result.

Final three-run measurements after same-request graph-byte reuse were
stable: median Builder round trip was 926.2 ms, median server duration
was 681.9 ms, median index preparation was 407.9 ms, and median document
resolution was 3.7 ms. Compared with the uncached 3,848.5 ms server /
3,259.7 ms index baseline, that is approximately 82% less server time
and 87% less index-preparation time for this post query.

A final DRY/correctness review centralized request-local byte-cache
keying and its 2 MiB accounting, removed repeated Diagnostics size rows
and SDK metadata assembly, typed phase keys, and corrected response-size
measurement so internal performance metadata is excluded as the UI
promises.

The module cache was also only bounded for completed entries. Many
simultaneous unique-project compilations in one process could exceed 32
pending entries until they settled. Scoping reuse to one repository
request removes cross-user retention and tenant cache competition while
preserving duplicate-work elimination inside the request.

## Todo

- [x] Add generic resource timing and response-size metrics.
- [x] Add Assets phase, document graph, source validation, storage-read
count/byte metrics, and compiler/document-graph storage-read durations.
- [x] Render ordered collapsible Diagnostics sections with only the
first open by default, per-metric explanations, interaction-only info
icons, and per-section JSON copy actions.
- [x] Reuse compiler and document-graph content bytes through runtime
resolution within the current request, without precomputation.
- [x] Remove cross-request process caching and retain request-scoped
compilation reuse.
- [x] Compile unpinned previews from concrete query values.
- [x] Separate deployment-wide diagnostic preparation from normal query
phase counters.
- [x] Skip redundant canonical metadata synchronization when stored
candidates already match the source snapshot and requested tier.
- [x] Narrow single-query Build reads to the three fields required by
Assets plan construction.
- [x] Expose cumulative compiler and document-graph storage-read timing
as explained child rows.
- [x] Add regression coverage, including red/green verification.
- [x] Review the final diff for correctness, duplication, and
unnecessary complexity.
- [x] Verify concrete-query compilation against the supplied post
resource: compiler fetches fell from 25 to 1.
- [x] Remeasure the canonical metadata fast path: canonical metadata
fell from 239.2 ms to 74.7 ms and index preparation from 1,044.9 ms to
684.8 ms.
- [x] Review documentation impact: no authoritative `docs/` page covers
this internal Diagnostics tooling, so no documentation update is
required.
- [x] Run focused tests, affected package typechecks, lint, formatting,
and the fixture pipeline.

## Verification

- Content engine tests: 23 passed.
- Asset uploader tests: 195 passed, including repository, canonical
synchronization, request-cache, and storage-timing coverage.
- Builder focused Diagnostics tests: 10 passed in the latest
storage-timing pass; 15 passed in the preceding final pass and 35 across
the earlier instrumentation scope. The collapsible view test verifies
section order, default state, and deferred content interaction.
- Project tests: 84 passed, including 5 focused preview tests. The
focused regression failed against the full-Build loader and passed after
switching the single-query path to the narrow content-engine loader.
- SDK resource loader tests: 24 passed.
- Current canonical metadata regressions failed under the old
always-synchronize expectations and pass with one metadata load and no
synchronization; stale metadata still exercises and passes the repair
fallback.
- The Timing clipboard test failed with an empty payload and passed with
the complete raw duration/phase JSON; copying a collapsed section is
also verified not to open it.
- The collapsible default-state test failed when Timing was temporarily
opened alongside Database and sizes, then passed after restoring the
first-section-only behavior.
- The concrete-preview regression failed before the optimization by
reading both `post-a` and `post-b`, then passed by reading only
`post-a`.
- A repository-reuse regression failed without the request cache and
passed when reuse became the repository default.
- The content-read regression failed before the original compiler reuse
optimization with three storage calls and passed afterward with one.
- The two-document graph regression failed with four physical storage
calls and passed with two after graph reads populated the request-local
runtime cache.
- The response-size regression failed at 127 bytes when internal
performance metadata was included and passed at the actual 66-byte
payload after separating that metadata.
- Builder, content engine, asset uploader, project, and SDK typechecks
passed.
- Oxfmt, oxlint, and `git diff --check` passed.
- The full fixture link/sync/build pipeline passed with no tracked
fixture changes after each runtime change.

## Known limitations

- Detailed Diagnostics intentionally prepares both the concrete query
database and the deployment-wide Published database. `Published
diagnostics` reports the latter separately; normal resource loads do not
perform it.
- Builder round-trip and server duration cover the complete resource
batch and therefore include Published diagnostics when the detailed view
requests it.
- Phase durations are aggregated when a batch contains multiple Assets
queries.
- Compiler and document-graph storage-read timings sum completed fetch
durations. They exclude request-local byte-cache hits and can exceed
wall-clock parent phases when reads run concurrently.
- Content-read counters report completed normal-query reads;
deployment-wide diagnostic reads are excluded, and failed storage reads
are represented by the enclosing phase timing and error.
- Request-local byte reuse retains selected content only until the
current query or batch finishes. Document-graph additions are capped by
the existing 2 MiB hydrated-content limit; content beyond the cap is
still streamed and can be fetched again at runtime.
- Source snapshot and final source validation remain separate reads. The
metadata fast path removes redundant repair work without weakening that
concurrent-change check.

## Measurement gate

PR deployments use the development backend. The supplied production
project returns 500 on both the branch deployment and
`main.development.webstudio.is`, while the production host returns 200.
Final phase timings for the exact project require its corresponding
development project/session or an explicitly authorized
production-backed deployment. An unrelated fixture will not be used to
claim the production improvement.
 2026-08-11 12:31:23 +02:00
Oleg Isonen c44ef705cf perf: Speed up Builder resource loading (#6011) 
## Problem

Assets-backed Builder pages can take 6–7 seconds to become usable even
when their Assets resources only request metadata. On the
production-derived Blog project, a single Assets query measured 8,703 ms
and an immediate repeat measured 8,110 ms. Four sibling Assets resources
repeated the same authorization, development-build load/parse,
compilation, canonical-asset preparation, runtime metadata load, and
document work.

## Architecture

The loader now treats remote resources holistically while preserving
provider boundaries:

- The client scheduler is resource-provider neutral. It versions logical
requests, frees capacity when obsolete requests are replaced, ignores
stale results, and aborts physical batches only when every member is
obsolete.
- Ordinary Assets requests in one server resource-loader call use a
deferred Assets provider and execute as one repository batch.
Independent HTTP, GraphQL, and other resources start immediately instead
of waiting for Assets preparation.
- Arbitrary remote responses are **not** coalesced or persistently
cached. Their auth headers, cookies, upstream freshness, and `Vary`
semantics remain isolated.
- The Assets provider authorizes and creates its API context once per
batch, then loads only `props`, `dataSources`, and `resources` from the
development Build.
- Queries covered by the saved Build compilation plan reuse that exact
plan/artifact, preserving published byte limits, truncation, and
pinned-revision behavior. Uncovered unpinned queries use a deterministic
literal union and safely fall back to their existing individual path if
shared preparation cannot preserve behavior.
- Request-scoped document loading uses npm's byte-bounded `lru-cache` so
sibling queries do not reread the same referenced author/content files.
- Sibling dynamic queries with the same filter, sort, output, and
content shape stitch into one document-graph resolution while keeping
filter values, pagination windows, results, and failures independent.
Aggregate document/byte-limit failures fall back to isolated execution.
- Provider-neutral batching lives in the SDK runtime, and project
asset-query preview orchestration lives in the shared project package so
Builder and MCP paths can reuse the same behavior.
- Cancellation flows from the client batch through the
SDK/server/build/query path, with checkpoints between expensive phases.
- Runtime asset metadata remains conservatively artifact-wide because
URL/size filters, structured references, markdown hydration,
materialized rows, and saved plans have different reachability rules. To
make 1,000-asset artifacts safe, PostgREST `.in(...)` lookups are
deduplicated and split into 100-ID chunks with bounded concurrency.

## Implementation checklist

- [x] Trace the production Blog resource path and identify repeated work
- [x] Copy the production Blog project into an ignored disposable local
project
- [x] Record production and local baselines
- [x] Add provider-neutral client scheduling, stale-result protection,
and cancellation
- [x] Batch ordinary Assets queries without coupling or caching
HTTP/GraphQL responses
- [x] Load only Build fields required to create the saved content
compilation plan
- [x] Preserve saved-plan, truncation, fallback, diagnostics, and
revision semantics
- [x] Share Assets artifacts, runtime metadata, and revision-keyed
document reads per request
- [x] Stitch compatible query shapes into one graph resolution with
isolated fallback
- [x] Extract reusable SDK batching and project preview orchestration
for MCP/runtime consumers
- [x] Bound large runtime asset metadata lookups for the 1,000-asset
project limit
- [x] Add regression coverage and demonstrate new tests failing before
their fixes
- [x] Run focused and broad tests, type checks, lint, format,
package-boundary, and diff checks
- [x] Review documentation impact; no authoritative `docs/` content
changes are required
- [x] Regenerate and verify the affected asset-query runtime fixture; no
production data is changed
- [x] Address all actionable PR review comments
- [x] Rerun the final parity-preserving implementation against the
copied Blog project

## Measurements

Production trace before the change:

- Updates Assets query: 8,703 ms
- Immediate identical repeat: 8,110 ms
- Query artifact: 9,344 bytes / 7 documents
- Saved source database: 42,086 bytes / 28 documents

Copied-project local baseline:

- Builder navigation until resource loading: 5,111 ms
- Exact four-resource batches: 1,448 / 797 / 1,377 / 774 / 3,018 ms

During implementation, before the final saved-plan parity tightening:

- Exact four-resource batch: 174 ms first run, then 51–80 ms
- Actual Builder resource-loader requests: 234–246 ms

Final parity-preserving copied-project rerun:

- Preflight: 10 ms
- Exact four-resource batch: 435 ms cold, then 119 / 97 / 90 / 79 ms
warm
- Every response returned HTTP 200 with the same 16,884-byte payload

The final cold batch is 70% faster than the best copied-project cold
baseline, and the fastest warm batch is 90% faster than the best warm
baseline. The production trace remains the authoritative production
baseline; these before/after comparisons use the same disposable local
project and captured request payload.

## Verification

Passed:

- Full package suites: Builder (212 files / 2,496 tests), asset-uploader
(22 / 193), content-engine (40 / 274), project (7 / 84), SDK (29 / 508),
and project-build (82 / 2,070)
- Final focused regressions: Builder preview/API/resource integration (3
files / 42 tests), SDK resource loader (23 tests), and document-graph
resolution (6 tests)
- Builder, asset-uploader, content-engine, project, SDK, and
project-build type checks
- Fixture regeneration is deterministic; the generated vendor exactly
matches the source runtime bundle
- Fixture typecheck and production build
- Changed-file Oxfmt, repository Oxlint, and package-boundary checks
- `git diff --check`
- `git diff --name-only HEAD --exit-code`

New behavioral tests were demonstrated to fail before implementation and
pass afterward, including Assets batch coordination, saved-plan wiring,
truncation and pinned revisions, shared document reads, cancellation
checkpoints, stale/replaced client requests, and large PostgREST ID
filters.

Agent evaluations were not run because they require separate explicit
approval.

## Compatibility and remaining work

The public resource result/error shape and on-demand diagnostics
contract stay compatible. Generic HTTP/GraphQL caching semantics are
unchanged. Compilation already in progress is not force-cancelled
because safely interrupting shared compilation requires
ownership/reference tracking; cancellation prevents later phases from
starting.

Closes #6003
 2026-08-04 18:39:41 +02:00
Oleg Isonen 5cefbc0320 perf: Speed up Builder resource loading (#5980) 
## Summary

Remove avoidable latency from Builder resource loading and make Assets
resources use the content database efficiently.

## Why

Builder currently applies a one-second debounce to every uncached
resource request. Assets queries also perform server-side preparation
work that should be reused or deferred, making pagination and route
changes noticeably slower than expected.

## Implementation plan

- [x] Dispatch the complete set of currently computable page resources
without the one-second debounce.
- [x] Preserve batching, deduplication, exact-request caching,
invalidation, and bounded queue draining.
- [x] Add regression tests for immediate dispatch, batching,
deduplication, invalidation, and queue draining.
- [x] Cache compiled Assets content databases by the relevant
project/build and asset revision.
- [x] Run pagination, filtering, sorting, and metadata selection against
the cached database without rebuilding it for each request.
- [x] Fetch only referenced document content that is required by the
query and keep graph reads parallel.
- [x] Avoid generating and returning full diagnostics artifacts for
ordinary canvas resource loads; load detailed diagnostics on demand.
- [x] Preserve distinct loading and confirmed-empty states for article
and author resources.
- [x] Add coverage for paginated overview and article navigation
behavior.
- [x] Deep-review the complete change and remove redundant plan
construction, duplicated JSON comparison, path-based diagnostics
inference, and synthetic canvas diagnostics.
- [x] Make resource and diagnostics invalidation race-safe, including
invalidation during in-flight requests.
- [x] Run formatting, tests, lint, type checking, package-boundary
checks, and affected fixtures.

## Verification

- `pnpm oxfmt --check`
- `pnpm lint`
- `pnpm check:package-boundaries`
- `pnpm --filter @webstudio-is/content-engine test` (262 tests)
- `pnpm --filter @webstudio-is/asset-uploader test` (184 tests)
- `pnpm --filter @webstudio-is/builder test` (2,483 tests)
- Typechecks for Content Engine, Project Build, Asset Uploader, and
Builder
- `pnpm fixtures` (passed; no fixture diff)

The new invalidation regression tests were verified to fail against the
previous behavior and pass with the fix. Agent evaluations were not run
because they require separate approval.

## Known limitations

None identified.
 2026-08-03 01:15:56 +02:00
Oleg Isonen 56a22fde8a feat: Add storage-backed document graph resolution (#5966) 
## Summary

Introduce a document graph architecture for Assets resources so
structured documents can reference one another while keeping full
document payloads on CDN-backed storage.

The content database will retain the searchable index and dependency
graph. At runtime, the server will select a root document, determine its
complete dependency set before performing I/O, fetch each unique
document concurrently, and assemble the final result server-side.

## Implementation todos

### Document graph foundation

- [x] Add normalized internal reference contracts for whole documents,
JSON paths, Markdown body, and Markdown frontmatter.
- [x] Add an immutable graph abstraction with deterministic ordering,
dependency traversal, revision validation, and cycle detection.
- [x] Add a storage-independent resolver with dependency closure,
bounded parallel loading, shared-content deduplication, representation
selection, assembly hooks, cancellation, and structured errors.
- [x] Add focused unit coverage for the isolated document-graph
abstraction.
- [x] Define and validate the external remote-document reference syntax.
- [x] Add a pure graph compiler that resolves canonical document URLs
into IDs and revisions and validates missing, duplicate, and cyclic
dependencies.

### Build-time graph

- [x] Add a storage-independent source-catalog compiler for bounded
parallel reference discovery and deterministic graph compilation.

- [x] Add a versioned deterministic document graph artifact with
canonical serialization and integrity verification.

- [x] Discover and normalize statically declared references while Assets
resources are processed.
- [x] Resolve document identities and revisions and report missing,
stale, or cyclic dependencies with document context.
- [x] Persist document nodes and dependency edges alongside searchable
resource metadata.
- [x] Include persisted graph data in artifact integrity and size
validation.

### Document representations

- [x] Add a standalone JSON document adapter for exact reference
markers, compiler occurrences, JSON path selection, and immutable
assembly.
- [x] Add a standalone Markdown document adapter for bounded decoding,
frontmatter references, immutable assembly, and source, body, and
frontmatter selection.

- [x] Parse fetched JSON and Markdown documents through format-specific
adapters.
- [x] Select complete documents, JSON fragments, Markdown body, and
Markdown frontmatter through the shared reference model.
- [x] Assemble nested and shared references into the final server-side
value.

### Runtime resolution

- [x] Enforce reachable-document, per-document byte, and aggregate-byte
limits before parsing and assembly.

- [x] Add a revision-keyed bounded source-cache wrapper with replayable
bytes, stale-entry fallback, and in-flight deduplication.

- [x] Add a storage-neutral loaded-source contract that rejects revision
mismatches before parsing.

- [x] Resolve graph-backed property paths before filtering, sorting,
pagination, and projection; skip graph loading for unrelated queries.
- [x] Add an injected HTTP/CDN document loader with revision metadata
validation.
- [x] Apply runtime payload limits, bounded concurrency, caching, and
structured failure mapping.
- [x] Ensure shared documents are fetched and parsed once per
resolution.

### Product integration

- [x] Integrate equivalent behavior in local preview, SSR, SSG, and
published runtime paths.
- [x] Preserve existing embedded Assets resource behavior without
requiring migration.
- [x] Add observability for graph loading, document fetching, caching,
and resolution failures.
- [x] Document the public reference and resolution behavior.

### Review hardening

- [x] Enforce graph, artifact, cache, and published-storage identity
consistency.
- [x] Compile only graph sources reachable from selected build entries.
- [x] Resolve graph-backed fields before query evaluation and avoid
unnecessary graph loading.
- [x] Consolidate duplicated graph schemas, utilities, and document
content-type detection.
- [x] Correct runtime documentation and use structural evaluation
validation.

### Verification

- [x] Keep a deterministic high-impact agent evaluation for
Markdown-to-JSON references across the dynamic blog overview and article
routes.
- [x] Add a separate discoverability evaluation that must learn the
reference workflow from MCP guidance without prompt-supplied syntax,
queries, bindings, or call order.
- [ ] Re-run the complete local agent evaluation suite after the
discoverability fixture change and compare every fixture with its
accepted baseline.

- [x] Add integration coverage for CDN-backed hydration, parallel
fetching, caching, and shared dependencies.
- [x] Add integration coverage for missing references, cycles, revision
changes, limits, and cancellation.
- [x] Verify equivalent behavior across preview, SSR, SSG, and published
deployments.
- [x] Verify that existing embedded Assets resources retain their
current behavior.

## Intended outcome

- Storage-backed documents do not require their complete payloads to be
embedded in the content database.
- The server knows the complete statically discoverable dependency set
before fetching document content.
- Shared dependencies are fetched once per resolution and independent
resources can be fetched in parallel.
- JSON fragments and Markdown body/frontmatter representations use the
same document-resolution system.
- Reference and payload revisions cannot be mixed silently.
- Existing Assets resources continue to work without migration.

## Verification results

- `@webstudio-is/content-engine`: 37 files / 244 tests passed; typecheck
passed.
- `@webstudio-is/asset-uploader`: 22 files / 182 tests passed; typecheck
passed.
- CLI: 57 files / 925 tests passed, including generated SSR and SSG
build/prerender coverage; typecheck passed.
- Repository-wide `pnpm oxfmt --check` passed across 2,757 files.
- Package-boundary validation passed.
- Previous `pnpm evaluations` run: all 4 original high-impact fixtures
passed. The complete 5-fixture rerun is pending explicit approval after
adding the discoverability fixture.
 2026-08-02 10:39:06 +02:00
Oleg Isonen 3ccb46cec7 fix: restore Marketplace settings and optimize Assets publishing (#5964) 
## Outcome

Restore the Marketplace section for projects without existing
marketplace metadata, make the complete review-submission workflow
available over MCP, clarify shared Assets database diagnostics, and
reduce published database size by materializing fully static Assets
queries.

## Implementation

### Marketplace and MCP

- Initialize the Marketplace form from a safe partial default instead of
rendering nothing when metadata is unconfigured.
- Keep review submission disabled until the complete metadata schema is
valid.
- Allow `get-marketplace-product` to return `null` and
`update-marketplace-product` to create the first marketplace product
through ProjectSession/MCP.
- Add a confirmed `submit-marketplace-product` server operation that
validates configured metadata before setting review status to `PENDING`.
- Document the MCP workflow: upload the thumbnail, update metadata,
publish production, then submit.
- Correct the Marketplace thumbnail upload example to use the current
MCP input field.

### Assets database

- Show query size and merged published-database size separately in
resource diagnostics.
- Clarify that only the merged database counts toward the publish limit.
- Allow the Assets query editor to scroll long query source.
- Materialize fully static metadata-only Assets queries at publish time.
- Store materialized results as compact field columns and rows keyed by
a deterministic normalized-query hash.
- Serve materialized rows through the existing Assets response contract
without candidate documents or a runtime field catalog.
- Keep route- or variable-dependent queries on the existing
document-backed runtime path.
- Exclude overview-only fields from dynamic detail documents so the
merged database does not duplicate them.
- Preserve byte-limit selection, truncation diagnostics, integrity
verification, caching, and backward compatibility with existing
artifacts.

### Release

- Keep the release lockfile stable when preparing the CLI package.
- Regenerate runtime contracts, server-only API metadata, and CLI
documentation where required.

## Todo

- [x] Restore the empty Marketplace settings state
- [x] Prevent empty review submissions
- [x] Support first-time Marketplace metadata creation over MCP
- [x] Add MCP review submission
- [x] Document the end-to-end MCP workflow
- [x] Correct query and merged-database diagnostics
- [x] Make long Assets query source scrollable
- [x] Materialize static overview queries
- [x] Preserve dynamic detail-query behavior
- [x] Add regression coverage with fail/pass proof
- [x] Run relevant package and integration checks

## Verification

- [x] Content engine tests: 158 passed
- [x] Asset uploader tests: 179 passed
- [x] SDK tests: 505 passed
- [x] CLI tests: 921 passed
- [x] CLI prebuild integration tests: 46 passed
- [x] Builder Assets resource and diagnostics tests: 14 passed
- [x] Project runtime and ProjectSession tests: 133 passed
- [x] Builder API router tests: 30 passed
- [x] Protocol operation tests: 18 passed
- [x] Content engine build and declaration generation
- [x] Content engine, asset uploader, SDK, Builder, and CLI typechecks
- [x] Repository lint and formatting checks
- [x] Generated API and documentation checks
- [x] New materialization tests demonstrated failing with the
implementation disabled and passing after restoration

## Notes

- The representative five-post overview fixture shrinks from 2,122 bytes
to 1,197 bytes while returning the same sorted and paginated result.
- Materialization is intentionally limited to static, metadata-only
field queries without runtime asset fields. Unsupported or
data-incompatible queries safely use the existing runtime database.
- The complete `pnpm evaluations` agent suite was not run because
repository policy requires separate explicit approval.
- No database schema or migration changes are required.
 2026-07-31 13:16:21 +02:00
Oleg Isonen efbc3098c4 feat: add queryable Assets resources (#5946) 
Closes #5919
Ref #5920

## Outcome

Extend the existing Assets system resource with optional queries for
Markdown, JSON, and standard file metadata. Unconfigured Assets
resources retain the existing fetch-all behavior and activate none of
the content-query infrastructure.

Files are the authoring model. A standalone content engine converts an
immutable snapshot of only the files required by configured queries into
one bounded, deterministic runtime database. Builder preview, SSR, and
SSG use the same query semantics without requiring a database service
for a published site.

## Final architecture

```text
Markdown / JSON files
        │
        ▼
ContentSource adapter
(FS or Webstudio Assets)
        │ immutable snapshot
        ▼
ContentCompilationPlan
(union of configured query requirements)
        │
        ▼
ContentCompiler
(parse, normalize, discover fields, enforce budget)
        │
        ▼
ContentArtifact
(bounded, deterministic, immutable JSON)
        │
        ▼
ContentDatabase
(filter, OR groups, sort, paginate, project, read content)
        │
        ├── Builder preview
        ├── SSR runtime
        └── SSG build only
```

The authoring/storage layer owns mutable files. The content engine
consumes immutable snapshots and produces immutable, disposable
artifacts.


Query authoring is a separate OpenAPI-driven path:

```text
Assets REST OpenAPI operation
+ standard request JSON Schema
+ dynamic field paths
        │
        ▼
Generic OpenAPI adapter
        │ resolved schemas, controls, paths, and transport mapping
        ▼
Generic query-builder core and React renderer
        │ arbitrary provider-defined value
        ▼
Assets REST request adapter
        │
        ▼
{ query: value }
```

The selected standard OpenAPI operation is the only source of query UI
semantics. Request parameters and JSON Schema describe the editable
data; the generic renderer derives controls and owns layout.

### Activation and compilation

- The compiler does not index every asset. It combines all reachable
configured Assets-resource queries into one `ContentCompilationPlan`.
- Static constraints such as folder, path, and file type select
candidate files. Dynamic values such as a route-bound slug cannot reduce
the build-time candidate set.
- The plan contains the union of fields used by filtering, sorting,
routing, projection, excerpts, and requested full content.
- Documents shared by multiple resources are stored once in one
deployment-wide runtime database.
- Markdown/frontmatter, JSON properties, excerpts, and full bodies are
prepared only when required.
- No configured content query means no parsing, artifact, or shipped
runtime dependency. Fresh generated apps need no content-runtime
cleanup; incremental generation removes stale integration if a
previously configured query is deleted.

### Runtime database limit

- `CONTENT_DATABASE_MAX_BYTES` limits the complete serialized runtime
database, not source files, images, or unrelated assets.
- The default is `500 * 1024` bytes and is presented as 500 KiB.
- Counted data includes indexed metadata, frontmatter/JSON properties,
excerpts, the field catalog, serialization overhead, and full Markdown
bodies only when requested.
- Prepublish validation builds and serializes the exact database that
will be attached to the immutable build, so it knows the unbounded size,
bounded size, included documents, omitted documents, and affected
resources.
- Overflow never stores partial documents. Candidates are ordered by
`createdAt` descending, canonical path ascending, and stable document ID
ascending. Missing dates follow dated documents. A document that does
not fit is skipped while smaller later documents are still considered.
- Exceeding the limit warns but does not block publishing.
- Builder preview can report exact bounded versus unbounded results when
dynamic expressions have concrete preview values. Prepublish warnings
remain conservative for future route values.

### Snapshot consistency

The source contract guarantees a consistent compilation, while adapters
provide the mechanism:

- Webstudio Assets captures asset metadata and immutable content
references.
- Filesystem sources capture file identity and verify files did not
change before accepting the artifact.
- A changed source retries once, then fails clearly instead of producing
a mixed-revision database.
- This requires no universal snapshot service, source-tree copy, or new
versioned-asset infrastructure.

### Package ownership

- `@webstudio-is/content-engine`: runtime artifact schema, query
execution, field catalog, compact runtime statistics.
- `@webstudio-is/content-engine/compiler`: source snapshots, compilation
plans, Markdown/JSON parsing, deterministic compilation, byte
accounting, and compiler diagnostics. This entry exists to keep
Remark/YAML/future MDX dependencies out of runtime bundles.
- `@webstudio-is/query-builder`: resolves standard OpenAPI operation
schemas into an internal provider-neutral editor model, validation,
mutations, and source serialization. It owns no Assets,
provider-authored UI contract, or fixed REST-query semantics.
- `@webstudio-is/query-builder/react`: container-neutral schema renderer
whose layout is renderer policy. It remains separate from the content
engine.
- SDK: Webstudio Assets/Resource domain models and mapping Webstudio
expressions into generic content requirements.
- Protocol: external Assets REST/OpenAPI transport contracts.
- Asset uploader/repository: asset and folder mutations, generic
metadata, immutable byte references, and storage access. Its hosted
adapter supplies Webstudio Assets snapshots and parsed-metadata cache
persistence to the content engine; parsing, artifact creation, and query
semantics remain engine-owned.
- Builder and generated-app integrations decide when to compile, cache,
embed, load, or discard a database.

### Runtime lifecycle

- Builder caches immutable `ContentDatabase` instances by source
revision plus compilation-plan hash and coalesces concurrent
compilation. Matching operations reuse the same instance; field
discovery uses a dedicated minimal all-properties plan that excludes
excerpts and bodies.
- SSR attaches the validated bounded database to the build and includes
the small runtime only when required.
- SSG uses the database only while generating pages and omits it and the
runtime from final output unless a future runtime-query feature requires
them.
- Local development compiles from the filesystem and uses identical
query semantics.
- Published pages query their local artifact; they do not call Builder
REST endpoints.
- Published artifacts need no external index store, invalidation,
garbage collection, SaaS content proxy, new bucket, or new environment
variable.

## Implemented foundation

These existing parts remain useful and should be moved behind the final
ownership boundaries rather than reimplemented.

- [x] Upgrade the existing Assets system resource in place and preserve
unconfigured fetch-all behavior.
- [x] Implement typed filtering, nested boolean groups, deterministic
sorting, pagination, projections, dynamic values, and content hydration.
- [x] Add Markdown frontmatter, excerpt, JSON-property, and
field-catalog extraction.
- [x] Add reusable query-control implementations and a separately
imported React UI as a foundation for the final OpenAPI-driven contract.
- [x] Add Assets query authoring and dynamic field discovery as existing
behavior to migrate into the Assets OpenAPI operation.
- [x] Add synchronized visual and editable expression authoring.
- [x] Unify Assets REST operations under `/rest/assets/*`.
- [x] Define authenticated collection, item, content, upload, folder,
query, field-catalog, and OpenAPI operations.
- [x] Support Builder cookie authentication and public API token
authentication.
- [x] Add Builder preview, MCP/CLI operations, SSR execution, SSG path
generation, and conditional activation.
- [x] Add revision-keyed parsed metadata persistence for the hosted
Builder.
- [x] Add focused query, API, authorization, Builder, CLI, SSR, SSG,
migration, and fixture coverage.

## Remaining implementation plan

### 1. Establish the content engine

- [x] Rename the generic `@webstudio-is/asset-resource` implementation
to `@webstudio-is/content-engine`.
- [x] Move existing behavior instead of creating a parallel
implementation.
- [x] Keep the primary entry runtime-only and add only the compiler
entry needed for bundle separation.
- [x] Remove the old package after all consumers migrate.
- [x] Ensure the engine has no Builder, SDK, asset-uploader, PostgreSQL,
R2, or deployment-target dependency.

### 2. Define source snapshots and compilation plans

- [x] Introduce `ContentSource.openSnapshot()` and a revision-bearing
`ContentSourceSnapshot`.
- [x] Include canonical path, stable document ID, content type,
immutable content reference, size, and optional `createdAt`.
- [x] Implement Webstudio Assets and filesystem snapshot guarantees
using their adapter-specific mechanisms.
- [x] Introduce `ContentCompilationPlan` for the union of reachable
configured resource requirements.
- [x] Separate statically selectable candidates from dynamic runtime
predicates.
- [x] Add mutation-during-compilation tests for replacement, deletion,
folder movement, and filesystem changes.

### 3. Separate storage from content processing

- [x] Keep upload, replacement, folders, byte storage, revisions, and
generic metadata in the Assets repository/uploader.
- [x] Move parsing, excerpts, field discovery, query execution, and
artifact creation into the content engine.
- [x] Remove content-index coordination from asset mutation business
logic.
- [x] Preserve original source `createdAt` across content replacement
and expose it as optional queryable metadata.
- [x] Keep hosted parsed-document persistence behind the content
compiler's optional cache interface.

### 4. Build deterministic compilation

- [x] Move the existing Markdown, frontmatter, JSON, canonicalization,
hashing, document-building, and tests into the compiler.
- [x] Keep frontmatter behind a dedicated module and continue using the
existing Remark/Micromark pipeline.
- [x] Parse only candidate files and only the tiers required by the
compilation plan.
- [x] Deduplicate documents and fields required by multiple resources.
- [x] Define a versioned, deterministic `ContentArtifact` with integrity
information.
- [x] Keep verbose parse and omission diagnostics outside the runtime
artifact.
- [x] Reject incompatible artifact versions cleanly.

### 5. Enforce the deployment-wide database budget

- [x] Replace the old asset-index limit with
`CONTENT_DATABASE_MAX_BYTES`, defaulting to 500 KiB.
- [x] Have applications read configuration and pass the byte limit into
the compiler.
- [x] Measure the exact serialized database, including selected bodies
and overhead.
- [x] Implement deterministic skip-and-continue packing of complete
documents.
- [x] Return compiler diagnostics with unbounded/bounded bytes,
included/omitted counts, omitted identities, largest contributors, and
affected resources.
- [x] Expose only compact artifact statistics at runtime through
`ContentDatabase.getStats()`.
- [x] Add exact boundary, deterministic ordering, deduplication, and
oversized-document tests.

### 6. Finalize the runtime database

- [x] Define `ContentDatabase` over an immutable artifact.
- [x] Preserve filtering, nested AND/OR groups, sorting, offset/limit,
projection, field references, and content retrieval.
- [x] Expose `getFieldCatalog()` and compact `getStats()`.
- [x] Keep Markdown/YAML/compiler dependencies out of the runtime entry.
- [x] Add execution guards and maximum-size runtime benchmarks.

### 7. Make query authoring OpenAPI-driven

The selected standard OpenAPI operation is the complete query-authoring
contract. The query builder resolves parameters and request-body JSON
Schema, derives its internal editor model, and applies renderer-owned
layout policy. It must not embed or fetch a second capabilities/UI
language through `x-webstudio-query` or another side channel.

- [x] Remove `x-webstudio-query` and all vendor query-definition
metadata from the Assets OpenAPI operation.
- [x] Resolve local OpenAPI references and derive the editable request
value, defaults, validation, labels, choices, arrays, objects, and
unions solely from standard operation parameters and request-body JSON
Schema.
- [x] Represent dynamic Assets field paths with standard JSON Schema
vocabulary in the `queryAssets` request schema.
- [x] Keep the normalized editor representation internal to
query-builder; do not expose another provider-authored capabilities
contract.
- [x] Remove provider-controlled layout metadata such as `group`; make
compact scalar packing and all other layout renderer policy.
- [x] Keep Webstudio expression binding as a consumer/editor concern
layered over schema values, not part of the OpenAPI contract.
- [x] Preserve the existing Assets filter, sort, output selection,
full-content/range selection, limit/offset, editable source, and
configured-field behavior.
- [x] Keep the Assets resource adapter responsible for serializing the
edited query into its request body.
- [x] Remove the obsolete Assets query-capabilities endpoint, URL, and
UI-side capability parsing.
- [x] Add a non-Assets OpenAPI proof with differently named parameters
and no vendor metadata.
- [x] Verify the generic packages contain no Assets imports, fixed
Assets property contract, or provider-authored layout model.
- [x] Update focused core, React, SDK, protocol/OpenAPI, Builder, and
source round-trip tests.

### 8. Manage ContentDatabase lifecycle in Builder

- [x] Cache databases by source revision plus normalized
compilation-plan hash.
- [x] Coalesce concurrent compilation requests.
- [x] Reuse cached database instances across field discovery,
autocomplete, query preview, and response preview when their normalized
compilation plans match.
- [x] Replace it when source revision or query requirements change.
- [x] Preserve the optional revision-keyed hosted parsed-document cache
without making it a source of truth.
- [x] Show exact preview warnings when concrete values reveal missing
bounded results.

### 9. Materialize and verify immutable builds

- [x] Compile and serialize the exact deployment-wide database during
prepublish validation.
- [x] Attach that validated bounded database to the immutable build
without rebuilding it from later asset state.
- [x] Show non-blocking prepublish warnings with database bytes,
omissions, and potentially affected resources.
- [x] Avoid claiming exact future route matches when dynamic values are
unknown.
- [x] Include the database/runtime only when a reachable configured
query requires them.
- [x] Ensure detail resources requesting Markdown content include all
statically eligible bodies needed by dynamic routes.

### 10. Integrate all targets

- [x] SSR: load the build's bounded database and conditionally include
the small runtime.
- [x] SSG: query the same database during generation, then omit
database/runtime output when no runtime query is needed.
- [x] Local/CLI: compile from the filesystem source with the same
semantics.
- [x] Preserve overview pagination, alternative route fields,
draft-as-user-data behavior, and full article rendering.
- [x] Verify ordinary Assets and fresh query-free sites perform no
parser, artifact, content-runtime dependency install, or cleanup work.

### 11. Correct API and package ownership

- [x] Move external Assets REST/OpenAPI contracts to
`@webstudio-is/protocol`.
- [x] Keep Webstudio domain models and Assets system-resource
configuration in the SDK.
- [x] Keep both cookie and token authentication behavior on the unified
Assets REST surface.
- [x] Remove pass-through route wrappers/reexports that add no transport
behavior.
- [x] Keep published runtime queries local to the artifact rather than
routing them through Builder REST.

### 12. Remove superseded deployment infrastructure

- [x] Remove external deployment-index upload, fetch, invalidation,
cleanup, and garbage-collection code.
- [x] Remove the SaaS asset-origin adapter requirement; close
webstudio-is/webstudio-saas#652 as superseded because it has no
independent purpose.
- [x] Remove index-bucket compatibility configuration and avoid adding
Infisical variables.
- [x] Remove obsolete external-index paths, package dependencies,
routes, scripts, and tests.
- [x] Consolidate unreleased intermediate migrations where safe and
retain only schema required by the final hosted cache/API design.

### 13. Verification and release readiness

- [x] Move and preserve existing tests while changing ownership; add new
tests only for changed contracts and behavior.
- [x] Run content-engine compiler/runtime tests, query-builder tests,
SDK/protocol/API tests, Builder tests, CLI tests, and affected
typechecks.
- [x] Run formatting, generated-file idempotence, fixtures, clean
Supabase-compatible migrations, E2E shards, and complete agent
evaluations.
- [x] Verify SSR and SSG conditional inclusion and query-free generated
package manifests.
- [x] Run the five-article MCP blog workflow: authoring, field
discovery, overview, dynamic detail routes, full Markdown rendering,
publishing, and published-site verification.
- [x] Run a maximum-budget database test and a
mutation-during-compilation test.
- [x] Perform a final architecture, security, API compatibility,
migration, bundle, correctness, and DRY review.
- [x] Reconcile every checkbox and replace the verification section with
results from the final architecture before release.

## Future MDX work

MDX is intentionally separate from this release.

- The compiler contract should allow a future build-only MDX parser.
- MDX compilation must not enter the published runtime.
- Embedded JSX must use an explicit compiled representation rather than
runtime source evaluation.
- Webstudio components, tokens, styles, and component allowlists remain
Webstudio integration concerns.
- The future MDX work should reuse `ContentArtifact` and
`ContentDatabase` where practical.

## Superseded design

The following earlier design choices are no longer the target
architecture:

- GROQ or GraphQL as the Assets query language.
- One external or long-lived index object per resource/query.
- A deployment index fetched by the Worker from SaaS/CDN infrastructure.
- Lazy production reads of source Markdown bodies.
- A separate index bucket, asset origin, invalidation service, or
garbage collector.
- A 4 MiB published index limit.
- Rebuilding an old build's database from the current mutable asset set.
- Bundling query infrastructure into projects without a configured
content query.

## Verification baseline

The embedded runtime-database architecture previously passed the checks
below. The OpenAPI-driven query-authoring change additionally passed:

- Query-builder core: 4 files / 19 tests.
- Query-builder React: 1 file / 7 tests.
- SDK Assets source/config: 1 file / 7 tests.
- Protocol Assets OpenAPI: 1 file / 9 tests, including independent
OpenAPI validation.
- Builder Assets OpenAPI description: 1 file / 5 tests.
- Query-builder, React, SDK, protocol, and Builder typechecks.
- Package-boundary validation and formatting of every affected file.

Local verification baseline:

- Content engine: 15 files / 104 tests.
- Asset uploader: 24 files / 175 tests.
- Builder: 204 files / 2,423 tests.
- CLI: 54 files / 866 tests.
- Complete workspace typecheck, formatting, lint, package boundaries,
generated fixtures, and fixture builds.
- The complete migration history applied successfully to a fresh
Supabase-compatible database, including every migration after the
consolidated Assets migration.
- Builder production build and full local E2E suite passed.
- Complete agent evaluations passed: authenticated page (170,859 ms),
design input (207,101 ms), and font assets (67,050 ms).
- Final review additionally verified correct REST edit/view/build
permits, cookie and public-token read paths, immutable snapshot retries,
exact database budgeting, query-free dependency omission, and SSR/SSG
conditional runtime inclusion.

Final staging verification passed against the embedded runtime-database
architecture: MCP uploaded and updated five Markdown articles,
discovered dynamic frontmatter fields, previewed a five-item sorted
overview and one full-body slug query, updated both Assets resources,
found zero binding issues, rendered `/blog` and `/blog/dynamic-routes`
at HTTP 200, published build version 266, and verified both published
routes at HTTP 200 with the expected article metadata and Markdown body.
The workflow also found and fixed offset-form asset timestamps failing
canonical datetime validation and made MCP `preview.start({ port: 0 })`
correctly select an available local port.

## Release constraints

- No new SaaS content service, bucket, or environment variable.
- No published-site database dependency.
- No parser/compiler dependency in query-free sites or runtime-only
bundles.
- The validated content database is immutable build output.
- Publishing warnings do not block deployment.
- The selected standard OpenAPI operation and its request JSON Schema
are the only source of query UI semantics.
 2026-07-31 00:17:46 +02:00
Oleg Isonen dde30ffb54 feat: Improve MCP authoring, compact output, and agent onboarding (#5883) 
## Summary

This improves how agents discover, author, inspect, and verify Webstudio
projects while reducing unnecessary MCP context usage.

## What changed

- Adds an **Agents** section to Project Settings with ready-to-run setup
commands and client configuration for Claude Code, Codex, Cursor, and VS
Code.
- Makes large list and schema responses compact and paginated by
default, with **verbose** available when complete records or diagnostics
are needed.
- Applies consistent compact output to assets, instances, styles, design
tokens, CLI schemas, and project-session metadata.
- Returns actionable expression warnings for props, resources, and
Collection bindings, including unknown variables and risky render-time
mutation resources.
- Improves Collection expression guidance so agents can bind array items
with the correct scoped variables.
- Reports rendered audit outcomes explicitly as complete, partial,
confirmation-required, or failed.
- Expands runtime output schemas so MCP clients can validate responses
reliably.
- Strengthens the packaged CLI smoke workflow with a realistic
seven-page stdio MCP scenario.

## Architecture

- Consolidates `project-build` from 65 file-oriented entrypoints to 11
architecture-owned surfaces.
- Separates the core model, contracts, runtime authoring, transfer,
state, project sessions, MCP, visual tooling, persistence codecs, and
authenticated server services.
- Isolates Node-only JSX evaluation from browser-safe transfer utilities
and authenticated server modules.
- Keeps persistence codecs independent from authenticated database
services, preventing unrelated module initialization.
- Defines shared instance-path types once and reuses them across runtime
modules.
- Preserves Immer initialization through explicit package side-effect
metadata.
- Adds package tests that prevent a return to source-file entrypoints or
generic exposure of Node-only surfaces.

## Validation

- Full repository checks, tests, typechecks, generated-file checks, and
fixtures
- All 163 Builder test files and 2,141 tests
- All 71 project-build test files and 1,794 tests
- Lint and package-boundary validation
- Full packed CLI release smoke test, including install, preview, agent
setup, seven-page stdio MCP workflow, rendered audit, and structured
commands
 2026-07-15 18:11:28 +02:00
Oleg Isonen 1ddf6b02c9 feat: API and CLI for everything (#5838) 
Closes #5287
 2026-07-01 01:20:50 +02:00
Oleg Isonen 33f720fb9f fix: Fix asset resource on published sites (#5565) 
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
 2026-01-15 18:26:32 +01:00
Oleg Isonenandzehjotkah 0116966822 feat: Updated asset manager: new file types, asset resource, filters, sorting (#5539) 
## Description

- [x] new file types support
- [x] categories filter
- [x] sorting filter
- [x] asset resource for access from custom code


## Steps for reproduction

1. click button
3. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
0000)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file

---------

Co-authored-by: zehjotkah <111518041+zehjotkah@users.noreply.github.com>
 2025-12-24 18:25:46 +01:00
zehjotkah 1f2e7a25ee feat: add Current Date resource, system.pathname, and Time component custom formatting (#5434) 
Closes https://github.com/webstudio-is/webstudio/issues/3510

## Summary

This PR adds a **Current Date** system resource, extends the `system`
object with a `pathname` property, and enhances the **Time component**
with custom date formatting capabilities to provide more flexibility and
prevent hydration errors.

### Key Features

1. **Current Date System Resource** - Provides normalized date/time
information that prevents React hydration errors
2. **System Pathname** - Adds `pathname` property directly to the
`system` object for easy access in expressions
3. **Time Component Custom Formatting** - Adds a `format` prop with
template tokens for flexible date/time display

## Motivation

These changes solve important use cases:

- **Current Date**: Prevents React hydration errors when displaying
dynamic dates (e.g., copyright years in footers). Previously, using `new
Date()` in expressions would cause server/client mismatch.
- **System Pathname**: Provides easy access to the current page path in
expressions without needing to parse the URL manually.
- **Custom Time Formatting**: Allows users to format dates with simple
templates (e.g., `YYYY-MM-DD`) without being limited to
`Intl.DateTimeFormat` options, and provides a simple way to exclude
specific date or time components to avoid hydration issues.

## Changes

### Current Date Resource
- Created
[`apps/builder/app/shared/$resources/current-date.server.ts`](apps/builder/app/shared/$resources/current-date.server.ts)
- Provides date fields: `year`, `month`, `day`, `iso`, `timestamp`
- All values normalized to midnight UTC (00:00:00.000Z) to prevent
hydration errors
- Registered in
[`rest.resources-loader.ts`](apps/builder/app/routes/rest.resources-loader.ts)
- Added to SystemResourceForm dropdown in UI

### System Pathname
- Added `pathname` property to
[`System`](packages/sdk/src/schema/pages.ts) type
- Updated [`system.ts`](apps/builder/app/shared/system.ts) to compute
and include pathname
- Normalized empty pathname to `'/'` for consistency
- Available in expressions as `system.pathname`

### Time Component Enhancements
- Added [`formatDate()`](packages/sdk-components-react/src/time.tsx:353)
function supporting template tokens:
  - `YYYY`, `YY` - year (4-digit, 2-digit)
  - `MM`, `M` - month (zero-padded, no padding)
  - `DD`, `D` - day (zero-padded, no padding)
  - `HH`, `H` - hours (zero-padded, no padding)
  - `mm`, `m` - minutes (zero-padded, no padding)
  - `ss`, `s` - seconds (zero-padded, no padding)
- Added `format` prop to
[`Time`](packages/sdk-components-react/src/time.tsx:386) component that
overrides `dateStyle`/`timeStyle` when provided
- Updated component logic to check for `format` prop first before
falling back to `Intl.DateTimeFormat`
- Added comprehensive tests for date formatting in
[`time.test.ts`](packages/sdk-components-react/src/time.test.ts)
- Updated component metadata in
[`time.ws.ts`](packages/sdk-components-react/src/time.ws.ts) to include
`format` in initialProps
- Example: `format="YYYY-MM-DD HH:mm:ss"` displays as "2025-11-03
18:47:25"

### Template & Fixture Updates
- Updated all CLI templates (defaults, react-router, SSG) to include
`pathname` in system objects
- Updated all fixture files (47 files total) to include `pathname` in
system objects
- All 694 tests passing
- All typecheck passing

## Usage

### Current Date Resource
Users can now add the Current Date resource from the Data Variables
panel:
1. Click + next to Data Variables
2. Select "System Resource"
3. Choose "Current Date" from the dropdown
4. Access in expressions:
   - `date.year` - for copyright year
   - `date.month` - current month (1-12)
   - `date.day` - current day (1-31)
   - `date.iso` - ISO 8601 string
   - `date.timestamp` - Unix timestamp

### System Pathname
Access the current page path directly:
- `system.pathname` - current page path (e.g., `/about`, `/blog/post-1`)

### Time Component Custom Format
Use the `format` prop for custom date/time display:
```jsx
<Time datetime="2025-11-03T18:47:25Z" format="YYYY-MM-DD" />
// Displays: 2025-11-03

<Time datetime="2025-11-03T18:47:25Z" format="YYYY-MM-DD HH:mm:ss" />
// Displays: 2025-11-03 18:47:25

<Time datetime="2025-11-03T18:47:25Z" format="M/D/YYYY" />
// Displays: 11/3/2025
```
Technical Notes
- Current Date values are normalized to midnight UTC to ensure
consistency throughout the day and prevent hydration mismatches
- The pathname property is computed from the request URL and normalized
to '/' when empty
- Custom format templates in the Time component are processed before
falling back to Intl.DateTimeFormat
- All changes are backward compatible
 2025-11-03 23:17:53 +01:00
Ivan Starkov 482839e83a chore: Remove prisma from the Builder app (#4182) 
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
 2024-10-01 13:51:50 +03:00
Ivan Starkov f46abe2f04 fix: Support new project links in the cli (#4049) 
## Description

closes #4048

## Steps for reproduction

see fixtures
We are still supporting `BUILDER_URL_DEPRECATED` (in a single fixture)
and all other fixtures uses modern link format


## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
 2024-09-02 16:59:06 +03:00
Ivan Starkov aa0f35222c fix: Sitemap resource loading issue (#4042) 
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env` file
 2024-08-31 19:47:01 +03:00
Oleg Isonen cc6dc6481b fix: Fix sitemap urls when using folders (#3281) 
## Description

1. What is this PR about (link the issue and add a short description)

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env.example`
and the `builder/env-check.js` if mandatory
 2024-05-02 14:10:44 +02:00
Ivan Starkov d8994140ba feat: Sitemap resource (#3228) 
## Description

## First Part 

Here is only new local resource `/$resources/sitemap.xml` support

It can't be implemented as route for now as we have 2 issues:
1. Cloudflare does not allow server fetches from same domain
2. Remix does not have fetch like sveltekit allowing to bypass real http
roundtrip when fetching local resources

The idea is to write/use it like any route in case of anything of above
will be solved.

Most PR Size is generated new fixture page.
Move SiteData into the resource.

Here is example of rendering this resource using html.

Project to play:

https://sitemap.staging.webstudio.is/builder/cab934af-8615-411a-99c1-5822a98eac74

Published site:
https://sitemap-saas-t6j12.wstd.work/

<img width="785" alt="image"
src="https://github.com/webstudio-is/webstudio/assets/5077042/b62af518-37bb-4daa-9f57-9145b4c77364">

## Next steps:

- Add `origin` to the system, or even better add `system.hostUrl` with
origin, hostname etc
- Add xml element and hardcoded sitemap.xml page

## Steps for reproduction

1. click button
4. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env.example`
and the `builder/env-check.js` if mandatory
 2024-04-24 13:05:53 +03:00