Ivan Starkov aaaf57da9f feat: Add Twitter card support & exclude pages from search/index (#2545) ...

## Description

ref #2544

Corresponding SAAS PR
https://github.com/webstudio-is/webstudio-saas/pull/215

- Support of page property - `excludePageFromSearch`
- Introduced functionality to enable the meta robot noindex tag for the
entire website.
- Implemented robots.txt support, which appears to be essential for
Twitter meta cards functionality.

Needs corresponding PR in SAAS deploy.

## Steps for reproduction

1. click button
2. expect xyz

## Code Review

- [ ] hi @kof, I need you to do
  - conceptual review (architecture, feature-correctness)
  - detailed review (read every line)
  - test it on preview

## Before requesting a review

- [ ] made a self-review
- [ ] added inline comments where things may be not obvious (the "why",
not "what")

## Before merging

- [ ] tested locally and on preview environment (preview dev login:
5de6)
- [ ] updated [test
cases](https://github.com/webstudio-is/webstudio/blob/main/apps/builder/docs/test-cases.md)
document
- [ ] added tests
- [ ] if any new env variables are added, added them to `.env.example`
and the `builder/env-check.js` if mandatory

2023-11-02 09:39:06 +02:00

..

builder

feat: Add Twitter card support & exclude pages from search/index (#2545)

2023-11-02 09:39:06 +02:00