Bump release version to 0.9.35rc580

Updated the README to clarify support for archiving content requiring login and running JS during archiving.

.github/ISSUE_TEMPLATE/1-bug_report.yml

@@ -30,8 +30,8 @@ body:
       description: Please provide the exact steps you took to trigger the issue (including any shell commands run, URLs visited, buttons clicked, etc.).
       render: markdown
       placeholder: |
-        1. Started ArchiveBox by running: `docker run -v $PWD:/data -p 8000:8000 archivebox/archivebox` in iTerm2
-        2. Went to the https://127.0.0.1:8000/add/ page in Google Chrome
+        1. Started ArchiveBox by running: `docker run -v $PWD:/data -p 5797:5797 archivebox/archivebox` in iTerm2
+        2. Went to the https://127.0.0.1:5797/add/ page in Google Chrome
         3. Typed 'https://example.com/xyz' into the 'Add URL' input field
         4. Clicked the 'Add+' button
         5. Got a 500 error and saw the errors below in terminal
@@ -138,7 +138,7 @@ body:
             archivebox:
                 image: archivebox/archivebox:latest
                 ports:
-                    - 8000:8000
+                    - 5797:5797
                 volumes:
                     - ./data:/data
                 environment:

.github/configs/ci-tooling.json

@@ -116,6 +116,10 @@
     }
   ],
   "docker_binaries": [
+    {
+      "name": "{BASH_BINARY}",
+      "binproviders": "env,brew,apt"
+    },
     {
       "name": "{DOCKER_BINARY}",
       "binproviders": "env"

.github/pages/README.md

@@ -0,0 +1,110 @@
+# archivebox.io website
+
+This repository owns its complete website. Build and deploy it without checking
+out another repository's website, fetching shared snippets, or calling a shared
+workflow. Common files are ordinary committed copies; update them deliberately.
+
+## Repository-specific files
+
+- Content: `publicsite/index.html`.
+- Site/gallery generation: `bin/generate_ui_screenshot_gallery.py`.
+- Application capture: `bin/collect_ui_screenshots.sh`.
+- Deployment branch: `dev`.
+
+## Common publication contract
+
+1. Build pull requests without deploying them.
+2. Build website pushes immediately, independently of application capture CI.
+3. Restore validated screenshots from trusted default-branch capture jobs.
+   Keep the original capture revision, timestamp, coverage and image checksums.
+4. Retain `site-screenshots` artifacts for 90 days. After expiry, restore the
+   published image set. Never replace a complete gallery with an empty one when
+   restoration fails. Apple device groups are independently validated artifacts;
+   Android's checked-in bootstrap captures retain their local provenance.
+5. Render current content and layout around those captures. Site revision and
+   capture revision are separate facts; reusing captures must not relabel them.
+6. Serialize production restore/build/deploy under one concurrency group and
+   check out the latest default-branch revision when each build starts. Keep the
+   built revision in provenance; later release-bot commits must not block deploys.
+7. Keep platform-specific build, installation, capture, and completeness checks
+   in the application's existing capture workflow. Site-only edits do not need
+   to rerun those jobs.
+
+## Common presentation contract
+
+The base header, Apps menu and icons, footer, brand styles and navigation behavior
+should remain identical copies across the seven product sites. Put this site's
+name, navigation, CTA and extra footer resources outside the common base files.
+Preserve real links, anchor IDs, no-JavaScript navigation, keyboard operation,
+mobile layout, reduced-motion preferences and existing product-specific content.
+Screenshots stay in English. Translation content, when added, lives in separate
+HTML files; no runtime translation service or cross-repository dependency.
+
+## Editing and building
+
+`base/` contains identical independent copies across the seven product sites.
+Common Python scripts share the same behavior; repository formatters may change
+line wrapping without changing their syntax trees. The base owns the shared logo, Apps menu/icons, footer directory, navigation
+behavior and styles. `site.json` and `nav.html` contain this site's name, routes,
+links and CTA. Product content and additional footer resources stay in the native
+HTML/Jinja/Liquid templates listed above. The `ARCHIVEBOX:*` comments are build
+slots; the renderer fills them with static HTML, never browser-side fetches.
+
+The existing builder calls `site.py render` after generating its content. This
+copies `base/chrome.css` and `base/chrome.js` to `site-base/`, assembles the common
+HTML, validates local resources and writes the site revision to root `build.json`.
+Screenshot manifests are left untouched. No build reads another repository.
+
+Common changes are manual copy/paste edits. Compare `base/` and `site.py` with
+another product repository when updating them; do not add a synchronization job,
+package dependency or reusable cross-repository workflow. `nav.html`, `site.json`
+and the marked workflow build section are intentionally repository-specific.
+
+Build: `uv run --no-project python .github/pages/site.py render _site --source publicsite` after restoring screenshots and refreshing the gallery.
+
+Check the built site with real Chromium (also run by Pages CI):
+
+```sh
+uv run --no-config --no-project --with playwright==1.63.0 playwright install chromium
+uv run --no-config --no-project --with playwright==1.63.0 python .github/pages/verify.py SITE_OUTPUT --evidence /tmp/site-evidence
+```
+
+Use this workflow's `SITE_OUTPUT` directory. The check covers desktop/mobile
+layout, keyboard dismissal, no-JavaScript links, local resources and visible
+images, and uploads its screenshots as `site-verification` for 14 days.
+
+## Branding and future translations
+
+`base/chrome.css` owns the shared system font stacks, plum accent and CTA shape,
+including the existing `.button` and `.ab-btn` classes. Product styles own their
+layouts, feature sections and platform-specific button states. Shared system
+fonts need no external font downloads. `site.py` emits the same complete SEO/OG/
+Twitter tags everywhere while retaining each native template's title, description,
+canonical URL, locale and social artwork; edit those values with that page's copy.
+
+For translations, keep complete separate HTML files under `es/`, `fr/` and `zh/`,
+and list their output paths in `site.json` alongside the English pages. Share CSS,
+JS, social artwork and the English screenshot gallery. Translate each page's title
+and description, set its own canonical and `lang` (`zh` for Chinese), and add
+reciprocal `hreflang` links only for translations that actually exist. The renderer
+preserves these tags and computes shared asset paths for nested pages. Localized
+header/footer copy and the small language selector can be added with those pages;
+there is no language redirect, placeholder translation or translation runtime now.
+
+## Screenshot strips
+
+`marquee.py` and `base/marquee.css` / `base/marquee.js` are also identical copies.
+Product homepages choose their placement with an `ARCHIVEBOX:MARQUEE` comment.
+The renderer takes real images, captions, dimensions and links from the generated
+`screenshots/index.html`; there is no second screenshot list or browser fetch.
+Desktop/tablet/mobile galleries contribute their desktop views. Apple uses
+`MARQUEE-CLIENT` and `MARQUEE-SERVER` slots to keep its two products separate.
+Catalog sites without a slot do not load the strip assets.
+
+The strip scrolls once, pauses on hover, focus or manual interaction, and has an
+explicit Play/Pause button. Reduced-motion users start paused. Without JavaScript,
+normal image links and horizontal scrolling remain available. Screenshot capture
+manifests and provenance are never rewritten by this presentation step.
+
+Standalone one- or two-image illustrations remain in their authored source.
+Gallery strips with three or more images use the latest restored captures.

.github/pages/artifacts.py

@@ -0,0 +1,102 @@
+"""Common screenshot artifact transport; coverage checks belong to each app."""
+
+import json
+import subprocess
+from pathlib import PurePosixPath
+from urllib.request import urlopen
+
+
+def api(repo, path):
+    return json.loads(subprocess.check_output(["gh", "api", f"repos/{repo}/{path}"]))
+
+
+def trusted(run, repo, workflow, branch, successful=True):
+    return (
+        (run.get("head_repository") or {}).get("full_name", "").lower() == repo.lower()
+        and run["head_branch"] == branch
+        and run["event"] in ("push", "workflow_dispatch")
+        and run["path"] == f".github/workflows/{workflow}"
+        and (not successful or run["conclusion"] == "success")
+    )
+
+
+def runs(repo, workflow, branch, successful=True, artifact_names=("site-screenshots",)):
+    # Ask for capture artifacts directly instead of checking every historical
+    # workflow run (most app runs never produce a complete screenshot set).
+    # Artifact creation time also includes fresh captures from old rerun jobs.
+    candidates = []
+    for name in artifact_names:
+        rows = subprocess.check_output(
+            [
+                "gh",
+                "api",
+                f"repos/{repo}/actions/artifacts?name={name}&per_page=100",
+                "--paginate",
+                "--jq",
+                ".artifacts[] | select(.expired == false) | @json",
+            ],
+            text=True,
+        )
+        candidates.extend(json.loads(row) for row in rows.splitlines())
+    seen = set()
+    for artifact in sorted(candidates, key=lambda item: item["created_at"], reverse=True):
+        source = artifact.get("workflow_run") or {}
+        run_id = source.get("id")
+        if (
+            artifact["name"] not in artifact_names
+            or not run_id
+            or run_id in seen
+            or source.get("head_branch") != branch
+            or source.get("head_repository_id") != source.get("repository_id")
+        ):
+            continue
+        seen.add(run_id)
+        run = api(repo, f"actions/runs/{run_id}")
+        if trusted(run, repo, workflow, branch, successful):
+            yield run
+
+
+def names(repo, run):
+    rows = subprocess.check_output(
+        [
+            "gh",
+            "api",
+            f"repos/{repo}/actions/runs/{run['id']}/artifacts?per_page=100",
+            "--paginate",
+            "--jq",
+            ".artifacts[] | @json",
+        ],
+        text=True,
+    )
+    return {artifact["name"] for row in rows.splitlines() if not (artifact := json.loads(row))["expired"]}
+
+
+def download(repo, run, name, destination):
+    subprocess.run(
+        [
+            "gh",
+            "run",
+            "download",
+            str(run["id"]),
+            "--repo",
+            repo,
+            "--name",
+            name,
+            "--dir",
+            str(destination),
+        ],
+        check=True,
+    )
+
+
+def relative_path(name):
+    path = PurePosixPath(name)
+    if not name or path.is_absolute() or ".." in path.parts or any(c in name for c in ":\\?#%"):
+        raise ValueError(f"Unsafe screenshot path: {name}")
+    return path
+
+
+def fetch(base, name):
+    relative_path(name)
+    with urlopen(base + name, timeout=60) as response:
+        return response.read()

.github/pages/base/chrome.css

@@ -0,0 +1,146 @@
+/* Common brand tokens. Native templates own product layout and content. */
+:root {
+  --abx-font-sans: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
+  --abx-font-mono: ui-monospace, "SFMono-Regular", Consolas, "Liberation Mono", Menlo, monospace;
+  --abx-accent: #9b2854;
+  --abx-accent-hover: #782040;
+  --abx-paper: #fcfaf7;
+  --abx-ink: #28252a;
+  --abx-muted: #716c74;
+  --abx-line: #e9e2e3;
+  --font-sans: var(--abx-font-sans);
+  --font-mono: var(--abx-font-mono);
+  --ab-font-sans: var(--abx-font-sans);
+  --ab-font-serif: var(--abx-font-sans);
+  --ab-font-label: var(--abx-font-sans);
+  --ab-font-mono: var(--abx-font-mono);
+  --ab-accent: var(--abx-accent);
+  --ab-accent-hover: var(--abx-accent-hover);
+  --ab-on-accent: #fff;
+}
+body { font-family: var(--abx-font-sans); }
+code, pre, kbd, samp { font-family: var(--abx-font-mono); }
+/* Reuse existing CTA classes; platform selection continues to control colors. */
+.button, .ab-btn {
+  display: inline-flex; align-items: center; justify-content: center; gap: 10px;
+  min-height: 44px; height: auto; max-width: 100%; padding: 10px 20px;
+  border-radius: 24px; font-family: var(--abx-font-sans);
+  font-size: 14px; font-weight: 650; line-height: 1.4;
+  text-align: center; white-space: normal; text-decoration: none;
+}
+.button.primary, .ab-btn--primary { background: var(--abx-accent); border-color: var(--abx-accent); color: #fff; }
+.button.primary:hover, .ab-btn--primary:hover { background: var(--abx-accent-hover); border-color: var(--abx-accent-hover); color: #fff; }
+.button:focus-visible, .ab-btn:focus-visible { outline: 3px solid #d67c9f; outline-offset: 4px; }
+.button svg, .button img, .ab-btn svg, .ab-btn img { flex-shrink: 0; }
+@media (prefers-reduced-motion: reduce) { .button, .ab-btn { transition: none; } .button:hover { transform: none; } }
+
+/* Shared ArchiveBox site chrome. Independent copy; edit alongside the other product sites, without a build dependency. */
+/* Native anchor navigation leaves room for the heading and its surrounding content. */
+html { scroll-behavior: smooth; scroll-padding-top: clamp(96px, 25vh, 240px); }
+@media (prefers-reduced-motion: reduce) { html { scroll-behavior: auto; } }
+.abx-header, .abx-footer { box-sizing: border-box; font-family: var(--abx-font-sans); font-size: 14px; line-height: 1.5; letter-spacing: normal; text-align: left; }
+.abx-header *, .abx-footer * { box-sizing: border-box; }
+.abx-header { position: sticky; top: 0; z-index: 100; display: flex; align-items: center; justify-content: space-between; gap: 24px; width: 100%; max-width: none; margin: 0; padding: 15px max(24px, calc((100% - 1280px) / 2)); border: 0; border-bottom: 1px solid #e9e2e3; background: #fcfaf7f5; color: #28252a; backdrop-filter: blur(16px); }
+.abx-header a, .abx-footer a { text-decoration: none; text-underline-offset: .24em; }
+.abx-header .abx-brand, .abx-footer .abx-brand { display: inline-flex; flex: 0 0 auto; align-items: center; gap: 11px; margin: 0; padding: 0; color: inherit; font-size: 19px; font-weight: 750; line-height: 1.2; letter-spacing: -.5px; white-space: nowrap; }
+.abx-brand .abx-logo { display: block; flex: 0 0 auto; width: 32px; height: 32px; margin: 0; }
+.abx-header .abx-subsite { transform: translateY(3px); margin-left: 2px; color: #716c74; font-size: 12px; font-weight: 450; letter-spacing: 0; }
+.abx-header .abx-nav { display: flex; align-items: center; justify-content: flex-end; flex-wrap: wrap; gap: 0; width: auto; max-width: none; margin: 0; padding: 0; font-size: 13px; font-weight: 600; }
+.abx-nav > a, .abx-nav > details { margin: 0 0 0 15px; padding: 0 0 0 15px; border: 0; border-left: 1px solid #ded4d8; border-radius: 0; background: none; }
+.abx-nav > :first-child { margin-left: 0; padding-left: 0; border-left: 0; }
+.abx-header .abx-nav a { display: inline-flex; align-items: center; min-height: 36px; color: #4d424b; white-space: nowrap; }
+.abx-header .abx-nav a:hover { color: #9b2854; }
+.abx-header .abx-nav .abx-cta { margin-left: 18px; padding: 9px 17px; border: 1px solid #9b2854; border-radius: 24px; background: #9b2854; color: #fff; }
+.abx-header .abx-nav .abx-cta:hover { background: #782040; color: #fff; }
+.abx-header .abx-apps { position: relative; }
+.abx-apps > summary { display: flex; align-items: center; gap: 7px; min-height: 36px; list-style: none; cursor: pointer; color: #4d424b; }
+.abx-apps > summary::-webkit-details-marker { display: none; }
+.abx-apps > summary::before { content: '›'; display: inline-block; color: #9b2854; font-size: 22px; line-height: 1; transition: transform .15s; }
+.abx-apps[open] > summary::before { transform: rotate(90deg); }
+.abx-header .abx-app-links { position: absolute; right: 0; top: calc(100% + 12px); width: 440px; max-width: calc(100vw - 24px); max-height: min(70vh, 580px); overflow-y: auto; padding: 8px; border: 1px solid #e9e2e3; border-radius: 16px; background: #fcfaf7; box-shadow: 0 18px 48px #2d152522, 0 3px 8px #2d152508; }
+.abx-header .abx-app-row { display: flex; align-items: center; gap: 8px; min-height: 44px; padding: 4px 6px; border-radius: 10px; transition: background .15s; }
+.abx-header .abx-app-row:hover, .abx-header .abx-app-row:focus-within { background: #f2e8ed; }
+.abx-header .abx-app-links .abx-app-name { flex: 1; min-width: 0; min-height: 34px; padding: 5px 4px; border-radius: 6px; font-size: 13px; line-height: 1.35; white-space: normal; }
+.abx-header .abx-platforms { display: flex; flex: 0 0 auto; align-items: center; gap: 5px; }
+.abx-header .abx-app-links .abx-platform, .abx-header .abx-app-links .abx-tool { display: inline-flex; align-items: center; justify-content: center; flex: 0 0 auto; height: 30px; min-height: 30px; padding: 5px; border: 1px solid #e8dfe3; border-radius: 9px; background: #fffdfb; color: #4d424b; box-shadow: 0 2px 3px #2d152510, inset 0 1px 0 #fff; transition: transform .15s, box-shadow .15s, border-color .15s; }
+.abx-header .abx-app-links .abx-platform { width: 30px; }
+.abx-header .abx-platform svg { display: block; width: 18px; height: 18px; margin: 0; fill: currentColor; }
+.abx-header .abx-app-links .abx-platform-docker { color: #2386ce; }
+.abx-header .abx-app-links .abx-platform-windows, .abx-header .abx-app-links .abx-platform-app-store, .abx-header .abx-app-links .abx-platform-safari { color: #1879c4; }
+.abx-header .abx-app-links .abx-platform-android { color: #29965d; }
+.abx-header .abx-app-links .abx-platform-firefox { color: #e46b24; }
+.abx-header .abx-app-links .abx-platform-brave { color: #e8552b; }
+.abx-header .abx-app-links .abx-platform-edge { color: #168b95; }
+.abx-header .abx-app-links .abx-platform:hover, .abx-header .abx-app-links .abx-platform:focus-visible, .abx-header .abx-app-links .abx-tool:hover, .abx-header .abx-app-links .abx-tool:focus-visible { transform: translateY(-2px); border-color: #c99bb0; background: #fff; box-shadow: 0 5px 10px #2d15251c; }
+.abx-header .abx-app-links .abx-tool { padding: 5px 8px; font-size: 11px; white-space: nowrap; }
+.abx-header .abx-tools-row { margin-top: 5px; padding-top: 9px; border-top: 1px solid #e9e2e3; border-radius: 0 0 10px 10px; }
+.abx-header .abx-tools-row .abx-app-name { flex: 0 0 auto; }
+.abx-header .abx-tools-row .abx-platforms { flex: 0 1 auto; flex-wrap: wrap; justify-content: flex-end; margin-left: auto; }
+.abx-header .abx-app-links code { display: inline; vertical-align: baseline; font-family: ui-monospace, SFMono-Regular, Consolas, monospace; font-size: 1em; font-weight: inherit; line-height: inherit; margin: 0; padding: 0; border: 0; background: transparent; color: inherit; }
+.abx-header .abx-icon-stack { isolation: isolate; gap: 0; padding: 4px 5px; }
+.abx-header .abx-icon-stack .abx-integration { position: relative; z-index: 1; transform: translateY(var(--stack-y, 0px)) rotate(var(--stack-angle, -8deg)); transition: transform .18s ease, box-shadow .18s ease, border-color .18s ease; }
+.abx-header .abx-icon-stack .abx-integration + .abx-integration { margin-left: -10px; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(2) { --stack-angle: 9deg; --stack-y: -2px; z-index: 2; color: #569a31; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(3) { --stack-angle: -6deg; --stack-y: 2px; z-index: 3; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(4) { --stack-angle: 7deg; --stack-y: -1px; z-index: 4; color: #f00; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(5) { --stack-angle: -10deg; --stack-y: 1px; z-index: 5; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(6) { --stack-angle: 6deg; --stack-y: -2px; z-index: 6; }
+.abx-header .abx-icon-stack .abx-integration:nth-child(7) { --stack-angle: -5deg; z-index: 7; }
+.abx-header .abx-icon-stack .abx-integration:not(:hover, :focus-visible):has(~ .abx-integration:is(:hover, :focus-visible)) { transform: translate(-6px, var(--stack-y, 0px)) rotate(var(--stack-angle, -8deg)); }
+.abx-header .abx-icon-stack .abx-integration:is(:hover, :focus-visible) ~ .abx-integration:not(:hover, :focus-visible) { transform: translate(6px, var(--stack-y, 0px)) rotate(var(--stack-angle, -8deg)); }
+.abx-header .abx-icon-stack .abx-integration:is(:hover, :focus-visible) { z-index: 20; transform: translateY(-7px) rotate(0deg) scale(1.12); border-color: #c99bb0; box-shadow: 0 8px 14px #2d15252b; }
+@media (hover: none) { .abx-header .abx-icon-stack .abx-integration + .abx-integration { margin-left: -6px; } }
+@media (prefers-reduced-motion: reduce) { .abx-header .abx-icon-stack .abx-integration { transition: none; } }
+.abx-footer { position: relative; z-index: 1; display: block; width: 100%; max-width: none; margin: 72px 0 0; padding: 60px max(24px, calc((100% - 1280px) / 2)) 24px; border: 0; border-top: 3px solid #7e284a; background: radial-gradient(ellipse at 12% 0, #54243b55, transparent 48%), linear-gradient(120deg, #201923, #14151b 75%); color: #ddd2dd; color-scheme: dark; }
+.abx-footer .abx-footer-intro { display: flex; align-items: flex-start; justify-content: space-between; gap: 32px; margin: 0 0 42px; }
+.abx-footer .abx-brand { color: #fff4ed; font-size: 24px; }
+.abx-footer .abx-brand .abx-logo { width: 40px; height: 40px; }
+.abx-footer .abx-tagline { max-width: 430px; margin: 12px 0 0; color: #b8acbb; font-size: 14px; line-height: 1.7; }
+.abx-footer .abx-footer-motto { margin: 2px 0 0; color: #d8b39f; font: italic 25px/1.45 Georgia, serif; text-align: right; }
+.abx-footer .abx-footer-grid { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 42px; margin: 0; }
+.abx-footer .abx-footer-column { margin: 0; padding: 20px 0 0; border: 0; border-top: 1px solid #67505e66; background: none; }
+.abx-footer .abx-footer-column h2 { margin: 0 0 19px; padding: 0; border: 0; color: #e4acbb; font: 650 12px/1.4 var(--abx-font-sans); letter-spacing: .12em; text-transform: uppercase; }
+.abx-footer .abx-footer-column ul { display: block; list-style: none; margin: 0; padding: 0; }
+.abx-footer .abx-footer-column li { display: flex; align-items: center; gap: 10px; margin: 0; padding: 0; min-height: 36px; }
+.abx-footer .abx-footer-column li::before { content: none; }
+.abx-footer .abx-footer-column a { color: #ddd2dd; font-size: 13px; line-height: 1.55; }
+.abx-footer .abx-footer-column a:first-child { padding: 6px 0; }
+.abx-footer a:hover { color: #fff; text-decoration: underline; }
+.abx-footer .abx-source { display: inline-flex; align-items: center; justify-content: center; flex: 0 0 28px; width: 28px; height: 28px; border: 1px solid transparent; border-radius: 50%; color: #a58d9f; }
+.abx-footer .abx-source:hover { color: #f3b5cb; border-color: #80516a; background: #ffffff08; }
+.abx-source svg { width: 15px; height: 15px; fill: currentColor; }
+.abx-footer .abx-footer-bottom { display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: 12px 24px; margin-top: 32px; padding-top: 21px; border-top: 1px solid #67505e66; color: #a99bab; font-size: 12px; }
+.abx-footer .abx-footer-bottom p { margin: 0; font-size: inherit; color: inherit; }
+.abx-footer .abx-footer-bottom a { color: #cebbc9; }
+.abx-footer .abx-footer-local { margin: 24px 0 0; padding: 16px 0 0; border: 0; border-top: 1px solid #67505e66; border-radius: 0; background: none; color: #baacbc; font-size: 12px; }
+.abx-footer-local > summary { cursor: pointer; color: #d6bfce; }
+.abx-footer-local > div { padding-top: 18px; }
+.abx-footer-local a { color: #e5aabd; }
+.abx-footer-local .brand, .abx-footer-local .footer-intro, .abx-footer-local .footer-grid, .abx-footer-local .footer-meta { color: inherit; }
+.abx-footer-local .footer-card { background: transparent; border-color: #67505e66; }
+.abx-footer-local h2, .abx-footer-local h3, .abx-footer-local strong { color: #e4d8e3; }
+.abx-footer-local .footer-link-copy span, .abx-footer-local .footer-card-subtitle { color: #baacbc; }
+.abx-header a:focus-visible, .abx-header summary:focus-visible, .abx-footer a:focus-visible, .abx-footer summary:focus-visible { outline: 3px solid #d67c9f; outline-offset: 4px; }
+@media (max-width: 1100px) {
+  .abx-header { flex-wrap: wrap; gap: 8px 24px; padding: 12px 20px; }
+  .abx-header .abx-nav { justify-content: flex-start; }
+  .abx-header .abx-app-links { position: fixed; top: auto; left: 20px; right: auto; width: 440px; margin-top: 10px; }
+}
+@media (max-width: 700px) {
+  .abx-header { position: relative; padding: 14px 18px; }
+  .abx-header .abx-brand { flex-wrap: wrap; row-gap: 5px; }
+  .abx-header .abx-nav { width: 100%; gap: 4px 0; font-size: 12px; }
+  .abx-nav > a, .abx-nav > details { margin-left: 10px; padding-left: 10px; }
+  .abx-header .abx-nav .abx-cta { margin-left: 10px; padding: 7px 13px; }
+  .abx-header .abx-app-links { left: 12px; right: 12px; width: auto; max-height: 60vh; }
+  .abx-footer { padding: 40px 22px 22px; margin-top: 48px; }
+  .abx-footer .abx-footer-intro { display: block; margin-bottom: 30px; }
+  .abx-footer .abx-footer-motto { margin-top: 20px; text-align: left; font-size: 22px; }
+  .abx-footer .abx-footer-grid { grid-template-columns: 1fr; gap: 26px; }
+  .abx-footer .abx-footer-column li { min-height: 40px; }
+}
+@media (prefers-reduced-motion: reduce) { .abx-apps > summary::before, .abx-header .abx-app-row, .abx-header .abx-app-links .abx-platform, .abx-header .abx-app-links .abx-tool { transition: none; } .abx-header .abx-app-links .abx-platform:hover, .abx-header .abx-app-links .abx-tool:hover { transform: none; } }
+
+.abx-footer-local > div > a + a { margin-left: 12px; }
+
+@media (prefers-reduced-motion: reduce) { .abx-header .abx-icon-stack .abx-integration:is(:hover, :focus-visible) { transform: none; } }

.github/pages/base/chrome.js

@@ -0,0 +1,18 @@
+(() => {
+  const menu = document.querySelector('.abx-apps');
+  document.addEventListener('keydown', event => {
+    if (event.key === 'Escape' && menu?.open) { menu.open = false; menu.querySelector('summary').focus(); }
+  });
+  document.addEventListener('click', event => { if (menu?.open && !menu.contains(event.target)) menu.open = false; });
+  function revealAnchor() {
+    let target;
+    try { target = document.getElementById(decodeURIComponent(window.location.hash.slice(1))); } catch { return; }
+    if (!target) return;
+    let details = target.closest('details');
+    let opened = false;
+    while (details) { if (!details.open) { details.open = true; opened = true; } details = details.parentElement.closest('details'); }
+    if (opened) window.requestAnimationFrame(() => target.scrollIntoView());
+  }
+  window.addEventListener('hashchange', revealAnchor);
+  revealAnchor();
+})();

.github/pages/base/footer-end.html

@@ -0,0 +1,2 @@
+  <div class="abx-footer-bottom"><p>Built in the open. Supported by our community.</p><p><a href="https://github.com/ArchiveBox">GitHub organization</a> · <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Security-Overview">Security &amp; privacy</a> · <a href="https://archivebox.io">ArchiveBox.io ↗</a></p></div>
+</footer>

.github/pages/base/footer-start.html

@@ -0,0 +1,3 @@
+<footer class="abx-footer" aria-label="ArchiveBox ecosystem">
+  <div class="abx-footer-intro"><div><a class="abx-brand" href="https://archivebox.io/"><svg class="abx-logo" viewBox="0 0 40 40" aria-hidden="true"><rect width="40" height="40" rx="10" fill="#9b2854"/><path d="M10 14h20v17H10z" fill="#fff4ed"/><path d="M8 9h24v6H8z" fill="#fff4ed"/><path d="M16 19h8v3h-8z" fill="#9b2854"/><path d="M12 31V17" stroke="#9b2854" stroke-width="2"/></svg><span>ArchiveBox</span></a><p class="abx-tagline">Open-source tools for keeping a lasting copy of the web.<br>Your data. Your devices. Your archive.</p></div><p class="abx-footer-motto">A home for the web<br>you want to keep.</p></div>
+  <div class="abx-footer-grid"><section class="abx-footer-column"><h2>Apps &amp; tools</h2><ul><li><a href="https://archivebox.io/">ArchiveBox Server</a><a class="abx-source" href="https://github.com/ArchiveBox/ArchiveBox" aria-label="ArchiveBox Server source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://app.archivebox.io/#get-started">ArchiveBox for macOS &amp; iOS</a><a class="abx-source" href="https://github.com/ArchiveBox/ios-archivebox" aria-label="ArchiveBox for macOS &amp; iOS source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://android.archivebox.io/">ArchiveBox for Android</a><a class="abx-source" href="https://github.com/ArchiveBox/android-archivebox" aria-label="ArchiveBox for Android source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://electron.archivebox.io/">ArchiveBox Desktop</a><a class="abx-source" href="https://github.com/ArchiveBox/electron-archivebox" aria-label="ArchiveBox Desktop source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://extension.archivebox.io/">ArchiveBox Browser Extension</a><a class="abx-source" href="https://github.com/ArchiveBox/archivebox-browser-extension" aria-label="ArchiveBox Browser Extension source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://plugins.archivebox.io/">ArchiveBox Plugin Library</a><a class="abx-source" href="https://github.com/ArchiveBox/abx-plugins" aria-label="ArchiveBox Plugin Library source on GitHub" title="View source on GitHub"><svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .75a11.25 11.25 0 0 0-3.56 21.92c.56.1.77-.24.77-.54v-2.1c-3.14.68-3.8-1.33-3.8-1.33-.51-1.3-1.25-1.65-1.25-1.65-1.02-.7.08-.69.08-.69 1.13.08 1.73 1.16 1.73 1.16 1 1.72 2.63 1.22 3.27.93.1-.73.39-1.22.71-1.5-2.5-.28-5.13-1.25-5.13-5.56 0-1.23.44-2.23 1.16-3.02-.12-.28-.5-1.43.11-2.98 0 0 .95-.3 3.1 1.15a10.8 10.8 0 0 1 5.64 0c2.15-1.46 3.1-1.15 3.1-1.15.62 1.55.23 2.7.12 2.98.72.79 1.16 1.8 1.16 3.02 0 4.32-2.64 5.27-5.15 5.55.4.35.76 1.03.76 2.08v3.11c0 .3.2.65.77.54A11.25 11.25 0 0 0 12 .75Z"/></svg></a></li><li><a href="https://abx-dl.archivebox.io/">One-shot abx-Line truncated

.github/pages/base/header.html

@@ -0,0 +1,14 @@
+<header class="abx-header">
+  <a class="abx-brand" href="https://archivebox.io/"><svg class="abx-logo" viewBox="0 0 40 40" aria-hidden="true"><rect width="40" height="40" rx="10" fill="#9b2854"/><path d="M10 14h20v17H10z" fill="#fff4ed"/><path d="M8 9h24v6H8z" fill="#fff4ed"/><path d="M16 19h8v3h-8z" fill="#9b2854"/><path d="M12 31V17" stroke="#9b2854" stroke-width="2"/></svg><span>ArchiveBox</span><span class="abx-subsite">__SITE_NAME__</span></a>
+  <nav class="abx-nav" aria-label="Main navigation">
+    <details class="abx-apps"><summary>Apps</summary><div class="abx-app-links">
+      <div class="abx-app-row"><a class="abx-app-name" href="https://archivebox.io/#install-compose">ArchiveBox Server</a><div class="abx-platforms"><a class="abx-platform abx-platform-docker" href="https://archivebox.io/#install-compose" aria-label="ArchiveBox Server for Docker" title="ArchiveBox Server for Docker"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 640 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M349.9 236.3l-66.1 0 0-59.4 66.1 0 0 59.4zm0-204.3l-66.1 0 0 60.7 66.1 0 0-60.7zm78.2 144.8l-66.1 0 0 59.4 66.1 0 0-59.4zM271.8 104.7l-66.1 0 0 60.1 66.1 0 0-60.1zm78.1 0l-66.1 0 0 60.1 66.1 0 0-60.1zm276.8 100c-14.4-9.7-47.6-13.2-73.1-8.4-3.3-24-16.7-44.9-41.1-63.7l-14-9.3-9.3 14c-18.4 27.8-23.4 73.6-3.7 103.8-8.7 4.7-25.8 11.1-48.4 10.7l-434.7 0c-8.7 50.8 5.8 116.8 44 162.1 37.1 43.9 92.7 66.2 165.4 66.2 157.4 0 273.9-72.5 328.4-204.2 21.4 .4 67.6 .1 91.3-45.2 1.5-2.5 6.6-13.2 8.5-17.1l-13.3-8.9zM115.6 176.8l-66 0 0 59.4 66.1 0 0-59.4-.1 0zm78.1 0l-66.1 0 0 59.4 66.1 0 0-59.4zm78.1 0l-66.1 0 0 59.4 66.1 0 0-59.4zm-78.1-72.1l-66.1 0 0 60.1 66.1 0 0-60.1z"/></svg></a><a class="abx-platform abx-platform-apple" href="https://app.archivebox.io/#archivebox-serverapp" aria-label="ArchiveBox Server for macOS" title="ArchiveBox Server for macOS"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M319.1 268.7c-.2-36.7 16.4-64.4 50-84.8-18.8-26.9-47.2-41.7-84.7-44.6-35.5-2.8-74.3 20.7-88.5 20.7-15 0-49.4-19.7-76.4-19.7-55.8 .9-115.1 44.5-115.1 133.2 0 26.2 4.8 53.3 14.4 81.2 12.8 36.7 59 126.7 107.2 125.2 25.2-.6 43-17.9 75.8-17.9 31.8 0 48.3 17.9 76.4 17.9 48.6-.7 90.4-82.5 102.6-119.3-65.2-30.7-61.7-90-61.7-91.9zM262.5 104.5c27.3-32.4 24.8-61.9 24-72.5-24.1 1.4-52 16.4-67.9 34.9-17.5 19.8-27.8 44.3-25.6 71.9 26.1 2 49.9-11.4 69.5-34.3z"/></svg></a><a class="abx-platform abx-platform-windows" href="https://electron.archivebox.io/" aria-label="ArchiveBox Server for Windows" title="ArchiveBox Server for Windows"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M0 93.7l183.6-25.3 0 177.4-183.6 0 0-152.1zM0 418.3l183.6 25.3 0-175.2-183.6 0 0 149.9zm203.8 28l244.2 33.7 0-211.6-244.2 0 0 177.9zm0-380.6l0 180.1 244.2 0 0-213.8-244.2 33.7z"/></svg></a><a class="abx-platform abx-platform-linux" href="https://archivebox.io/#install-apt" aria-label="ArchiveBox Server for Linux" title="ArchiveBox Server for Linux"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M220.9 123.3c1 .5 1.8 1.7 3 1.7 1.1 0 2.8-.4 2.9-1.5 .2-1.4-1.9-2.3-3.2-2.9-1.7-.7-3.9-1-5.5-.1-.4 .2-.8 .7-.6 1.1 .3 1.3 2.3 1.1 3.4 1.7zM199 125c1.2 0 2-1.2 3-1.7 1.1-.6 3.1-.4 3.5-1.6 .2-.4-.2-.9-.6-1.1-1.6-.9-3.8-.6-5.5 .1-1.3 .6-3.4 1.5-3.2 2.9 .1 1 1.8 1.5 2.8 1.4zM420 403.8c-3.6-4-5.3-11.6-7.2-19.7-1.8-8.1-3.9-16.8-10.5-22.4-1.3-1.1-2.6-2.1-4-2.9-1.3-.8-2.7-1.5-4.1-2 9.2-27.3 5.6-54.5-3.7-79.1-11.4-30.1-31.3-56.4-46.5-74.4-17.1-21.5-33.7-41.9-33.4-72 .5-45.9 5.1-131.2-75.8-131.3-102.4-.2-76.8 103.4-77.9 135.2-1.7 23.4-6.4 41.8-22.5 64.7-18.9 22.5-45.5 58.8-58.1 96.7-6 17.9-8.8 36.1-6.2 53.3-6.5 5.8-11.4 14.7-16.6 20.2-4.2 4.3-10.3 5.9-17 8.3s-14 6-18.5 14.5c-2.1 3.9-2.8 8.1-2.8 12.4 0 3.9 .6 7.9 1.2 11.8 1.2 8.1 2.5 15.7 .8 20.8-5.2 14.4-5.9 24.4-2.2 31.7 3.8 7.3 11.4 10.5 20.1 12.3 17.3 3.6 40.8 2.7 59.3 12.5 19.8 10.4 39.9 14.1 55.9 10.4 11.6-2.6 21.1-9.6 25.9-20.2 12.5-.1 26.3-5.4 48.3-6.6 14.9-1.2 33.6 5.3 55.1 4.1 .6 2.3 1.4 4.6 2.5 6.7l0 .1c8.3 16.7 23.8 24.3 40.3 23 16.6-1.3 34.1-11 48.3-27.9 13.6-16.4 36-23.2 50.9-32.2 7.4-4.5 13.4-10.1 13.9-18.3 .4-8.2-4.4-17.3-15.5-29.7zM223.8 87.3c9.8-22.2 34.2-21.8 44-.4 6.5 14.2 3.6 30.9-4.3 40.4-1.6-.8-5.9-2.6-12.6-4.9 1.1-1.2 3.1-2.7 3.9-4.6 4.8-11.8-.2-27-9.1-27.3-7.3-.5-13.9 10.8-11.8 23-4.1-2-9.4-3.5-13-4.4-1-6.9-.3-14.6 2.9-21.8zM183.1 75.8c10.1 0 20.8 14.2 19.1 33.5-3.5 1-7.1 2.5-10.2 4.6 1.2-8.9-3.3-20.1-9.6-19.6-8.4 .7-9.8 21.2-1.8 28.1 1 .8 1.9-.2-5.9 5.5-15.6-14.6-10.5-52.1 8.4-52.1zm-13.6 60.7c6.2-4.6 13.6-1Line truncated
+      <div class="abx-app-row"><a class="abx-app-name" href="https://app.archivebox.io/">ArchiveBox Desktop</a><div class="abx-platforms"><a class="abx-platform abx-platform-apple" href="https://app.archivebox.io/" aria-label="ArchiveBox Desktop for macOS" title="ArchiveBox Desktop for macOS"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M319.1 268.7c-.2-36.7 16.4-64.4 50-84.8-18.8-26.9-47.2-41.7-84.7-44.6-35.5-2.8-74.3 20.7-88.5 20.7-15 0-49.4-19.7-76.4-19.7-55.8 .9-115.1 44.5-115.1 133.2 0 26.2 4.8 53.3 14.4 81.2 12.8 36.7 59 126.7 107.2 125.2 25.2-.6 43-17.9 75.8-17.9 31.8 0 48.3 17.9 76.4 17.9 48.6-.7 90.4-82.5 102.6-119.3-65.2-30.7-61.7-90-61.7-91.9zM262.5 104.5c27.3-32.4 24.8-61.9 24-72.5-24.1 1.4-52 16.4-67.9 34.9-17.5 19.8-27.8 44.3-25.6 71.9 26.1 2 49.9-11.4 69.5-34.3z"/></svg></a><a class="abx-platform abx-platform-windows" href="https://windows.archivebox.io/" aria-label="ArchiveBox Desktop for Windows" title="ArchiveBox Desktop for Windows"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M0 93.7l183.6-25.3 0 177.4-183.6 0 0-152.1zM0 418.3l183.6 25.3 0-175.2-183.6 0 0 149.9zm203.8 28l244.2 33.7 0-211.6-244.2 0 0 177.9zm0-380.6l0 180.1 244.2 0 0-213.8-244.2 33.7z"/></svg></a><a class="abx-platform abx-platform-linux" href="https://linux.archivebox.io/" aria-label="ArchiveBox Desktop for Linux" title="ArchiveBox Desktop for Linux"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M220.9 123.3c1 .5 1.8 1.7 3 1.7 1.1 0 2.8-.4 2.9-1.5 .2-1.4-1.9-2.3-3.2-2.9-1.7-.7-3.9-1-5.5-.1-.4 .2-.8 .7-.6 1.1 .3 1.3 2.3 1.1 3.4 1.7zM199 125c1.2 0 2-1.2 3-1.7 1.1-.6 3.1-.4 3.5-1.6 .2-.4-.2-.9-.6-1.1-1.6-.9-3.8-.6-5.5 .1-1.3 .6-3.4 1.5-3.2 2.9 .1 1 1.8 1.5 2.8 1.4zM420 403.8c-3.6-4-5.3-11.6-7.2-19.7-1.8-8.1-3.9-16.8-10.5-22.4-1.3-1.1-2.6-2.1-4-2.9-1.3-.8-2.7-1.5-4.1-2 9.2-27.3 5.6-54.5-3.7-79.1-11.4-30.1-31.3-56.4-46.5-74.4-17.1-21.5-33.7-41.9-33.4-72 .5-45.9 5.1-131.2-75.8-131.3-102.4-.2-76.8 103.4-77.9 135.2-1.7 23.4-6.4 41.8-22.5 64.7-18.9 22.5-45.5 58.8-58.1 96.7-6 17.9-8.8 36.1-6.2 53.3-6.5 5.8-11.4 14.7-16.6 20.2-4.2 4.3-10.3 5.9-17 8.3s-14 6-18.5 14.5c-2.1 3.9-2.8 8.1-2.8 12.4 0 3.9 .6 7.9 1.2 11.8 1.2 8.1 2.5 15.7 .8 20.8-5.2 14.4-5.9 24.4-2.2 31.7 3.8 7.3 11.4 10.5 20.1 12.3 17.3 3.6 40.8 2.7 59.3 12.5 19.8 10.4 39.9 14.1 55.9 10.4 11.6-2.6 21.1-9.6 25.9-20.2 12.5-.1 26.3-5.4 48.3-6.6 14.9-1.2 33.6 5.3 55.1 4.1 .6 2.3 1.4 4.6 2.5 6.7l0 .1c8.3 16.7 23.8 24.3 40.3 23 16.6-1.3 34.1-11 48.3-27.9 13.6-16.4 36-23.2 50.9-32.2 7.4-4.5 13.4-10.1 13.9-18.3 .4-8.2-4.4-17.3-15.5-29.7zM223.8 87.3c9.8-22.2 34.2-21.8 44-.4 6.5 14.2 3.6 30.9-4.3 40.4-1.6-.8-5.9-2.6-12.6-4.9 1.1-1.2 3.1-2.7 3.9-4.6 4.8-11.8-.2-27-9.1-27.3-7.3-.5-13.9 10.8-11.8 23-4.1-2-9.4-3.5-13-4.4-1-6.9-.3-14.6 2.9-21.8zM183.1 75.8c10.1 0 20.8 14.2 19.1 33.5-3.5 1-7.1 2.5-10.2 4.6 1.2-8.9-3.3-20.1-9.6-19.6-8.4 .7-9.8 21.2-1.8 28.1 1 .8 1.9-.2-5.9 5.5-15.6-14.6-10.5-52.1 8.4-52.1zm-13.6 60.7c6.2-4.6 13.6-10 14.1-10.5 4.7-4.4 13.5-14.2 27.9-14.2 7.1 0 15.6 2.3 25.9 8.9 6.3 4.1 11.3 4.4 22.6 9.3 8.4 3.5 13.7 9.7 10.5 18.2-2.6 7.1-11 14.4-22.7 18.1-11.1 3.6-19.8 16-38.2 14.9-3.9-.2-7-1-9.6-2.1-8-3.5-12.2-10.4-20-15-8.6-4.8-13.2-10.4-14.7-15.3-1.4-4.9 0-9 4.2-12.3zm3.3 334c-2.7 35.1-43.9 34.4-75.3 18-29.9-15.8-68.6-6.5-76.5-21.9-2.4-4.7-2.4-12.7 2.6-26.4l0-.2c2.4-7.6 .6-16-.6-23.9-1.2-7.8-1.8-15 .9-20 3.5-6.7 8.5-9.1 14.8-11.3 10.3-3.7 11.8-3.4 19.6-9.9 5.5-5.7 9.5-12.9 14.3-18 5.1-5.5 10-8.1 17.7-6.9 8.1 1.2 15.1 6.8 21.9 16l19.6 35.6c9.5 19.9 43.1 48.4 41 68.9zm-1.4-25.9c-4.1-6.6-9.6-13.6-14.4-19.6 7.1 0 14.2-2.2 16.7-8.9 2.3-6.2 0-14.9-7.4-24.9-13.5-18.2-38.3-32.5-38.3-32.5-13.5-8.4-21.1-18.7-24.6-29.9s-3-23.3-.3-35.2c5.2-22.9 18.6-45.2 27.2-59.2 2.3-1.7 .8 3.2-8.7 20.8-8.5 16.1-24.4 53.3-2.6 82.4 .6-20.7 5.5-41.8 13.8-61.5 12-27.4 37.3-74.9 39.3-112.7 1.1 .8 4.6 3.2 6.2 4.1 4.6 2.7 8.1 6.7 12.6 10.3 12.4 10 28.5 9.2 42.4 1.2 6.2-3.5 11.2-7.5 15.9-9 9.9-3.1 17.8-8.6 22.3-15 7.7 30.4 25.7 74.3 37.2 95.7 6.1 11.4 18.3 35.5 23.6 64.6 3.3-.1 7 .4 10.9 1.4 13.8-35.7-11.7-74.2-23.3-84.9-4.7-4.6-4.9-6.6-2.6-6.5 12.6 11.2 29.2 33.7 35.2 59 2.8 11.6 3.3 23.7 .4 35.7 16.4 6.8 35.9 17.9 30.7 34.8-2.2-.1-3.2 0-4Line truncated
+      <div class="abx-app-row"><a class="abx-app-name" href="https://app.archivebox.io/">ArchiveBox Mobile</a><div class="abx-platforms"><a class="abx-platform abx-platform-apple" href="https://app.archivebox.io/" aria-label="ArchiveBox Mobile for iOS" title="ArchiveBox Mobile for iOS"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 384 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M319.1 268.7c-.2-36.7 16.4-64.4 50-84.8-18.8-26.9-47.2-41.7-84.7-44.6-35.5-2.8-74.3 20.7-88.5 20.7-15 0-49.4-19.7-76.4-19.7-55.8 .9-115.1 44.5-115.1 133.2 0 26.2 4.8 53.3 14.4 81.2 12.8 36.7 59 126.7 107.2 125.2 25.2-.6 43-17.9 75.8-17.9 31.8 0 48.3 17.9 76.4 17.9 48.6-.7 90.4-82.5 102.6-119.3-65.2-30.7-61.7-90-61.7-91.9zM262.5 104.5c27.3-32.4 24.8-61.9 24-72.5-24.1 1.4-52 16.4-67.9 34.9-17.5 19.8-27.8 44.3-25.6 71.9 26.1 2 49.9-11.4 69.5-34.3z"/></svg></a><a class="abx-platform abx-platform-android" href="https://android.archivebox.io/" aria-label="ArchiveBox Mobile for Android" title="ArchiveBox Mobile for Android"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!-- Font Awesome Free 5.0.13 by Fonticons, Inc. https://fontawesome.com License: https://creativecommons.org/licenses/by/4.0/ --><path d="M89.6 204.5v115.8c0 15.4-12.1 27.7-27.5 27.7-15.3 0-30.1-12.4-30.1-27.7V204.5c0-15.1 14.8-27.5 30.1-27.5 15.1 0 27.5 12.4 27.5 27.5zm10.8 157c0 16.4 13.2 29.6 29.6 29.6h19.9l.3 61.1c0 36.9 55.2 36.6 55.2 0v-61.1h37.2v61.1c0 36.7 55.5 36.8 55.5 0v-61.1h20.2c16.2 0 29.4-13.2 29.4-29.6V182.1H100.4v179.4zm248-189.1H99.3c0-42.8 25.6-80 63.6-99.4l-19.1-35.3c-2.8-4.9 4.3-8 6.7-3.8l19.4 35.6c34.9-15.5 75-14.7 108.3 0L297.5 34c2.5-4.3 9.5-1.1 6.7 3.8L285.1 73c37.7 19.4 63.3 56.6 63.3 99.4zm-170.7-55.5c0-5.7-4.6-10.5-10.5-10.5-5.7 0-10.2 4.8-10.2 10.5s4.6 10.5 10.2 10.5c5.9 0 10.5-4.8 10.5-10.5zm113.4 0c0-5.7-4.6-10.5-10.2-10.5-5.9 0-10.5 4.8-10.5 10.5s4.6 10.5 10.5 10.5c5.6 0 10.2-4.8 10.2-10.5zm94.8 60.1c-15.1 0-27.5 12.1-27.5 27.5v115.8c0 15.4 12.4 27.7 27.5 27.7 15.4 0 30.1-12.4 30.1-27.7V204.5c0-15.4-14.8-27.5-30.1-27.5z"/></svg></a></div></div>
+      <div class="abx-app-row"><a class="abx-app-name" href="https://extension.archivebox.io/">ArchiveBox Browser Extension</a><div class="abx-platforms"><a class="abx-platform abx-platform-chrome" href="https://chromewebstore.google.com/detail/archivebox-exporter/habonpimjphpdnmcfkaockjnffodikoj" aria-label="ArchiveBox Browser Extension for Chrome" title="ArchiveBox Browser Extension for Chrome"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!--! Font Awesome Free 6.7.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2024 Fonticons, Inc. --><path d="M0 256C0 209.4 12.47 165.6 34.27 127.1L144.1 318.3C166 357.5 207.9 384 256 384C270.3 384 283.1 381.7 296.8 377.4L220.5 509.6C95.9 492.3 0 385.3 0 256zM365.1 321.6C377.4 302.4 384 279.1 384 256C384 217.8 367.2 183.5 340.7 160H493.4C505.4 189.6 512 222.1 512 256C512 397.4 397.4 511.1 256 512L365.1 321.6zM477.8 128H256C193.1 128 142.3 172.1 130.5 230.7L54.19 98.47C101 38.53 174 0 256 0C350.8 0 433.5 51.48 477.8 128V128zM168 256C168 207.4 207.4 168 256 168C304.6 168 344 207.4 344 256C344 304.6 304.6 344 256 344C207.4 344 168 304.6 168 256z"/></svg></a><a class="abx-platform abx-platform-safari" href="https://extension.archivebox.io/" aria-label="ArchiveBox Browser Extension for Safari" title="ArchiveBox Browser Extension for Safari"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!--! Font Awesome Free 7.3.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2026 Fonticons, Inc. --><path fill="currentColor" d="M274.7 274.7L237.3 237.3 166 346 274.7 274.7zM256 8a248 248 0 1 0 0 496 248 248 0 1 0 0-496zM411.8 182.8l14.8-6.1c1-.4 2-.6 3.1-.6s2.1 .2 3.1 .6 1.8 1 2.6 1.7 1.3 1.6 1.7 2.6 .6 2 .6 3.1-.2 2.1-.6 3.1-1 1.9-1.7 2.6-1.6 1.3-2.6 1.7L418 197.6c-2 .8-4.2 .8-6.1 0s-3.5-2.4-4.3-4.3-.8-4.2 0-6.1 2.4-3.5 4.3-4.3zM314.4 94l6.1-14.8c.8-2 2.4-3.5 4.3-4.3s4.2-.8 6.1 0 3.5 2.4 4.3 4.3 .8 4.2 0 6.1l-6.1 14.8c-.8 2-2.4 3.5-4.3 4.3s-4.2 .8-6.1 0c-1-.4-1.9-1-2.6-1.7s-1.3-1.6-1.7-2.6-.6-2-.6-3.1 .2-2.1 .6-3.1zM256 60c2.1 0 4.2 .8 5.7 2.3S264 65.9 264 68l0 16c0 2.1-.8 4.2-2.3 5.7s-3.5 2.3-5.7 2.3-4.2-.8-5.7-2.3-2.3-3.5-2.3-5.7l0-16c0-2.1 .8-4.2 2.3-5.7S253.9 60 256 60zM181 74.9c1-.4 2-.6 3.1-.6s2.1 .2 3.1 .6 1.9 1 2.6 1.7 1.3 1.6 1.7 2.6L197.6 94c.4 1 .7 2 .7 3.1s-.2 2.1-.6 3.1-1 1.9-1.7 2.6-1.6 1.4-2.6 1.8-2 .6-3.1 .6-2.1-.2-3.1-.6-1.9-1-2.6-1.8-1.3-1.7-1.7-2.7l-6.1-14.8c-.8-2-.8-4.2 0-6.1s2.4-3.5 4.3-4.3zm-63.6 42.5c1.5-1.5 3.5-2.3 5.7-2.3s4.2 .8 5.7 2.3L140 128.7c1.5 1.5 2.3 3.5 2.3 5.6s-.8 4.1-2.3 5.6c-1.5 1.5-3.5 2.3-5.7 2.3s-4.2-.8-5.7-2.3l-11.3-11.3c-1.5-1.5-2.3-3.5-2.3-5.6s.8-4.1 2.3-5.6l0 0zM60 256c0-2.1 .8-4.2 2.3-5.7S65.9 248 68 248l16 0c2.1 0 4.2 .8 5.7 2.3s2.3 3.5 2.3 5.7-.8 4.2-2.3 5.7-3.5 2.3-5.7 2.3l-16 0c-2.1 0-4.2-.8-5.7-2.3S60 258.1 60 256zm40.2 73.2l-14.8 6.1c-1 .4-2 .6-3.1 .6s-2.1-.2-3.1-.6-1.9-1-2.6-1.7-1.3-1.6-1.7-2.6-.6-2-.6-3.1 .2-2.1 .6-3.1 1-1.9 1.7-2.6 1.6-1.3 2.6-1.7L94 314.4c2-.8 4.2-.8 6.1 0s3.5 2.4 4.3 4.3 .8 4.2 0 6.1-2.4 3.5-4.3 4.3zm4.3-136c-.4 1-1 1.9-1.7 2.6s-1.6 1.3-2.6 1.7-2 .6-3.1 .6-2.1-.2-3.1-.6l-14.8-6.1c-2-.8-3.5-2.4-4.3-4.3s-.8-4.2 0-6.1 2.4-3.5 4.3-4.3 4.2-.8 6.1 0l14.8 6.1c2 .8 3.5 2.4 4.3 4.3s.8 4.2 0 6.1l0 0zM197.6 418l-6.1 14.8c-.4 1-1 1.9-1.7 2.6s-1.6 1.3-2.6 1.7-2 .6-3.1 .6-2.1-.2-3.1-.6-1.9-1-2.6-1.7-1.3-1.6-1.7-2.6-.6-2-.6-3.1 .2-2.1 .6-3.1l6.1-14.8c.4-1 1-1.9 1.7-2.7s1.6-1.4 2.6-1.8 2-.6 3.1-.6 2.1 .2 3.1 .6 1.9 1 2.6 1.8 1.3 1.7 1.7 2.6 .6 2 .6 3.1-.2 2.1-.7 3.1zM264 444c0 2.1-.8 4.2-2.3 5.7s-3.5 2.3-5.7 2.3-4.2-.8-5.7-2.3-2.3-3.5-2.3-5.7l0-16c0-2.1 .8-4.2 2.3-5.7s3.5-2.3 5.7-2.3 4.2 .8 5.7 2.3 2.3 3.5 2.3 5.7l0 16zm67-6.9c-1 .4-2 .6-3.1 .6s-2.1-.2-3.1-.6-1.9-1-2.6-1.7-1.3-1.6-1.7-2.6L314.4 418c-.8-2-.8-4.2 0-6.1s2.4-3.5 4.3-4.3 4.2-.8 6.1 0 3.5 2.4 4.3 4.3l6.1 14.8c.8 2 .8 4.2 0 6.1s-2.4 3.5-4.3 4.3zm63.6-42.5c-1.5 1.5-3.5 2.3-5.7 2.3s-4.2-.8-5.7-2.3L372 383.3c-1.5-1.5-2.3-3.5-2.3-5.6s.8-4.1 2.3-5.6 3.5-2.3 5.7-2.3 4.2 .8 5.7 2.3l11.3 11.3c1.5 1.5 2.3 3.5 2.3 5.6s-.8 4.1-2.3 5.6l0 0zM286.2 286.2L110.3 401.7 225.8 225.8 401.7 110.3 286.2 286.2zM437.1 331c-.8 2-2.4 3.5-4.3 4.3s-4.2 .8-6.1 0l-14.8-6.1c-2-.8-3.5-2.4-4.3-4.3s-.8-4.2 0-6.1c.4-1 1-1.9 1.7-2.6s1.6-1.3 2.6-1.7 2-.6 3.1-.6 2.1 .2 3.1 .6l14.8 6.1c2 .8 3.5 2.4 4.3 4.3s.8 4.2 0 6.1zm6.9-67l-16 0c-2.1 0-4.2-.8-5.7-2.3s-2.3-3.5-2.3-5.7 .8-4.2 2.3-5.7 3.5-2.3 5.7-2.3l16 0c2.1 0 4.2 .8 5.7 2.3s2.3 3.5 2.3 5.7-.8 4.2-2.3 5.7-3.5 2.3-5.7 2.3z"/></svg></a><a class="abx-platform abx-platform-firefox" href="https://addons.mozilla.org/firefox/addon/archivebox-exporter/" aria-label="ArchiveBox Browser Extension for Firefox" title="ArchiveBox Browser Extension for Firefox"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512Line truncated
+      <div class="abx-app-row abx-plugin-row"><a class="abx-app-name" href="https://plugins.archivebox.io/">ArchiveBox Plugin Library</a><div class="abx-platforms abx-icon-stack"><a class="abx-platform abx-integration" href="https://github.com/ArchiveBox/ArchiveBox/wiki/Setting-Up-Storage#amazon-s3--backblaze-b2--google-drive--etc-rclone" aria-label="Google Drive storage" title="Google Drive storage"><svg aria-hidden="true" focusable="false" xmlns="http://www.w3.org/2000/svg" xml:space="preserve" viewBox="0 0 512 512"><path d="m38.7 419.3 22.6 39c4.7 8.2 11.4 14.7 19.4 19.4l80.6-139.6H0c0 9.1 2.3 18.2 7 26.4z" style="fill:#0066da"/><path d="M256 173.9 175.4 34.3c-7.9 4.7-14.7 11.1-19.4 19.4L7 311.7c-4.6 8-7 17.1-7 26.4h161.3z" style="fill:#00ac47"/><path d="M431.4 477.7c7.9-4.7 14.7-11.1 19.4-19.4l9.4-16.1 44.9-77.7c4.7-8.2 7-17.3 7-26.4H350.7l34.3 67.4z" style="fill:#ea4335"/><path d="m256 173.9 80.6-139.6c-7.9-4.7-17-7-26.4-7H201.8c-9.4 0-18.5 2.6-26.4 7z" style="fill:#00832d"/><path d="M350.7 338.1H161.3L80.6 477.7c7.9 4.7 17 7 26.4 7h298c9.4 0 18.5-2.6 26.4-7z" style="fill:#2684fc"/><path d="M430.5 182.7 356 53.7c-4.7-8.2-11.4-14.7-19.4-19.4L256 173.9l94.7 164.2h161c0-9.1-2.3-18.2-7-26.4z" style="fill:#ffba00"/></svg></a><a class="abx-platform abx-integration" href="https://github.com/ArchiveBox/ArchiveBox/wiki/Setting-Up-Storage#amazon-s3--backblaze-b2--google-drive--etc-rclone" aria-label="Amazon S3 storage" title="Amazon S3 storage"><svg aria-hidden="true" focusable="false" fill="#569A31" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path d="M20.913 13.147l.12-.895c.947.576 1.258.922 1.354 1.071-.16.031-.562.046-1.474-.176zm-2.174 7.988a.547.547 0 0 0-.005.073c0 .084-.207.405-1.124.768a10.28 10.28 0 0 1-1.438.432c-1.405.325-3.128.504-4.853.504-4.612 0-7.412-1.184-7.412-1.704a.547.547 0 0 0-.005-.073L1.81 5.602c.135.078.28.154.432.227.042.02.086.038.128.057.134.062.272.122.417.18l.179.069c.154.058.314.114.478.168.043.013.084.029.13.043.207.065.423.127.646.187l.176.044c.175.044.353.087.534.127a23.414 23.414 0 0 0 .843.17l.121.023c.252.045.508.085.768.122.071.011.144.02.216.03.2.027.4.053.604.077l.24.027c.245.026.49.05.74.07l.081.009c.275.022.552.04.83.056l.233.012c.21.01.422.018.633.025a33.088 33.088 0 0 0 2.795-.026l.232-.011c.278-.016.555-.034.83-.056l.08-.008c.25-.02.497-.045.742-.072l.238-.026c.205-.024.408-.05.609-.077.07-.01.141-.019.211-.03.261-.037.519-.078.772-.122l.111-.02c.215-.04.427-.082.634-.125l.212-.047c.186-.041.368-.085.546-.13l.166-.042c.225-.06.444-.122.654-.189.04-.012.077-.026.115-.038a10.6 10.6 0 0 0 .493-.173c.058-.021.114-.044.17-.066.15-.06.293-.12.43-.185.038-.017.079-.034.116-.052.153-.073.3-.15.436-.228l-.976 7.245c-2.488-.78-5.805-2.292-7.311-3a1.09 1.09 0 0 0-1.088-1.085c-.6 0-1.088.489-1.088 1.088 0 .6.488 1.089 1.088 1.089.196 0 .378-.056.537-.148 1.72.812 5.144 2.367 7.715 3.15zm-7.42-20.047c5.677 0 9.676 1.759 9.75 2.736l-.014.113c-.01.033-.031.067-.048.101-.015.028-.026.057-.047.087-.024.033-.058.068-.09.102-.028.03-.051.06-.084.09-.038.035-.087.07-.133.105-.04.03-.074.06-.119.091-.053.036-.116.071-.177.107-.05.03-.095.06-.15.09-.068.036-.147.073-.222.11-.059.028-.114.057-.177.085-.084.038-.177.074-.268.111-.068.027-.13.054-.203.082-.097.036-.205.072-.31.107-.075.026-.148.053-.228.079-.111.035-.233.069-.35.103-.085.024-.165.05-.253.073-.124.034-.258.065-.389.098-.093.022-.181.046-.278.068-.139.032-.287.061-.433.091-.098.02-.191.041-.293.06-.155.03-.32.057-.482.084-.1.018-.198.036-.302.052-.166.026-.342.048-.515.072-.11.014-.213.03-.325.044-.181.023-.372.041-.56.06-.11.012-.218.025-.332.036-.188.016-.386.029-.58.043-.122.009-.24.02-.364.028-.207.012-.422.02-.635.028-.12.005-.234.012-.354.016a35.605 35.605 0 0 1-2.069 0c-.12-.004-.234-.011-.352-.016-.214-.008-.43-.016-.637-.028-.122-.008-.238-.02-.36-.027-.195-.015-.394-.028-.584-.044-.11-.01-.215-.024-.324-.035-.19-.02-.384-.038-.568-.06l-.315-.044c-.176-.024-.355-.046-.525-.073-.1-.015-.192-.033-.29-.05-.167-.028-.335-.055-.494-.086-.096-.018-.183-.038-.276-.056-.151-.032-.305-.062-.45-.095-.09-.02-.173-.043-.26-.064-.138-.034-.277-.067-.407-.102-.082-.022-.157-.046-.235-.069a11.75 11.75 0 0 1-.368-.108c-.075-.024-.141-.049-.213-.073-.11-.037-.223-.075-.325-.113-.067-.025-.125-.051-.188-.077-.096-.038-.195-.076-.282-.115-.06-.027-.11-.054-.166-.08-.08-.039-.162-.077-.233-.116-.052-.028-.094-.055-.142-.084-.063-.038-.13-.075-.185-.113-.043-.029-.075-.058-.113-.086-.048-.037-.098-.073-.139-.11-.032-.029-.054-.057-.08-.087-.033-.035-.069-.07-.093-.104-.02-.03-.031-.058-.046-.086-.018-.035-.039-.068-.049-.102l-.015-.113c.076-.977 4.074-2.736 9.748-2.736zm12.182 12.124c-.118-.628-.84-1.291-2.31-2.128l.963-7.16a.531.531 0 0 0 .005-.073C22.16 1.581 16.447 0 11.32 0 6.194 0 .482 1.581.482 3.851a.58.58 0 0 0 .005.072L2.819 21.25c.071 2.002 5.236 2.75 8.5 2.75 1.805 0 3.615-.188 5.098-.531.598-.138 1.133-.3 1.592-.48 1.18-.467 1.789-1.053 1.813-1.739l.945-7.018c.557.131 1.016.197 1.389.197.54 0 .902-.137 1.134-.413aLine truncated
+      <div class="abx-app-row abx-tools-row"><a class="abx-app-name" href="https://abx-dl.archivebox.io/">Tools</a><div class="abx-platforms"><a class="abx-tool" href="https://abx-dl.archivebox.io/"><span>One-shot <code>abx-dl</code> CLI</span></a><a class="abx-tool" href="https://github.com/ArchiveBox/ArchiveBox/tree/dev/archivebox/mcp">MCP</a><a class="abx-tool" href="https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#python-api-usage">API</a><a class="abx-tool" href="https://web.archivebox.io">Demo</a></div></div>
+    </div></details>
+    __SITE_NAV__
+  </nav>
+</header>

.github/pages/base/marquee.css

@@ -0,0 +1,9 @@
+/* Identical standalone gallery strip across ArchiveBox product sites. */
+.abx-marquee { min-width: 0; width: 100%; max-width: 100%; margin: 24px 0 40px; padding: 0; overflow: hidden; }
+.abx-marquee .abx-marquee-viewport { width: 100%; overflow-x: auto; scroll-behavior: auto; overscroll-behavior-x: contain; scrollbar-width: thin; padding: 8px 20px 12px; box-sizing: border-box; }
+.abx-marquee .abx-marquee-track { display: flex; align-items: flex-start; gap: 24px; width: max-content; margin-inline: auto; }
+.abx-marquee .abx-marquee-card { display: block; flex: none; width: min-content; text-decoration: none; color: #533344; }
+.abx-marquee .abx-marquee-card img { display: block; height: clamp(240px, 30vw, 400px); width: auto; max-width: none; max-height: none; object-fit: contain; border: 0; border-radius: 10px; background: transparent; padding: 0; margin: 0; }
+.abx-marquee :focus-visible { outline: 2px solid #9b2854; outline-offset: 3px; }
+.abx-marquee .abx-marquee-viewport:focus-visible { outline-offset: -3px; }
+@media (max-width: 600px) { .abx-marquee .abx-marquee-track { gap: 16px; } .abx-marquee .abx-marquee-card img { height: 240px; } }

.github/pages/base/marquee.js

@@ -0,0 +1,44 @@
+/* Scroll real gallery links once; keep them usable without JavaScript. */
+(() => {
+  document.querySelectorAll('.abx-marquee').forEach(gallery => {
+    const viewport = gallery.querySelector('.abx-marquee-viewport');
+    const reducedMotion = window.matchMedia('(prefers-reduced-motion: reduce)');
+    let paused = reducedMotion.matches;
+    let hovering = false;
+    let focused = false;
+    let visible = false;
+    let frame;
+    let previous;
+    let offset = viewport.scrollLeft;
+    function tick(time) {
+      const distance = viewport.scrollWidth - viewport.clientWidth;
+      if (distance <= 0) return;
+      if (previous !== undefined) {
+        offset = Math.min(offset + Math.min(time - previous, 50) * 0.035, distance);
+        viewport.scrollTo({ left: offset, behavior: 'instant' });
+      }
+      if (offset >= distance - 1) { paused = true; return; }
+      previous = time;
+      frame = window.requestAnimationFrame(tick);
+    }
+    function sync() {
+      window.cancelAnimationFrame(frame);
+      previous = undefined;
+      offset = viewport.scrollLeft;
+      if (!paused && !hovering && !focused && visible && !document.hidden) frame = window.requestAnimationFrame(tick);
+    }
+    viewport.addEventListener('pointerenter', event => { if (event.pointerType === 'mouse') { hovering = true; sync(); } });
+    viewport.addEventListener('pointerleave', () => { hovering = false; sync(); });
+    viewport.addEventListener('focusin', () => { focused = true; sync(); });
+    viewport.addEventListener('focusout', event => { focused = viewport.contains(event.relatedTarget); sync(); });
+    function pauseForInteraction() { paused = true; sync(); }
+    viewport.addEventListener('pointerdown', pauseForInteraction);
+    viewport.addEventListener('wheel', pauseForInteraction, { passive: true });
+    viewport.addEventListener('keydown', pauseForInteraction);
+    reducedMotion.addEventListener('change', () => { paused = reducedMotion.matches; sync(); });
+    document.addEventListener('visibilitychange', sync);
+    new window.IntersectionObserver(([entry]) => { visible = entry.isIntersecting; sync(); }).observe(gallery);
+    new window.ResizeObserver(sync).observe(viewport);
+    viewport.querySelectorAll('img').forEach(image => image.addEventListener('load', sync));
+  });
+})();

.github/pages/marquee.py

@@ -0,0 +1,94 @@
+"""Render homepage screenshot strips from the site's existing gallery HTML."""
+
+import re
+from html import escape
+from html.parser import HTMLParser
+from urllib.parse import urljoin
+
+
+class Gallery(HTMLParser):
+    """Only gallery figures are captures; navigation and decorative images are not."""
+
+    def __init__(self, base):
+        super().__init__()
+        self.base = base
+        self.stack = []
+        self.captures = []
+
+    def handle_starttag(self, tag, attrs):
+        attrs = dict(attrs)
+        if tag == "img" and any(name == "figure" for name, _ in self.stack):
+            context = [item for _, item in self.stack]
+            profile = next(
+                (
+                    item.get("data-profile") or item.get("data-viewport")
+                    for item in reversed(context)
+                    if item.get("data-profile") or item.get("data-viewport")
+                ),
+                None,
+            )
+            if profile not in (None, "desktop"):
+                return
+            source = attrs.get("src", "")
+            if not source or not attrs.get("alt"):
+                return
+            anchor = next((item["id"] for name, item in reversed(self.stack) if name in ("article", "figure") and item.get("id")), "")
+            server = any(item.get("data-platform") == "server" for item in context)
+            self.captures.append(
+                {
+                    "src": urljoin(self.base, source),
+                    "href": self.base + "#" + anchor if anchor else urljoin(self.base, source),
+                    "title": attrs["alt"],
+                    "width": attrs.get("width", ""),
+                    "height": attrs.get("height", ""),
+                    "product": "server" if server else "client",
+                },
+            )
+        if tag not in {"area", "base", "br", "col", "embed", "hr", "img", "input", "link", "meta", "param", "source", "track", "wbr"}:
+            self.stack.append((tag, attrs))
+
+    def handle_endtag(self, tag):
+        for index in range(len(self.stack) - 1, -1, -1):
+            if self.stack[index][0] == tag:
+                del self.stack[index:]
+                break
+
+
+def render(output, baseurl=""):
+    homepage = output / "index.html"
+    text = homepage.read_text()
+    slots = re.findall(r"<!-- ARCHIVEBOX:(MARQUEE(?:-CLIENT|-SERVER)?) -->", text)
+    if not slots:
+        return False
+    base = "/" + baseurl.strip("/") + "/" if baseurl.strip("/") else "/"
+    gallery = Gallery(base + "screenshots/")
+    gallery.feed((output / "screenshots/index.html").read_text())
+    for slot in slots:
+        product = slot.removeprefix("MARQUEE-").lower() if "-" in slot else None
+        captures = [capture for capture in gallery.captures if product is None or capture["product"] == product]
+        cards = []
+        for capture in captures:
+            dimensions = "".join(f' {key}="{escape(capture[key])}"' for key in ("width", "height") if capture[key])
+            cards.append(
+                f'<a class="abx-marquee-card" href="{escape(capture["href"])}"><img src="{escape(capture["src"])}" alt="{escape(capture["title"])}"{dimensions} loading="lazy" decoding="async"></a>',
+            )
+        viewport_id = slot.lower()
+        content = (
+            f'<section class="abx-marquee" aria-label="App screenshots"><div class="abx-marquee-viewport" id="{viewport_id}" tabindex="0" aria-label="Screenshots. Scroll sideways to explore."><div class="abx-marquee-track">{"".join(cards)}</div></div></section>'
+            if cards
+            else ""
+        )
+        start, end = f"<!-- ARCHIVEBOX:{slot} -->", f"<!-- /ARCHIVEBOX:{slot} -->"
+        replacement = start + "\n" + content + "\n" + end
+        if end in text:
+            text = re.sub(
+                re.escape(start) + r".*?" + re.escape(end),
+                lambda _, replacement=replacement: replacement,
+                text,
+                count=1,
+                flags=re.DOTALL,
+            )
+        else:
+            text = text.replace(start, replacement, 1)
+    homepage.write_text(text)
+    return True

.github/pages/nav.html

@@ -0,0 +1,4 @@
+<!-- Repository-specific navigation and primary CTA. -->
+<a href="__SITE_BASE__#use-cases">Use cases</a><a href="__SITE_BASE__#configuration">Configuration</a><a href="__SITE_BASE__#layout-security">Layout &amp; security</a><a href="https://github.com/ArchiveBox/ArchiveBox/wiki">Docs</a>
+    <a href="https://github.com/ArchiveBox/ArchiveBox">GitHub ↗</a>
+    <a class="abx-cta" href="__SITE_BASE__#install">Get started</a>

.github/pages/site.json

@@ -0,0 +1,8 @@
+{
+  "name": "Web Archive",
+  "url": "https://archivebox.io/",
+  "pages": [
+    "index.html",
+    "screenshots/index.html"
+  ]
+}

.github/pages/site.py

@@ -0,0 +1,215 @@
+"""Assemble and verify this repository's static Pages site (standard library only)."""
+
+import argparse
+import json
+import re
+import shutil
+import subprocess
+from datetime import UTC, datetime
+from html import escape, unescape
+from html.parser import HTMLParser
+from pathlib import Path
+from urllib.parse import unquote, urlsplit
+
+from marquee import render as render_marquee
+
+HERE = Path(__file__).resolve().parent
+ROOT = HERE.parents[1]
+
+
+def replace_block(text, name, content):
+    start, end = f"<!-- ARCHIVEBOX:{name} -->", f"<!-- /ARCHIVEBOX:{name} -->"
+    pattern = re.escape(start) + r".*?" + re.escape(end)
+    if re.search(pattern, text, re.DOTALL):
+        return re.sub(
+            pattern,
+            lambda _: f"{start}\n{content}\n{end}",
+            text,
+            count=1,
+            flags=re.DOTALL,
+        )
+    if text.count(start) != 1:
+        raise ValueError(f"Expected one {name} placeholder")
+    return text.replace(start, f"{start}\n{content}\n{end}", 1)
+
+
+def render_metadata(text):
+    """Keep product copy in native templates; render one consistent SEO block."""
+    page = Page()
+    page.feed(text)
+    title = unescape(re.search(r"<title>(.*?)</title>", text, re.DOTALL)[1])
+    meta = page.metadata
+    description = meta.get("description", meta.get("og:description", ""))
+    image = meta.get("og:image", meta.get("twitter:image", ""))
+    if not description or not image or not page.canonical:
+        raise ValueError("Each page needs a description, social image and canonical")
+    values = {
+        "description": description,
+        "robots": "index,follow,max-image-preview:large",
+        "theme-color": "#9b2854",
+        "og:type": "website",
+        "og:site_name": meta.get("og:site_name", "ArchiveBox"),
+        "og:locale": meta.get("og:locale", "en_US"),
+        "og:url": page.canonical,
+        "og:title": title,
+        "og:description": description,
+        "og:image": image,
+        "og:image:type": meta.get("og:image:type", "image/png"),
+        "og:image:width": meta.get("og:image:width", "1200"),
+        "og:image:height": meta.get("og:image:height", "630"),
+        "og:image:alt": meta.get("og:image:alt", title),
+        "twitter:card": "summary_large_image",
+        "twitter:site": "@ArchiveBoxApp",
+        "twitter:title": title,
+        "twitter:description": description,
+        "twitter:image": image,
+        "twitter:image:alt": meta.get("og:image:alt", title),
+    }
+
+    def strip_managed_tag(match):
+        tag = Page()
+        tag.feed(match[0])
+        if tag.canonicals or any(key in values for key in tag.metadata):
+            return ""
+        # System fonts are shared across sites, without a network font dependency.
+        if "fonts.googleapis.com" in match[0] or "fonts.gstatic.com" in match[0]:
+            return ""
+        return match[0]
+
+    text = re.sub(r"<(?:meta|link)\b[^>]*>", strip_managed_tag, text)
+    text = re.sub(r"<title>.*?</title>", "", text, flags=re.DOTALL)
+    tags = [
+        f"<title>{escape(title)}</title>",
+        f'<link rel="canonical" href="{escape(page.canonical, quote=True)}">',
+    ]
+    for key, value in values.items():
+        attr = "property" if key.startswith("og:") else "name"
+        tags.append(f'<meta {attr}="{key}" content="{escape(value, quote=True)}">')
+    if "<!-- ARCHIVEBOX:METADATA -->" not in text:
+        text = text.replace("</head>", "<!-- ARCHIVEBOX:METADATA -->\n</head>", 1)
+    return replace_block(text, "METADATA", "\n".join(tags))
+
+
+def render(output, source=None, baseurl=""):
+    config = json.loads((HERE / "site.json").read_text())
+    output = output.resolve()
+    if source:
+        source = source.resolve()
+        if output == source or output in source.parents or source in output.parents:
+            raise ValueError("Build output must be separate from the source directory")
+        if output == ROOT or output in ROOT.parents:
+            raise ValueError("Build output must not replace a repository")
+        if output.exists():
+            shutil.rmtree(output)
+        shutil.copytree(source, output)
+    revision = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=ROOT, text=True).strip()
+    assets = output / "site-base"
+    assets.mkdir(parents=True, exist_ok=True)
+    has_marquee = render_marquee(output, baseurl)
+    common_assets = ["chrome.css", "chrome.js"] + (["marquee.css", "marquee.js"] if has_marquee else [])
+    for name in common_assets:
+        shutil.copyfile(HERE / "base" / name, assets / name)
+    for name in config["pages"]:
+        page = output / name
+        text = render_metadata(page.read_text())
+        base = "../" * (len(Path(name).parts) - 1) or "./"
+        header = (HERE / "base/header.html").read_text().replace("__SITE_NAME__", escape(config["name"]))
+        header = header.replace("__SITE_NAV__", (HERE / "nav.html").read_text().strip())
+        text = replace_block(text, "HEADER", header)
+        for kind in ["FOOTER-START", "FOOTER-END"]:
+            text = replace_block(text, kind, (HERE / "base" / f"{kind.lower()}.html").read_text().strip())
+        text = text.replace("__SITE_BASE__", base).replace("__SITE_REVISION__", revision)
+        resources = f'<link rel="stylesheet" href="{base}site-base/chrome.css?v={revision}">\n<script src="{base}site-base/chrome.js?v={revision}" defer></script>'
+        if has_marquee and name == "index.html":
+            resources += f'\n<link rel="stylesheet" href="{base}site-base/marquee.css?v={revision}">\n<script src="{base}site-base/marquee.js?v={revision}" defer></script>'
+        if "<!-- ARCHIVEBOX:ASSETS -->" not in text:
+            text = text.replace("</head>", "<!-- ARCHIVEBOX:ASSETS -->\n</head>", 1)
+        text = replace_block(text, "ASSETS", resources)
+        page.write_text(text)
+    # Site build provenance is independent of all original screenshot manifests.
+    provenance_path = output / "build.json"
+    provenance = json.loads(provenance_path.read_text()) if provenance_path.exists() else {}
+    provenance.update(
+        revision=revision,
+        generatedAt=datetime.now(UTC).isoformat(),
+        site=config["url"],
+    )
+    provenance_path.write_text(json.dumps(provenance, indent=2) + "\n")
+    (output / ".nojekyll").touch()
+    verify(output, baseurl)
+    print(f"Assembled {len(config['pages'])} pages at {output} from {revision}")
+
+
+class Page(HTMLParser):
+    def __init__(self):
+        super().__init__()
+        self.ids = set()
+        self.metadata = {}
+        self.canonical = ""
+        self.resources = []
+        self.headers = self.footers = self.canonicals = 0
+
+    def handle_starttag(self, tag, attrs):
+        attrs = dict(attrs)
+        if tag == "meta":
+            for key in ["name", "property"]:
+                if attrs.get(key):
+                    self.metadata[attrs[key]] = attrs.get("content", "")
+        if tag == "link" and attrs.get("rel") == "canonical":
+            self.canonical = attrs["href"]
+        if attrs.get("id"):
+            self.ids.add(attrs["id"])
+        classes = attrs.get("class", "").split()
+        self.headers += "abx-header" in classes
+        self.footers += "abx-footer" in classes
+        self.canonicals += tag == "link" and attrs.get("rel") == "canonical"
+        if tag in ["img", "script"] and attrs.get("src"):
+            self.resources.append(attrs["src"])
+        if tag == "link" and attrs.get("rel") in [
+            "stylesheet",
+            "icon",
+            "apple-touch-icon",
+        ]:
+            self.resources.append(attrs["href"])
+
+
+def verify(output, baseurl=""):
+    config = json.loads((HERE / "site.json").read_text())
+    output = output.resolve()
+    prefix = "/" + baseurl.strip("/") if baseurl.strip("/") else ""
+    for name in config["pages"]:
+        path = output / name
+        text = path.read_text()
+        page = Page()
+        page.feed(text)
+        if (page.headers, page.footers, page.canonicals) != (1, 1, 1):
+            raise ValueError(f"{name}: expected exactly one shared header, footer and canonical")
+        if "__SITE_" in text:
+            raise ValueError(f"{name}: unresolved site placeholder")
+        for resource in page.resources:
+            url = urlsplit(resource)
+            if url.scheme or url.netloc or not url.path:
+                continue
+            resource_path = unquote(url.path)
+            if prefix and resource_path.startswith(prefix + "/"):
+                resource_path = resource_path[len(prefix) :]
+            local = (output / resource_path.lstrip("/") if resource_path.startswith("/") else path.parent / resource_path).resolve()
+            if not local.is_relative_to(output) or not local.is_file():
+                raise ValueError(f"{name}: missing local resource {resource}")
+        for asset in ["chrome.css", "chrome.js"]:
+            if (output / "site-base" / asset).read_bytes() != (HERE / "base" / asset).read_bytes():
+                raise ValueError(f"{name}: stale common asset {asset}")
+    print(f"Verified shared layout and local resources for {len(config['pages'])} pages")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("command", choices=["render", "verify"])
+    parser.add_argument("output", type=Path)
+    parser.add_argument("--source", type=Path)
+    parser.add_argument("--baseurl", default="")
+    args = parser.parse_args()
+    if args.command == "render":
+        render(args.output, args.source, args.baseurl)
+    else:
+        verify(args.output, args.baseurl)

.github/pages/verify.py

@@ -0,0 +1,199 @@
+"""Check the built site in real Chromium at desktop and phone widths."""
+
+import argparse
+import json
+from functools import partial
+from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
+from pathlib import Path
+from threading import Thread
+from urllib.parse import urljoin, urlsplit
+
+from playwright.sync_api import expect, sync_playwright
+
+HERE = Path(__file__).resolve().parent
+
+
+class Handler(SimpleHTTPRequestHandler):
+    def handle(self):
+        # Browser navigation can cancel an in-flight lazy image response.
+        try:
+            super().handle()
+        except (BrokenPipeError, ConnectionResetError):
+            pass
+
+    def log_message(self, *args):
+        pass
+
+
+def verify(output, evidence):
+    config = json.loads((HERE / "site.json").read_text())
+    evidence.mkdir(parents=True, exist_ok=True)
+    server = ThreadingHTTPServer(("127.0.0.1", 0), partial(Handler, directory=str(output.resolve())))
+    thread = Thread(target=server.serve_forever, daemon=True)
+    thread.start()
+    origin = f"http://127.0.0.1:{server.server_port}"
+    try:
+        with sync_playwright() as playwright:
+            browser = playwright.chromium.launch()
+            for name in config["pages"]:
+                route = name.removesuffix("index.html")
+                page = browser.new_page(reduced_motion="reduce")
+                errors = []
+                missing = []
+                page.on("pageerror", lambda error, errors=errors: errors.append(str(error)))
+                page.on(
+                    "response",
+                    lambda response, missing=missing: (
+                        missing.append(response.url) if response.url.startswith(origin) and response.status >= 400 else None
+                    ),
+                )
+                for width in [1440, 390]:
+                    page.set_viewport_size({"width": width, "height": 900})
+                    page.goto(f"{origin}/{route}", wait_until="domcontentloaded")
+                    header = page.locator(".abx-header")
+                    expect(header).to_have_count(1)
+                    expect(header).to_be_visible()
+                    expect(header).to_be_in_viewport()
+                    brand = header.locator(".abx-brand")
+                    expect(brand).to_be_visible()
+                    expect(brand).to_contain_text("ArchiveBox")
+                    expect(brand.locator(".abx-logo")).to_be_visible()
+                    expect(brand.locator(".abx-logo")).to_be_in_viewport()
+                    expect(header).to_have_css("display", "flex")
+                    expect(page.locator(".abx-footer")).to_have_count(1)
+                    expect(page.locator(".abx-footer-column")).to_have_count(3)
+                    expect(header.locator(".abx-cta")).to_be_visible()
+                    menu = header.locator(".abx-apps")
+                    menu.locator("summary").focus()
+                    page.keyboard.press("Enter")
+                    expect(menu).to_have_attribute("open", "")
+                    expect(menu.locator("a").first).to_be_visible()
+                    page.keyboard.press("Escape")
+                    expect(menu).not_to_have_attribute("open", "")
+                    menu.locator("summary").click()
+                    box = header.bounding_box()
+                    assert box
+                    page.mouse.click(box["x"] + 2, box["y"] + 5)
+                    expect(menu).not_to_have_attribute("open", "")
+                    assert page.evaluate("document.documentElement.scrollWidth <= innerWidth"), f"{name}: overflow at {width}"
+                    if not route and "screenshots/index.html" in config["pages"]:
+                        strips = page.locator(".abx-marquee")
+                        assert strips.count() > 0, "Homepage must use its screenshot gallery"
+                        for strip in strips.all():
+                            strip.scroll_into_view_if_needed()
+                            assert not strip.inner_text().strip(), "Screenshot strips should have no visible captions or links"
+                            cards = strip.locator(".abx-marquee-card")
+                            assert cards.count() > 0, "Screenshot strip is empty"
+                            expect(strip.locator("button")).to_have_count(0)
+                            expect(strip.locator("a:not(.abx-marquee-card)")).to_have_count(0)
+                            viewport = strip.locator(".abx-marquee-viewport")
+                            overflowing = viewport.evaluate(
+                                "node => node.scrollWidth > node.clientWidth",
+                            )
+                            if overflowing:
+                                page.mouse.move(0, 0)
+                                page.emulate_media(reduced_motion="no-preference")
+                                initial = viewport.evaluate("node => node.scrollLeft")
+                                page.wait_for_function(
+                                    "([node, initial]) => node.scrollLeft > initial",
+                                    arg=[viewport.element_handle(), initial],
+                                )
+                                viewport.hover()
+                                stopped = viewport.evaluate("node => node.scrollLeft")
+                                viewport.evaluate(
+                                    "node => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)))",
+                                )
+                                assert viewport.evaluate("node => node.scrollLeft") == stopped
+                                page.emulate_media(reduced_motion="reduce")
+                                page.mouse.move(0, 0)
+                                stopped = viewport.evaluate("node => node.scrollLeft")
+                                viewport.evaluate(
+                                    "node => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve)))",
+                                )
+                                assert viewport.evaluate("node => node.scrollLeft") == stopped
+                                viewport.focus()
+                                page.keyboard.press("ArrowRight")
+                                assert viewport.evaluate("node => node.scrollWidth > node.clientWidth")
+                            else:
+                                centered = viewport.evaluate(
+                                    """node => {
+                                        const view = node.getBoundingClientRect();
+                                        const track = node.querySelector('.abx-marquee-track').getBoundingClientRect();
+                                        return Math.abs((track.left - view.left) - (view.right - track.right)) < 2;
+                                    }""",
+                                )
+                                assert centered, "Short screenshot strips must be centered"
+                            for card in cards.all():
+                                href = card.get_attribute("href")
+                                assert href and "/screenshots/" in href
+                                response = page.request.get(urljoin(page.url, href))
+                                assert response.ok, f"Broken screenshot link: {href}"
+                                anchor = urlsplit(href).fragment
+                                if anchor:
+                                    assert f'id="{anchor}"' in response.text(), f"Missing screenshot anchor: {href}"
+                    # Decode every local content image using the real browser loader,
+                    # including images in offscreen carousel tracks and lazy galleries.
+                    images = page.locator("main img").evaluate_all("nodes => [...new Set(nodes.map(image => image.src))]")
+                    local_images = [url for url in images if url.startswith(origin + "/")]
+                    page.evaluate(
+                        """async urls => {
+                        for (const url of urls) {
+                            const image = new Image();
+                            image.src = url;
+                            await image.decode();
+                            if (!image.naturalWidth) throw new Error(`Broken image: ${url}`);
+                        }
+                    }""",
+                        local_images,
+                    )
+                    page.evaluate("scrollTo({top: 0, behavior: 'instant'})")
+                    page.screenshot(path=str(evidence / f"{route.replace('/', '-') or 'home-'}{width}.png"))
+                    page.locator(".abx-footer").screenshot(path=str(evidence / f"{route.replace('/', '-') or 'home-'}footer-{width}.png"))
+                    for link in header.locator(".abx-nav > a").all():
+                        url = urlsplit(link.get_attribute("href") or "")
+                        if url.scheme or url.netloc:
+                            continue
+                        response = page.request.get(urljoin(page.url, url.geturl()))
+                        assert response.ok, f"Broken navigation: {url.geturl()}"
+                        if url.fragment:
+                            body = response.text()
+                            assert f'id="{url.fragment}"' in body or f"id='{url.fragment}'" in body, (
+                                f"Missing navigation anchor: {url.geturl()}"
+                            )
+                    if route == "" and page.locator("#resources").count():
+                        page.goto(f"{origin}/#resources")
+                        expect(page.locator("#resources")).to_be_visible()
+                assert not errors, errors
+                assert not missing, missing
+                page.close()
+                context = browser.new_context(java_script_enabled=False, viewport={"width": 390, "height": 900})
+                plain = context.new_page()
+                plain.goto(f"{origin}/{route}", wait_until="domcontentloaded")
+                expect(plain.locator(".abx-header")).to_be_visible()
+                expect(plain.locator(".abx-header .abx-brand")).to_be_in_viewport()
+                expect(plain.locator(".abx-header .abx-logo")).to_be_visible()
+                plain.locator(".abx-apps summary").click()
+                expect(plain.locator(".abx-app-links a").first).to_be_visible()
+                expect(plain.locator(".abx-footer-column a").first).to_be_visible()
+                if not route and "screenshots/index.html" in config["pages"]:
+                    assert plain.locator(".abx-marquee-card").count() > 0
+                    expect(plain.locator(".abx-marquee button")).to_have_count(0)
+                    expect(plain.locator(".abx-marquee-card").first).to_be_visible()
+                context.close()
+                print(
+                    f"PASS {name}: desktop/mobile, keyboard, no-JS, images and resources",
+                    flush=True,
+                )
+            browser.close()
+    finally:
+        server.shutdown()
+        server.server_close()
+        thread.join()
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("output", type=Path)
+    parser.add_argument("--evidence", type=Path, default=Path("test-results/site"))
+    args = parser.parse_args()
+    verify(args.output, args.evidence)

.github/workflows/deploy-publicsite.yml

@@ -1,4 +1,5 @@
-name: Deploy Publicsite to GitHub Pages
+name: Publish website
+
 on:
   push:
     branches: [dev]
@@ -8,37 +9,73 @@ on:
     workflows: ['Capture publicsite screenshots']
     types: [completed]
     branches: [dev]
+
 permissions:
   contents: read
   actions: read
+
+env:
+  # Repository-specific output; all publication logic below is a local copy.
+  SITE_OUTPUT: _site
+  DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
+
+# One production queue covers restore, build and deploy. PRs have separate queues.
 concurrency:
-  # Failed captures must not displace a pending push deployment.
-  group: fast-pages-${{ github.event_name == 'workflow_run' && github.event.workflow_run.conclusion != 'success' && github.run_id || github.ref }}
+  group: pages-${{ github.event_name == 'pull_request' && github.ref || 'production' }}
   cancel-in-progress: false
+
 jobs:
   build:
-    if: github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success'
     runs-on: ubuntu-latest
-    timeout-minutes: 15
+    timeout-minutes: 20
+    outputs:
+      revision: ${{ steps.source.outputs.revision }}
     steps:
       - uses: actions/checkout@v4
         with:
-          ref: ${{ github.event_name == 'pull_request' && github.ref || 'dev' }}
+          ref: ${{ github.event_name == 'pull_request' && github.ref || env.DEFAULT_BRANCH }}
+          persist-credentials: false
+      - name: Record website source revision
+        id: source
+        run: echo "revision=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT"
+
+      # BEGIN REPOSITORY-SPECIFIC BUILD
       - uses: astral-sh/setup-uv@v7
       - name: Restore latest complete screenshots
-        run: uv run --no-project python bin/restore_site_screenshots.py archivebox publicsite/screenshots
+        run: uv run --no-project python bin/restore_site_screenshots.py publicsite/screenshots
         env:
           GH_TOKEN: ${{ github.token }}
       - name: Render current gallery layout without recapturing images
         run: uv run --no-project python bin/generate_ui_screenshot_gallery.py refresh publicsite/screenshots/index.html
-      - uses: actions/upload-pages-artifact@v3
+      - name: Assemble website with the local base components
+        run: uv run --no-project python .github/pages/site.py render _site --source publicsite
+      # END REPOSITORY-SPECIFIC BUILD
+
+      - name: Verify website in Chromium
+        run: |
+          uv run --no-config --no-project --with playwright==1.63.0 playwright install --with-deps chromium
+          uv run --no-config --no-project --with playwright==1.63.0 python .github/pages/verify.py "$SITE_OUTPUT" --evidence "$RUNNER_TEMP/site-evidence"
+      - name: Save website verification evidence
+        if: always()
+        uses: actions/upload-artifact@v4
         with:
-          path: publicsite
+          name: site-verification
+          path: ${{ runner.temp }}/site-evidence
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Upload website
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: ${{ env.SITE_OUTPUT }}
+
   deploy:
     if: github.event_name != 'pull_request'
     needs: build
     runs-on: ubuntu-latest
+    timeout-minutes: 10
     permissions:
+      contents: read
       pages: write
       id-token: write
     environment:

.github/workflows/docker.yml

@@ -246,6 +246,26 @@ jobs:
         if: inputs.push_digests
         run: echo ${{ steps.docker_build.outputs.digest }}
 
+      - name: Verify OpenCode install with a read-only browser profile
+        shell: bash
+        env:
+          IMAGE: ${{ inputs.push_digests && format('{0}@{1}', env.DOCKERHUB_IMAGE, steps.docker_build.outputs.digest) || matrix.local_tag }}
+        run: |
+          set -Eeuo pipefail
+          PROFILE_DIR="$(mktemp -d)"
+          "$DOCKER_BINARY" run --rm --platform '${{ matrix.platform }}' \
+            --volume "$PROFILE_DIR:/home/archivebox/.config/chromium:ro" \
+            "$IMAGE" bash -c '
+              set -Eeuo pipefail
+              test -w "$XDG_CONFIG_HOME"
+              test ! -w "$XDG_CONFIG_HOME/chromium"
+              # Check the shipped executable before the installer can repair
+              # an image that accidentally omitted this opt-in dependency.
+              "$ABXPKG_LIB_DIR/pnpm/packages/opencode/node_modules/.bin/opencode" --version
+              archivebox install opencode
+              "$ABXPKG_LIB_DIR/pnpm/packages/opencode/node_modules/.bin/opencode" --version
+            '
+
       - name: Validate exact built image version and commit
         if: ${{ !inputs.push_digests }}
         shell: bash
@@ -296,8 +316,8 @@ jobs:
             MANIFEST="$("$DOCKER_BINARY" buildx imagetools inspect --raw "${DOCKERHUB_IMAGE}@${IMAGE_DIGEST}")"
           fi
           COMPRESSED_SIZE="$(MANIFEST="$MANIFEST" "$UV_BINARY" run --no-cache --no-project python -c 'import json, os; manifest = json.loads(os.environ["MANIFEST"]); print(manifest["config"]["size"] + sum(layer["size"] for layer in manifest["layers"]))')"
-          LIMIT=$((750 * 1024 * 1024))
-          echo "${DIGEST} compressed_size=$((COMPRESSED_SIZE / 1024 / 1024)) MiB limit=750 MiB"
+          LIMIT=$((900 * 1024 * 1024))
+          echo "${DIGEST} compressed_size=$((COMPRESSED_SIZE / 1024 / 1024)) MiB limit=900 MiB"
           (( COMPRESSED_SIZE < LIMIT ))
 
       - name: Resolve real Docker CLI test dependencies through abxpkg
@@ -378,6 +398,12 @@ jobs:
           IMAGE="${{ matrix.local_tag }}" \
             "$BASH_BINARY" bin/validate_docker_uid_gid.sh --local-only
 
+      - name: Validate Docker port compatibility
+        shell: bash
+        env:
+          IMAGE: ${{ inputs.push_digests && format('{0}@{1}', env.DOCKERHUB_IMAGE, steps.docker_build.outputs.digest) || matrix.local_tag }}
+        run: '"$BASH_BINARY" bin/validate_docker_ports.sh'
+
       - name: Validate Docker CLI add/list and compose behavior
         if: inputs.full_tests && matrix.platform == 'linux/amd64'
         shell: bash
@@ -401,13 +427,13 @@ jobs:
           "$DOCKER_BINARY" compose run --rm archivebox init
           "$DOCKER_BINARY" compose up -d --wait
           "$DOCKER_BINARY" network connect archivebox_default www-nginx
-          index_html="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:8000/public/')"
-          admin_js="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:8000/static/admin/js/jquery.init.js')"
+          index_html="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:5797/public/')"
+          admin_js="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:5797/static/admin/js/jquery.init.js')"
           [[ "$index_html" == *ArchiveBox* ]]
           [[ "$admin_js" == *window.django* ]]
 
           "$DOCKER_BINARY" compose run --rm archivebox add --plugins=parse_txt_urls 'http://www-nginx/#test_docker'
-          index_html="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:8000/public/')"
+          index_html="$("$CURL_BINARY" --silent --location 'http://127.0.0.1:5797/public/')"
           [[ "$index_html" == *'http://www-nginx/#test_docker'* ]]
           "$DOCKER_BINARY" compose down
 
@@ -423,7 +449,7 @@ jobs:
             '${{ runner.temp }}/archivebox-image.tar' \
             '${{ steps.docker_build_pr.outputs.digest }}' \
             "$LOCAL_IMAGE_ID" \
-            "$((750 * 1024 * 1024))" <<'PY'
+            "$((900 * 1024 * 1024))" <<'PY'
           from hashlib import sha256
           import json
           import os

.github/workflows/release-candidate.yml

@@ -49,3 +49,17 @@ jobs:
       push_digests: true
       full_tests: false
     secrets: inherit
+
+  opencode:
+    needs: prepare
+    if: needs.prepare.outputs.action == 'ready' || needs.prepare.outputs.action == 'reserved'
+    strategy:
+      fail-fast: false
+      matrix:
+        test_file:
+          - archivebox/tests/test_opencode_agent.py
+          - archivebox/tests/test_opencode_browser.py
+    uses: ./.github/workflows/test-parallel.yml
+    with:
+      test_file: ${{ matrix.test_file }}
+    secrets: inherit

.github/workflows/release.yml

@@ -27,8 +27,11 @@ jobs:
       ((github.event.workflow_run.head_branch == 'dev' && github.event.workflow_run.name == 'Release candidate') ||
        (github.event.workflow_run.head_branch == 'main' && github.event.workflow_run.name == 'CI'))
     runs-on: ubuntu-24.04
+    permissions:
+      actions: read
+      contents: read
     outputs:
-      ready: ${{ steps.owner.outputs.ready }}
+      ready: ${{ steps.verified.outputs.ready }}
     steps:
       - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
         with:
@@ -45,6 +48,32 @@ jobs:
           TARGET="$(git ls-remote origin "refs/tags/release-candidate/$VERSION" | awk '{print $1}')"
           [[ "$TARGET" == "$RELEASE_SHA" ]] && echo 'ready=true' >> "$GITHUB_OUTPUT" || echo 'ready=false' >> "$GITHUB_OUTPUT"
 
+      - name: Require successful full CI for this commit
+        id: verified
+        if: steps.owner.outputs.ready == 'true'
+        timeout-minutes: 40
+        env:
+          GH_TOKEN: ${{ github.token }}
+        shell: bash
+        run: |
+          set -Eeuo pipefail
+          CI_RUN_ID=''
+          for _ in {1..12}; do
+            CI_RUN_ID="$(gh run list \
+              --repo "$GITHUB_REPOSITORY" \
+              --workflow ci.yml \
+              --commit "$RELEASE_SHA" \
+              --event push \
+              --limit 1 \
+              --json databaseId \
+              --jq '.[0].databaseId // empty')"
+            [[ -n "$CI_RUN_ID" ]] && break
+            sleep 5
+          done
+          [[ -n "$CI_RUN_ID" ]]
+          gh run watch "$CI_RUN_ID" --repo "$GITHUB_REPOSITORY" --exit-status
+          echo 'ready=true' >> "$GITHUB_OUTPUT"
+
   python-release:
     name: Publish tested Python artifacts
     needs: candidate

.github/workflows/screenshots.yml

@@ -5,7 +5,6 @@ on:
     branches:
       - dev
     paths:
-      - publicsite/**
       - archivebox/**
       - bin/collect_ui_screenshots.*
       - bin/generate_ui_screenshot_gallery.py
@@ -43,7 +42,7 @@ jobs:
       - name: Build UI screenshot gallery
         run: bin/collect_ui_screenshots.sh
         env:
-          BASE_URL: http://archivebox.localhost:8000
+          BASE_URL: http://archivebox.localhost:5797
 
       - name: Verify gallery was generated from this exact revision
         run: |

.github/workflows/test-parallel.yml

@@ -2,6 +2,12 @@ name: Parallel Tests
 
 on:
   workflow_call:
+    inputs:
+      test_file:
+        description: Run one discovered test file; empty runs the complete matrix
+        required: false
+        default: ""
+        type: string
 
 env:
   PYTHONIOENCODING: utf-8
@@ -58,10 +64,15 @@ jobs:
 
       - name: Discover every test exactly once
         id: discover
+        env:
+          TEST_FILE: ${{ inputs.test_file }}
         run: |
           set -Eeuo pipefail
           mapfile -t output < <("$PYTHON_BINARY" .github/scripts/discover_test_matrix.py)
           test "${#output[@]}" -eq 2
+          if [[ -n "$TEST_FILE" ]]; then
+            output[1]="$("$JQ_BINARY" -ce '[.[] | select(.paths == [env.TEST_FILE])] | select(length == 1)' <<< "${output[1]}")"
+          fi
           echo "tests=${output[1]}" >> "$GITHUB_OUTPUT"
           echo "${output[0]}"
           "$JQ_BINARY" -e 'length > 0 and length <= 256 and ([.[].paths[]] | length == (unique | length))' <<< "${output[1]}"

.gitignore

@@ -68,3 +68,5 @@ ArchiveBox.conf
 
 # Local Claude Code task locks (not project state)
 .claude/scheduled_tasks.lock
+
+/_site/

Dockerfile

@@ -1,8 +1,10 @@
 # syntax=docker/dockerfile:1.7
 
 # Multistage ArchiveBox Dockerfile that consumes the abx-dl runtime image.
-# abx-dl owns Python, Node, Chromium, and downloader plugin runtimes.
-# ArchiveBox owns sonic, supervisor, Django, and the app runtime.
+# WHY: abx-dl owns Python, Node, Chromium, and extraction dependencies so the
+# standalone downloader stays independent of ArchiveBox's server features.
+# ArchiveBox-only dependencies (Sonic, OpenCode, supervisor, Django) belong in
+# the layers added by this Dockerfile, never in the shared abx-dl base image.
 # Build abx-dl first, then point this file at it:
 #   docker buildx build ../abx-dl -f ../abx-dl/Dockerfile \
 #       --build-context abxbus=../abxbus \
@@ -10,10 +12,10 @@
 #       --build-context abx-plugins=../abx-plugins \
 #       -t archivebox/abx-dl:dev
 #   docker buildx build . -f Dockerfile \
-#       --build-arg ABX_DL_IMAGE=archivebox/abx-dl:1.12.289 \
+#       --build-arg ABX_DL_IMAGE=archivebox/abx-dl:1.12.342 \
 #       -t archivebox:multistage
 
-ARG ABX_DL_IMAGE=archivebox/abx-dl:1.12.289
+ARG ABX_DL_IMAGE=archivebox/abx-dl:1.12.342
 
 FROM archivebox/sonic:1.4.9 AS sonic
 FROM ${ABX_DL_IMAGE} AS archivebox-runtime-base
@@ -250,6 +252,19 @@ RUN --mount=type=cache,target=/opt/archivebox/lib/cache,sharing=locked,mode=1777
     && echo -e "\nARCHIVEBOX_USER=$ARCHIVEBOX_USER ARCHIVEBOX_UID=$(id -u "$ARCHIVEBOX_USER") ARCHIVEBOX_GID=$(id -g "$ARCHIVEBOX_USER")" | tee -a /VERSION.txt \
     && echo -e "TMP_DIR=$TMP_DIR\nABXPKG_LIB_DIR=$ABXPKG_LIB_DIR\nPLAYWRIGHT_BROWSERS_PATH=$PLAYWRIGHT_BROWSERS_PATH\nMACHINE_ID=$(cat /etc/machine-id)\n" | tee -a /VERSION.txt
 
+# The optional AI service belongs to ArchiveBox, not the downloader base image.
+# Always bundle its executable while leaving OPENCODE_ENABLED opt-in.
+RUN --mount=type=cache,target=/var/tmp/abxpkg-cache,sharing=locked,mode=1777,id=archivebox-opencode-$TARGETARCH \
+    chown -R "$DEFAULT_ARCHIVEBOX_UID:$DEFAULT_ARCHIVEBOX_GID" /var/tmp/abxpkg-cache \
+    && chmod 1777 /var/tmp/abxpkg-cache \
+    && env ABXPKG_TMP_CACHE_DIR=/var/tmp/abxpkg-cache XDG_CACHE_HOME=/var/tmp/abxpkg-cache \
+        setpriv --reuid="$ARCHIVEBOX_USER" --regid="$ARCHIVEBOX_USER" --init-groups abx-dl install opencode \
+    # pnpm includes musl variants that Debian's glibc runtime cannot use.
+    && find "$ABXPKG_LIB_DIR/pnpm/packages/opencode/node_modules" -type l -name 'opencode-linux-*-musl' -delete \
+    && find "$ABXPKG_LIB_DIR/pnpm/packages/opencode/node_modules/.pnpm" -maxdepth 1 -type d -name 'opencode-linux-*-musl@*' -exec rm -rf {} +
+RUN --network=none setpriv --reuid="$ARCHIVEBOX_USER" --regid="$ARCHIVEBOX_USER" --init-groups \
+    bash -c '"$ABXPKG_LIB_DIR/pnpm/packages/opencode/node_modules/.bin/opencode" --version && abx-dl install opencode'
+
 WORKDIR "$DATA_DIR"
 RUN echo "[+] Initializing image collection..." \
     && find "$DATA_DIR" -mindepth 1 -maxdepth 1 -exec rm -rf {} + \
@@ -282,10 +297,10 @@ RUN (echo -e "\n\n[√] Finished ArchiveBox multistage Docker build successfully
 
 WORKDIR "$DATA_DIR"
 VOLUME "$DATA_DIR"
-EXPOSE 8000
+EXPOSE 5797
 
 HEALTHCHECK --interval=30s --timeout=20s --retries=15 \
-    CMD curl --fail --silent --show-error --max-time 5 --connect-timeout 2 'http://admin.archivebox.localhost:8000/health/' | grep -q 'OK'
+    CMD curl --fail --silent --show-error --max-time 5 --connect-timeout 2 'http://admin.archivebox.localhost:5797/health/' | grep -q 'OK'
 
 ENTRYPOINT ["dumb-init", "--", "/app/bin/docker_entrypoint.sh"]
-CMD ["archivebox", "server", "--init", "0.0.0.0:8000"]
+CMD ["archivebox", "server", "--init", "0.0.0.0:5797"]

README.md

@@ -73,15 +73,15 @@ mkdir -p ~/archivebox/data && cd ~/archivebox
 curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml
 docker compose pull
 docker compose up -d --wait                                                # initializes new collections automatically
-# open http://admin.archivebox.localhost:8000 to finish setup
+# open http://admin.archivebox.localhost:5797 to finish setup
 # docker compose run --rm archivebox add 'https://example.com'
 # docker compose run --rm archivebox help
 <br/>
 <br/>
 # Option B: Or use it as a plain Docker container:
 mkdir -p ~/archivebox/data && cd ~/archivebox/data
-docker run -d --name archivebox -v "$PWD:/data" -p 8000:8000 archivebox/archivebox:dev
-# open http://admin.archivebox.localhost:8000 to finish setup
+docker run -d --name archivebox -v "$PWD:/data" -p 5797:5797 archivebox/archivebox:dev
+# open http://admin.archivebox.localhost:5797 to finish setup
 # docker run -it -v $PWD:/data archivebox/archivebox:dev add 'https://example.com'
 # docker run -it -v $PWD:/data archivebox/archivebox:dev help
 <br/>
@@ -93,14 +93,14 @@ archivebox init
 archivebox install
 # archivebox add 'https://example.com'
 # archivebox help
-# archivebox server 0.0.0.0:8000
+# archivebox server 0.0.0.0:5797
 <br/>
 <br/>
 # Option D: Or use the uv install shortcut for Option C
 curl -fsSL 'https://get.archivebox.io' | bash
 </code></pre>
 <br/>
-<sub>Open <a href="http://web.archivebox.localhost:8000"><code>http://web.archivebox.localhost:8000</code></a> for the public UI and <a href="http://admin.archivebox.localhost:8000"><code>http://admin.archivebox.localhost:8000</code></a> for the admin UI ➡️</sub><br/>
+<sub>Open <a href="http://web.archivebox.localhost:5797"><code>http://web.archivebox.localhost:5797</code></a> for the public UI and <a href="http://admin.archivebox.localhost:5797"><code>http://admin.archivebox.localhost:5797</code></a> for the admin UI ➡️</sub><br/>
 <sub>Set <code>BASE_URL</code> to change the public base domain. The default <code>auto</code> mode uses <code>web.</code> and <code>admin.</code> subdomains on <code>*.localhost</code>, but one host for ordinary DNS names. <code>BIND_ADDR</code> only controls the local listen address.</sub>
 </details>
 <br/>
@@ -129,10 +129,10 @@ curl -fsSL 'https://get.archivebox.io' | bash
 - [**Extracts a wide variety of content out-of-the-box**](https://github.com/ArchiveBox/ArchiveBox/issues/51): [media (yt-dlp), articles (readability), code (git), etc.](#output-formats)
 - [**Supports scheduled/realtime importing**](https://github.com/ArchiveBox/ArchiveBox/wiki/Scheduled-Archiving) from [many types of sources](#input-formats)
 - [**Uses standard, durable, long-term formats**](#output-formats) like HTML, JSON, PDF, PNG, MP4, TXT, and WARC
-- [**Powerful CLI**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#CLI-Usage), [**self-hosted web UI**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#UI-Usage), [Python API](https://docs.archivebox.io/en/dev/apidocs/archivebox/archivebox.html) (BETA), [REST API](https://github.com/ArchiveBox/ArchiveBox/issues/496) (ALPHA), or [desktop app](https://github.com/ArchiveBox/electron-archivebox)
+- [**Powerful CLI**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#CLI-Usage), [**self-hosted web UI**](https://github.com/ArchiveBox/ArchiveBox/wiki/Usage#UI-Usage), [Python API](https://docs.archivebox.io/en/dev/apidocs/archivebox/archivebox.html) (BETA), [REST API](https://github.com/ArchiveBox/ArchiveBox/issues/496) (ALPHA), or [desktop app](https://electron.archivebox.io/)
 - [**Saves all pages to archive.org as well**](https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#save_archive_dot_org) by default for redundancy (can be [disabled](https://github.com/ArchiveBox/ArchiveBox/wiki/Security-Overview#stealth-mode) for local-only mode)
-- Advanced users: support for archiving [content requiring login/paywall/cookies](https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#chrome_user_data_dir) (see wiki security caveats!)
-- Planned: support for running [JS during archiving](https://github.com/ArchiveBox/ArchiveBox/issues/51) to adblock, [autoscroll](https://github.com/ArchiveBox/ArchiveBox/issues/80), [modal-hide](https://github.com/ArchiveBox/ArchiveBox/issues/175), [thread-expand](https://github.com/ArchiveBox/ArchiveBox/issues/345)
+- Support for archiving [content requiring login/paywall/cookies](https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#chrome_user_data_dir) (see wiki security caveats!)
+- Support for running [JS during archiving](https://github.com/ArchiveBox/ArchiveBox/issues/51) to adblock, [infiniscroll](https://github.com/ArchiveBox/ArchiveBox/issues/80), [modal-hide](https://github.com/ArchiveBox/ArchiveBox/issues/175), [thread-expand](https://github.com/ArchiveBox/ArchiveBox/issues/345)
 
 <br/>
 
@@ -152,12 +152,12 @@ curl -fsSL 'https://get.archivebox.io' | bash
 #### ✳️&nbsp; Easy Setup
 
 <details>
-<summary><b><img src="https://archivebox.github.io/ios-archivebox/docs/icons/apple.svg" alt="Apple" height="28px" align="top"/> <code>ArchiveBox Server.app</code></b> (macOS)</summary>
+<summary><b><img src="https://app.archivebox.io/docs/icons/apple.svg" alt="Apple" height="28px" align="top"/> <code>ArchiveBox Server.app</code></b> (macOS)</summary>
 <br/>
 <ol>
 <li><a href="https://github.com/ArchiveBox/ios-archivebox/releases">Download <code>ArchiveBox Server.app</code></a>, move it to Applications, and open it.</li>
 <li>Set up an admin account and generate an API Key.</li>
-<li>Use the Web UI or <a href="https://archivebox.github.io/ios-archivebox/"><code>ArchiveBox.app</code></a>, and optionally connect REST API clients, the <a href="https://archivebox.github.io/ios-archivebox/">mobile app</a>, the <a href="https://archivebox.github.io/archivebox-browser-extension/">browser extension</a>, and other integrations to your server using your API Key.</li>
+<li>Use the Web UI or <a href="https://app.archivebox.io/"><code>ArchiveBox.app</code></a>, and optionally connect REST API clients, the <a href="https://app.archivebox.io/">mobile app</a>, the <a href="https://extension.archivebox.io/">browser extension</a>, and other integrations to your server using your API Key.</li>
 </ol>
 <br/>
 </details>
@@ -177,7 +177,7 @@ docker compose pull
 <li>Start the server, which initializes a new collection automatically.
 <pre lang="bash"><code style="white-space: pre-line">docker compose up -d --wait
 </code></pre></li>
-<li>Open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:8000/admin/">http://admin.archivebox.localhost:8000/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the web setup wizard.
+<li>Open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:5797/admin/">http://admin.archivebox.localhost:5797/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the web setup wizard.
 <pre lang="bash"><code style="white-space: pre-line">
 # run CLI commands inside the server container started above
 docker compose exec archivebox archivebox add 'https://example.com'
@@ -198,10 +198,10 @@ See <a href="#%EF%B8%8F-cli-usage">below</a> for more usage examples using the C
 <li>Install <a href="https://docs.docker.com/get-docker/">Docker</a> on your system (if not already installed).</li>
 <li>Create a new empty directory and start the server, which initializes the collection automatically (can be anywhere).
 <pre lang="bash"><code style="white-space: pre-line">mkdir -p ~/archivebox/data && cd ~/archivebox/data
-docker run -d --name archivebox -v $PWD:/data -p 8000:8000 archivebox/archivebox:dev
+docker run -d --name archivebox -v $PWD:/data -p 5797:5797 archivebox/archivebox:dev
 </code></pre>
 </li>
-<li>Open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:8000/admin/">http://admin.archivebox.localhost:8000/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the web setup wizard.
+<li>Open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:5797/admin/">http://admin.archivebox.localhost:5797/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the web setup wizard.
 <pre lang="bash"><code style="white-space: pre-line">
 # completely optional, CLI can always be used without running a server
 # docker exec archivebox archivebox [subcommand] [--help]
@@ -256,8 +256,8 @@ archivebox init     # initialize a new collection
 archivebox install  # install all the runtime dependencies (e.g. chrome, single-file, yt-dlp, etc.)
 </code></pre>
 </li>
-<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:8000/admin/">http://admin.archivebox.localhost:8000/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
-<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:8000
+<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:5797/admin/">http://admin.archivebox.localhost:5797/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
+<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:5797
 # completely optional, CLI can always be used without running a server
 # archivebox [subcommand] [--help]
 archivebox help
@@ -292,8 +292,8 @@ archivebox add 'https://example.com'
 </code></pre>
 <br/>
 </li>
-<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:8000/admin/">http://admin.archivebox.localhost:8000/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
-<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:8000
+<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:5797/admin/">http://admin.archivebox.localhost:5797/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
+<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:5797
 # completely optional, CLI can always be used without running a server
 # archivebox [subcommand] [--help]
 archivebox help
@@ -325,8 +325,8 @@ archivebox init
 archivebox install
 </code></pre>
 </li>
-<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:8000/admin/">http://admin.archivebox.localhost:8000/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
-<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:8000
+<li>Optionally start the server, then open <code>/admin/</code> on the hostname or IP used to reach ArchiveBox (local example: <a href="http://admin.archivebox.localhost:5797/admin/">http://admin.archivebox.localhost:5797/admin/</a>) to create the first admin. If <code>BASE_URL</code> is not configured yet, continue through the same web setup wizard used by Docker installs.
+<pre lang="bash"><code style="white-space: pre-line">archivebox server 0.0.0.0:5797
 # completely optional, CLI can always be used without running a server
 # archivebox [subcommand] [--help]
 archivebox help
@@ -365,17 +365,12 @@ See <a href="#%EF%B8%8F-cli-usage">below</a> for usage examples using the CLI, W
 <br/>
 <ol>
 <li>Install <a href="https://docs.docker.com/get-docker/">Docker</a> on your system (if not already installed).</li>
-<li>Download a binary release for your OS or build the native app from source<br/>
-<ul>
-<li>macOS: <a href="https://github.com/ArchiveBox/ArchiveBox/releases/download/v0.6.2/Electron-ArchiveBox-macOS-x64-0.6.2.app.zip" download><code>ArchiveBox.app.zip</code></a></li>
-<li>Linux: <code>ArchiveBox.deb</code> (alpha: <a href="https://github.com/ArchiveBox/electron-archivebox#quickstart">build manually</a>)</li>
-<li>Windows: <code>ArchiveBox.exe</code> (beta: <a href="https://github.com/ArchiveBox/electron-archivebox#quickstart">build manually</a>)</li>
-</ul>
-</li>
+<li><a href="https://electron.archivebox.io/#get-started">Download the latest installer for Windows, Linux, or Mac</a>.</li>
+<li>Install and open ArchiveBox Desktop, then create your local administrator account to start archiving.</li>
 </ol>
 <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/575ef92f-bb3e-4a7c-a4ba-986c1fd76ecf" width="320px">
 <br/>
-<i>✨ Alpha (contributors wanted!)</i>: for more info, see the: <a href="https://github.com/ArchiveBox/electron-archivebox">Electron ArchiveBox</a> repo.
+<a href="https://electron.archivebox.io/">Desktop app guide</a> · <a href="https://electron.archivebox.io/screenshots/">Screenshots</a>
 <br/>
 </details>
 
@@ -544,16 +539,16 @@ find ./archive/users -path '*/snapshots/*/*/*/index.html'  # inspect snapshot da
 <summary><b>🖥&nbsp; Web UI & API Usage</b></summary>
 <pre lang="bash"><code style="white-space: pre-line">
 # Start the server on bare metal (uv/apt/brew):
-archivebox server 0.0.0.0:8000                 # start the server
+archivebox server 0.0.0.0:5797                 # start the server
 <br/>
 # Or with Docker Compose:
 docker compose up                              # start the server
 <br/>
 # Or with a Docker container:
-docker run -v $PWD:/data -it -p 8000:8000 archivebox/archivebox:dev
+docker run -v $PWD:/data -it -p 5797:5797 archivebox/archivebox:dev
 </code></pre>
 
-<sup>Open <a href="http://admin.archivebox.localhost:8000/admin/"><code>http://admin.archivebox.localhost:8000/admin/</code></a> to create the first admin and finish web setup. Use <a href="http://web.archivebox.localhost:8000"><code>http://web.archivebox.localhost:8000</code></a> for the public UI. ➡️</sup><br/>
+<sup>Open <a href="http://admin.archivebox.localhost:5797/admin/"><code>http://admin.archivebox.localhost:5797/admin/</code></a> to create the first admin and finish web setup. Use <a href="http://web.archivebox.localhost:5797"><code>http://web.archivebox.localhost:5797</code></a> for the public UI. ➡️</sup><br/>
 <sup>Advanced: <code>archivebox manage createsuperuser</code> remains available for creating accounts from the CLI.</sup><br/>
 <sup>Set <code>BASE_URL</code> to change the public base domain. The default <code>auto</code> mode uses <code>web.</code> and <code>admin.</code> subdomains on <code>*.localhost</code>, but one host for ordinary DNS names. <code>BIND_ADDR</code> only controls the local listen address.</sup>
 <br/><br/>
@@ -570,18 +565,20 @@ docker compose run --rm archivebox config --set PERMISSIONS=public
 
 # restart the server to apply any config changes
 </code></pre>
+
+Snapshot caching follows each snapshot's saved `PERMISSIONS`, including inherited defaults and snapshot overrides. Public content can be cached by browsers and shared proxies/CDNs; private and unlisted content uses `Cache-Control: private` so only the user's browser can cache it. Both retain normal artifact cache lifetimes and validators for fast repeat viewing. Authentication handoffs are not stored. Changing permissions does not revoke copies already cached under the previous policy; purge any affected shared cache when needed.
 </details>
 
 <br/>
 
 <details>
-<summary><b><img src="https://archivebox.github.io/ios-archivebox/docs/icons/apple.svg" alt="Apple" height="22px" align="top"/> <img src="https://cdn.jsdelivr.net/gh/devicons/devicon@v2.17.0/icons/windows11/windows11-original.svg" alt="Windows" height="22px" align="top"/> 🐧&nbsp; Desktop / Mobile / Extension Usage</b></summary>
+<summary><b><img src="https://app.archivebox.io/docs/icons/apple.svg" alt="Apple" height="22px" align="top"/> <img src="https://cdn.jsdelivr.net/gh/devicons/devicon@v2.17.0/icons/windows11/windows11-original.svg" alt="Windows" height="22px" align="top"/> 🐧&nbsp; Desktop / Mobile / Extension Usage</b></summary>
 <br/>
 
-- 📱 [<code>ArchiveBox</code> iOS Mobile App](https://archivebox.github.io/ios-archivebox/)
-- 🖥️ [<code>ArchiveBox.app</code> macOS Desktop App](https://archivebox.github.io/ios-archivebox/)
-- 🧩 [ArchiveBox Browser Extension](https://archivebox.github.io/archivebox-browser-extension/)
-- 💻 [Linux/Windows Desktop App (Requires Docker Desktop)](https://github.com/ArchiveBox/electron-archivebox)
+- 📱 [<code>ArchiveBox</code> iOS Mobile App](https://app.archivebox.io/)
+- 🖥️ [<code>ArchiveBox.app</code> macOS Desktop App](https://app.archivebox.io/)
+- 🧩 [ArchiveBox Browser Extension](https://extension.archivebox.io/)
+- 💻 [Linux/Windows Desktop App (Requires Docker Desktop)](https://electron.archivebox.io/)
 
 <br/>
 </details>
@@ -641,7 +638,7 @@ docker run -it -v $PWD:/data archivebox/archivebox:dev add --depth=1 'https://ex
 - <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/ff20d251-5347-4b85-ae9b-83037d0ac01e" height="28px"/> <b>From the official <a href="https://github.com/ArchiveBox/archivebox-extension">ArchiveBox Browser Extension</a></b>  
   <i>Provides realtime archiving of browsing history or selected pages from Chrome/Chromium/Firefox/Safari browsers.</i>
 
-- <img src="https://archivebox.github.io/ios-archivebox/App/AppIcon.icon/Assets/ArchiveBox.png" alt="ArchiveBox app icon" height="28px"/> <b>From the official <a href="https://archivebox.github.io/ios-archivebox/">ArchiveBox macOS Desktop App or iOS Mobile App</a></b><br/>
+- <img src="https://app.archivebox.io/App/AppIcon.icon/Assets/ArchiveBox.png" alt="ArchiveBox app icon" height="28px"/> <b>From the official <a href="https://app.archivebox.io/">ArchiveBox macOS Desktop App or iOS Mobile App</a></b><br/>
   <i>Save directly from the Share menu or Siri/Shortcuts to your server.</i>
 
 - <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/64078483-21d7-4eb1-aa6e-9ad55afe45b8" height="22px"/> From manual imports of URLs from RSS, JSON, CSV, TXT, SQL, HTML, Markdown, etc. files  
@@ -1008,14 +1005,11 @@ For various reasons, many large sites (Reddit, Twitter, Cloudflare, etc.) active
 
 
 <ul>
-<li>Set <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Configuration#curl_user_agent"><code>CHROME_USER_AGENT</code>, <code>WGET_USER_AGENT</code>, <code>CURL_USER_AGENT</code></a> to impersonate a real browser (by default, ArchiveBox reveals that it's a bot when using the default user agent settings)</li>
-<li>Set up a logged-in browser session for archiving by <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Chromium-Install#import-an-existing-browser-profile">importing a dedicated browser profile into a persona</a></li>
+<li>Set up a logged-in browser session for archiving by <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Chromium-Install#import-an-existing-browser-profile">importing a chrome profile into a persona</a></li>
 <li>Rewrite your URLs before archiving to swap in alternative frontends that are more bot-friendly e.g.<br>
 <code>reddit.com/some/url</code> -&gt; <code>teddit.net/some/url</code>: <a href="https://github.com/mendel5/alternative-front-ends">https://github.com/mendel5/alternative-front-ends</a></li>
 </ul>
 
-In the future we plan on adding support for running JS scripts during archiving to block ads, cookie popups, modals, and fix other issues. Follow here for progress: <a href="https://github.com/ArchiveBox/ArchiveBox/issues/51">Issue #51</a>.
-
 </details>
 <br/>
 
@@ -1346,10 +1340,10 @@ archivebox init
 archivebox install         # detect and install all extractor dependencies
 
 # Run the development server w/ autoreloading (but no bg workers)
-archivebox server --debug --reload 0.0.0.0:8000
+archivebox server --debug --reload 0.0.0.0:5797
 
 # Run the production server (with bg workers but no autoreloading)
-archivebox server 0.0.0.0:8000
+archivebox server 0.0.0.0:5797
 ```
 
 #### 2. Option B: Build the docker container and use that for development instead
@@ -1361,10 +1355,10 @@ archivebox server 0.0.0.0:8000
 ./bin/build_docker.sh dev
 
 # Run the development server w/ autoreloading (but no bg workers)
-docker run -it -v $PWD/data:/data -v $PWD/archivebox:/app/archivebox -p 8000:8000 archivebox/archivebox:dev server --init --debug --reload 0.0.0.0:8000
+docker run -it -v $PWD/data:/data -v $PWD/archivebox:/app/archivebox -p 5797:5797 archivebox/archivebox:dev server --init --debug --reload 0.0.0.0:5797
 
 # Run the production server (with bg workers but no autoreloading)
-docker run -it -v $PWD/data:/data -v $PWD/archivebox:/app/archivebox -p 8000:8000 archivebox/archivebox:dev server --init
+docker run -it -v $PWD/data:/data -v $PWD/archivebox:/app/archivebox -p 5797:5797 archivebox/archivebox:dev server --init
 
 # (remove the --reload flag and add the --nothreading flag when profiling with the django debug toolbar)
 # When using --reload, make sure any files you create can be read by the user in the Docker container, eg with 'chmod a+rX'.
@@ -1386,11 +1380,11 @@ You can also run all these in Docker. For more examples see the GitHub Actions C
 archivebox config --set DEBUG=True
 
 # OR you can run a dev server with DEBUG=True in a few ways:
-archivebox server --debug --reload 0.0.0.0:8000
+archivebox server --debug --reload 0.0.0.0:5797
 # or
-archivebox server --debug 0.0.0.0:8000
+archivebox server --debug 0.0.0.0:5797
 # or
-env DEBUG=True daphne -b 0.0.0.0 -p 8000 archivebox.core.asgi:application
+env DEBUG=True daphne -b 0.0.0.0 -p 5797 archivebox.core.asgi:application
 ```
 
 https://stackoverflow.com/questions/1074212/how-can-i-see-the-raw-sql-queries-django-is-running
@@ -1425,7 +1419,7 @@ services:
 
 # or with plain Docker:
 docker build -t archivebox:dev https://github.com/ArchiveBox/ArchiveBox.git#dev
-docker run -it -v $PWD:/data -p 8000:8000 archivebox:dev
+docker run -it -v $PWD:/data -p 5797:5797 archivebox:dev
 
 # or with uv:
 uv tool install --python 3.13 --upgrade 'git+https://github.com/ArchiveBox/ArchiveBox.git@dev'

archivebox/api/v1_core.py

@@ -247,7 +247,7 @@ def get_archiveresult(request: HttpRequest, archiveresult_id: str):
 
 def _normalize_uploaded_archiveresult_plugin(plugin: str) -> str:
     normalized = str(plugin or "").strip().strip("/")
-    if not ARCHIVERESULT_UPLOAD_PLUGIN_RE.fullmatch(normalized):
+    if normalized in (".", "..") or not ARCHIVERESULT_UPLOAD_PLUGIN_RE.fullmatch(normalized):
         raise HttpError(400, "Invalid ArchiveResult plugin name")
     return normalized
 
@@ -386,6 +386,7 @@ def _write_archiveresult_files(
     existing_output_files: dict[str, dict[str, Any]] | None = None,
     allow_empty: bool = False,
 ) -> dict[str, dict[str, Any]]:
+    plugin_name = _normalize_uploaded_archiveresult_plugin(plugin_name)
     files = _get_archiveresult_upload_files(request, allow_empty=allow_empty)
     output_paths = _get_archiveresult_upload_form_values(request, "output_paths", "output_path")
     mime_types = _get_archiveresult_upload_form_values(request, "mime_types", "mime_type")

archivebox/cli/archivebox_help.py

@@ -68,7 +68,7 @@ def help() -> None:
     [blue]docker compose run[/blue] [dark_green]archivebox[/dark_green] [green]\\[command][/green] [green3][...args][/green3] [violet][--help][/violet] [grey53][--version][/grey53]
 
     [grey53]# using Docker:[/grey53]
-    [blue]docker run[/blue] -v [light_slate_blue]$PWD:/data[/light_slate_blue] [grey53]-p 8000:8000[/grey53] -it [dark_green]archivebox/archivebox[/dark_green] [green]\\[command][/green] [green3][...args][/green3] [violet][--help][/violet] [grey53][--version][/grey53]
+    [blue]docker run[/blue] -v [light_slate_blue]$PWD:/data[/light_slate_blue] [grey53]-p 5797:5797[/grey53] -it [dark_green]archivebox/archivebox[/dark_green] [green]\\[command][/green] [green3][...args][/green3] [violet][--help][/violet] [grey53][--version][/grey53]
 """
         if IN_DOCKER
         else ""
@@ -109,7 +109,7 @@ def help() -> None:
     [dark_green]archivebox[/dark_green] [green]add[/green] --depth=1 "https://example.com/some/page"
     [dark_green]archivebox[/dark_green] [green]list[/green] --sort=timestamp --csv=timestamp,downloaded_at,url,title
     [dark_green]archivebox[/dark_green] [green]schedule[/green] --every=day --depth=1 "https://example.com/some/feed.rss"
-    [dark_green]archivebox[/dark_green] [green]server[/green] [blue]0.0.0.0:8000[/blue]                [grey53]# Start the Web UI / API server[/grey53]
+    [dark_green]archivebox[/dark_green] [green]server[/green] [blue]0.0.0.0:5797[/blue]                [grey53]# Start the Web UI / API server[/grey53]
 """
         print(
             Panel(

archivebox/cli/archivebox_init.py

@@ -134,7 +134,7 @@ def init(force: bool = False, quick: bool = False, install: bool = False) -> Non
     print("    > Skipping orphan snapshot import during init.")
     print()
     print("    [violet]Hint:[/violet] To import orphaned snapshot directories and reconcile filesystem state, run:")
-    print("        archivebox update")
+    print("        archivebox update --rescan --migrate-only")
 
     print("\n[green]----------------------------------------------------------------------[/green]")
 

archivebox/cli/archivebox_persona.py

@@ -109,6 +109,7 @@ def create_personas(
     profile: str | None = None,
     source: str | None = None,
     browser_binary: str | None = None,
+    permissions: str | None = None,
 ) -> int:
     """
     Create Personas from names.
@@ -162,7 +163,10 @@ def create_personas(
             rprint(f'[red]Invalid persona name "{name}": {error_msg}[/red]', file=sys.stderr)
             continue
 
-        persona, created = Persona.objects.get_or_create(name=name)
+        persona, created = Persona.objects.get_or_create(
+            name=name,
+            defaults={"config": {"PERMISSIONS": permissions}} if permissions else {},
+        )
 
         if created:
             persona.ensure_dirs()
@@ -441,7 +445,11 @@ def main():
 
 @main.command("create")
 @click.argument("names", nargs=-1)
-@click.option("--import", "import_from", help="Import from chrome, chromium, brave, edge, a persona directory, or a live CDP URL")
+@click.option(
+    "--import",
+    "import_from",
+    help="Import from chrome, chrome-beta, chromium, brave, edge, a persona directory, or a live CDP URL",
+)
 @click.option("--source", help="Source browser user-data directory or exact profile path (including Docker mounts)")
 @click.option(
     "--browser-binary",
@@ -449,9 +457,30 @@ def main():
     help="Source Chromium browser executable; required for other Chromium-based browsers",
 )
 @click.option("--profile", help="Profile directory name under the user data dir (e.g. Default, Profile 1)")
-def create_cmd(names: tuple, import_from: str | None, profile: str | None, source: str | None, browser_binary: str | None):
+@click.option(
+    "--permissions",
+    type=click.Choice(["public", "unlisted", "private"]),
+    help="Visibility for a newly created persona (existing personas are unchanged)",
+)
+def create_cmd(
+    names: tuple,
+    import_from: str | None,
+    profile: str | None,
+    source: str | None,
+    browser_binary: str | None,
+    permissions: str | None,
+):
     """Create Personas, optionally importing from a browser profile."""
-    sys.exit(create_personas(names, import_from=import_from, profile=profile, source=source, browser_binary=browser_binary))
+    sys.exit(
+        create_personas(
+            names,
+            import_from=import_from,
+            profile=profile,
+            source=source,
+            browser_binary=browser_binary,
+            permissions=permissions,
+        ),
+    )
 
 
 @main.command("list")

archivebox/cli/archivebox_server.py

@@ -46,13 +46,13 @@ def _split_bind_spec(spec: str) -> tuple[str, str]:
     """Split a ``host:port`` / ``host`` / ``port`` spec into ``(host, port)``.
 
     The empty strings stand in for "not provided"; the caller fills in
-    defaults. Bracketed IPv6 literals like ``[::1]:8000`` are handled.
+    defaults. Bracketed IPv6 literals like ``[::1]:5797`` are handled.
     """
     spec = (spec or "").strip()
     if not spec:
         return "", ""
     if spec.startswith("["):
-        # Bracketed IPv6: ``[::1]`` or ``[::1]:8000``
+        # Bracketed IPv6: ``[::1]`` or ``[::1]:5797``
         end = spec.find("]")
         if end == -1:
             return spec, ""  # malformed; let validator reject it
@@ -77,11 +77,11 @@ def _parse_and_validate_bind_spec(spec: str) -> tuple[str, str]:
     (normalized to ``127.0.0.1``). Bare hostnames are rejected because the
     bind address feeds Daphne, which has to listen on a numeric address;
     public hostnames belong in ``BASE_URL`` instead. Empty values fall back
-    to ``127.0.0.1`` / ``8000``.
+    to ``127.0.0.1`` / ``5797``.
     """
     raw_host, raw_port = _split_bind_spec(spec)
     host = raw_host.strip()
-    port = (raw_port or "").strip() or "8000"
+    port = (raw_port or "").strip() or "5797"
 
     if host == "" or host.lower() == "localhost":
         host = "127.0.0.1"
@@ -198,7 +198,7 @@ def _print_server_startup_warnings(config, host: str, port: str) -> None:
             "[yellow]    routing cannot work against an IP address. Set BASE_URL explicitly, e.g.[/yellow]",
         )
         print(
-            "[yellow]      BASE_URL=https://archive.example.com archivebox server 0.0.0.0:8000[/yellow]",
+            "[yellow]      BASE_URL=https://archive.example.com archivebox server 0.0.0.0:5797[/yellow]",
         )
         if config.USES_SUBDOMAIN_ROUTING:
             print(
@@ -221,7 +221,7 @@ def _print_server_startup_warnings(config, host: str, port: str) -> None:
             "[yellow]    a reverse proxy / ingress / public hostname, e.g.[/yellow]",
         )
         print(
-            "[yellow]      BASE_URL=https://archive.example.com archivebox server 0.0.0.0:8000[/yellow]",
+            "[yellow]      BASE_URL=https://archive.example.com archivebox server 0.0.0.0:5797[/yellow]",
         )
         print()
 
@@ -254,7 +254,7 @@ def server(
         print()
 
     # First non-empty positional arg is the bind spec; otherwise inherit from
-    # config (which defaults to "127.0.0.1:8000"). _parse_and_validate_bind_spec
+    # config (which defaults to "127.0.0.1:5797"). _parse_and_validate_bind_spec
     # hard-errors on hostnames so the rest of the server can assume a numeric
     # bind host.
     bind_spec = next((arg for arg in runserver_args if arg), "")

archivebox/cli/archivebox_snapshot.py

@@ -44,6 +44,7 @@ SNAPSHOT_LIST_CHUNK_SIZE = 5000
 
 def iter_snapshot_json(queryset: QuerySet) -> Iterator[dict[str, object]]:
     from archivebox.config import VERSION
+    from archivebox.config.common import redact_sensitive_config
     from archivebox.core.models import SnapshotTag
 
     fields = (
@@ -53,6 +54,9 @@ def iter_snapshot_json(queryset: QuerySet) -> Iterator[dict[str, object]]:
         "title",
         "bookmarked_at",
         "created_at",
+        "modified_at",
+        "config",
+        "notes",
         "timestamp",
         "depth",
         "status",
@@ -80,6 +84,9 @@ def iter_snapshot_json(queryset: QuerySet) -> Iterator[dict[str, object]]:
                 "tags": ",".join(sorted(tags_by_snapshot[row["id"]])),
                 "bookmarked_at": row["bookmarked_at"].isoformat() if row["bookmarked_at"] else None,
                 "created_at": row["created_at"].isoformat() if row["created_at"] else None,
+                "modified_at": row["modified_at"].isoformat() if row["modified_at"] else None,
+                "config": redact_sensitive_config(row["config"]),
+                "notes": row["notes"],
                 "timestamp": row["timestamp"],
                 "depth": row["depth"],
                 "status": row["status"],

archivebox/cli/archivebox_update.py

@@ -7,6 +7,7 @@ import os
 import asyncio
 import shlex
 import time
+from itertools import islice
 
 from typing import TYPE_CHECKING, Any
 from collections.abc import Iterable
@@ -52,17 +53,20 @@ def _get_search_indexing_plugins() -> list[str]:
 def _build_filtered_snapshots_queryset(
     **kwargs,
 ):
-    from archivebox.core.models import Snapshot
     from archivebox.cli.archivebox_snapshot import build_snapshot_queryset
 
     limit = kwargs.pop("limit", None)
+    reverse = kwargs.pop("reverse", False)
     snapshots = build_snapshot_queryset(**kwargs)
+    snapshots = snapshots.order_by(kwargs.get("sort") or "-created_at", "-id")
+    if reverse:
+        snapshots = snapshots.reverse()
     if kwargs.get("resume"):
-        snapshots = snapshots.filter(timestamp__lte=kwargs["resume"])
+        snapshots = snapshots.filter(**{"timestamp__gte" if reverse else "timestamp__lte": kwargs["resume"]})
     snapshots = snapshots.select_related("crawl")
     if limit is not None and limit > 0:
         snapshot_ids = list(snapshots.values_list("id", flat=True)[:limit])
-        snapshots = Snapshot.objects.filter(id__in=snapshot_ids).select_related("crawl")
+        snapshots = snapshots.filter(id__in=snapshot_ids)
 
     return snapshots
 
@@ -73,6 +77,7 @@ def reindex_snapshots(
     search_plugins: list[str],
     batch_size: int,
     collect_ids: bool = False,
+    reverse: bool = False,
     wait_for_turn=None,
 ) -> dict[str, Any]:
     from archivebox.cli.archivebox_extract import run_plugins
@@ -94,16 +99,15 @@ def reindex_snapshots(
             plugin=plugin_name,
             status__in=completed_statuses,
         )
-        candidates = snapshots.annotate(has_completed_index=Exists(completed_result)).filter(has_completed_index=False).order_by("id")
-        after_id = None
+        candidates = snapshots.annotate(has_completed_index=Exists(completed_result)).filter(has_completed_index=False)
+        candidates = candidates.order_by("created_at", "id") if reverse else candidates.order_by("-created_at", "-id")
+        remaining = candidates.select_related(None).only("id", "timestamp", "created_at").paged_iterator(chunk_size=batch_size)
         while True:
             if wait_for_turn:
                 wait_for_turn()
-            page = candidates.filter(id__gt=after_id) if after_id is not None else candidates
-            batch = list(page.select_related(None).only("id", "timestamp")[:batch_size])
+            batch = list(islice(remaining, batch_size))
             if not batch:
                 break
-            after_id = batch[-1].id
             records = [{"type": "Snapshot", "id": str(snapshot.id)} for snapshot in batch]
             stats["processed"] += len(batch)
             stats["requested"] += len(batch)
@@ -161,18 +165,18 @@ def update(
     continuous: bool = False,
     index_only: bool = False,
     migrate_only: bool = False,
+    rescan: bool = False,
+    reverse: bool = False,
     stop_daemon_stack: bool = True,
 ) -> None:
     """
-    Update snapshots: migrate old dirs, reconcile DB, and re-queue for archiving.
+    Update the database without forcing a filesystem scan by default.
 
-    Three-phase operation (without filters):
-    - Phase 1: Drain legacy archive/ directories into the current layout
-    - Phase 2: Select only stale fs_version rows through the indexed column
-    - Phase 3: Run queued snapshot-level filesystem maintenance until idle
-
-    With filters: Only phase 2 (DB query), no filesystem operations.
-    Without filters: All phases (full update).
+    --migrate-only: eagerly finish pending filesystem migrations for known rows.
+    --index-only: reconcile known snapshot assets/metadata and backfill search.
+    --rescan: discover filesystem orphans first, then repair known snapshots.
+    Explicit scans default to newest first; --reverse selects oldest first.
+    Filters select existing database snapshots for targeted updates.
     """
 
     from rich import print
@@ -248,12 +252,28 @@ def update(
     exit_code = 0
 
     try:
+        if rescan and (is_filtered_update or resume or continuous):
+            raise click.UsageError("--rescan cannot be combined with filters, --resume, or --continuous; rerun it to resume automatically.")
+        if rescan:
+            migrate_only = True  # Discovery must not schedule new captures.
         wait_for_turn()
 
         with foreground_shutdown_signals(), foreground_parent_watchdog():
             while True:
-                do_migrate = migrate_only or not index_only
-                do_index = index_only or not migrate_only
+                if rescan:
+                    from archivebox.cli.rescan import rescan_snapshots
+
+                    rescan_snapshots(
+                        wait_for_turn=wait_for_turn,
+                        reverse=reverse,
+                        scan_legacy=lambda: drain_old_archive_dirs(batch_size=batch_size, reverse=reverse),
+                    )
+                do_migrate = migrate_only or is_filtered_update and not index_only
+                do_index = index_only
+                if not do_migrate and not do_index:
+                    print(
+                        "[*] Collection ready. Filesystem migrations remain lazy; use --migrate-only, --index-only, or --rescan for explicit scans.",
+                    )
 
                 if do_migrate:
                     if (
@@ -285,7 +305,8 @@ def update(
                             after=after,
                             resume=resume,
                             batch_size=batch_size,
-                            queue_for_archiving=True,
+                            queue_for_archiving=not migrate_only,
+                            reverse=reverse,
                             wait_for_turn=wait_for_turn,
                         )
                         print_stats(stats)
@@ -293,17 +314,13 @@ def update(
                     else:
                         stats_combined = {"phase1": {}, "phase2": {}}
 
-                        print("[*] Phase 1: Draining old archive/ directories (0.8.x → 0.9.x migration)...")
-                        stats_combined["phase1"] = drain_old_archive_dirs(
-                            resume_from=resume,
-                            batch_size=batch_size,
-                        )
-
                         print("[*] Phase 2: Selecting database snapshots with stale filesystem versions...")
                         stats_combined["phase2"] = process_all_db_snapshots(
                             batch_size=batch_size,
                             resume=resume,
                             wait_for_turn=wait_for_turn,
+                            reverse=reverse,
+                            migrate=True,
                         )
                         print_combined_stats(stats_combined)
                     # The due selectors are indexed and cheap when empty, so
@@ -311,7 +328,7 @@ def update(
                     # whole-table counts merely to decide whether to call it.
                     print("[*] Phase 3: Running filesystem maintenance until idle...")
                     if is_filtered_update:
-                        for snapshot_id in sorted(touched_snapshot_ids):
+                        for snapshot_id in sorted(touched_snapshot_ids, reverse=not reverse):
                             run_scoped_runner("--snapshot-id", snapshot_id)
                     elif migrate_only:
                         run_scoped_runner("--maintenance-only")
@@ -319,6 +336,25 @@ def update(
                         run_scoped_runner()
 
                 if do_index:
+                    from archivebox.cli.rescan import reconcile_known_snapshots
+
+                    snapshots = _build_filtered_snapshots_queryset(
+                        filter_patterns=filter_patterns,
+                        filter_type=filter_type,
+                        status=status,
+                        url__icontains=url__icontains,
+                        url__istartswith=url__istartswith,
+                        tag=tag,
+                        crawl_id=crawl_id,
+                        limit=limit,
+                        sort=sort,
+                        search=search,
+                        before=before,
+                        after=after,
+                        resume=resume,
+                        reverse=reverse,
+                    )
+                    reconcile_known_snapshots(snapshots, wait_for_turn=wait_for_turn)
                     search_plugins = _get_search_indexing_plugins()
                     if not search_plugins:
                         print("[*] No search indexing plugins are available, nothing to backfill.")
@@ -337,12 +373,14 @@ def update(
                             before=before,
                             after=after,
                             resume=resume,
+                            reverse=reverse,
                         )
                         stats = reindex_snapshots(
                             snapshots,
                             search_plugins=search_plugins,
                             batch_size=batch_size,
                             collect_ids=is_filtered_update,
+                            reverse=reverse,
                             wait_for_turn=wait_for_turn,
                         )
                         print_index_stats(stats)
@@ -358,6 +396,10 @@ def update(
         exit_code = 130
         exact_resume = err.__dict__.get("archivebox_resume")
         resume_cmd = ["archivebox", "update"]
+        if rescan:
+            resume_cmd.append("--rescan")
+        if reverse:
+            resume_cmd.append("--reverse")
         if migrate_only:
             resume_cmd.append("--migrate-only")
         if index_only:
@@ -405,7 +447,7 @@ def update(
             stop_own_supervisord_process()
 
 
-def drain_old_archive_dirs(resume_from: str | None = None, batch_size: int = 500) -> dict[str, int]:
+def drain_old_archive_dirs(resume_from: str | None = None, batch_size: int = 500, reverse: bool = False) -> dict[str, int]:
     """
     Drain old archive/ directories (0.8.x → 0.9.x migration).
 
@@ -471,18 +513,18 @@ def drain_old_archive_dirs(resume_from: str | None = None, batch_size: int = 500
 
     # Scan real legacy directories only; these still contain data to migrate.
     entries = [
-        (e.stat().st_mtime, e.path)
+        (float(e.name), e.path)
         for e in all_entries
         if e.is_dir(follow_symlinks=False) and Snapshot.is_legacy_archive_dir(Path(e.path))  # Skip symlinks and 0.9.x roots
     ]
-    entries.sort(reverse=True)  # Newest first
+    entries.sort(reverse=not reverse)  # Newest first
     print(f"[*] Found {len(entries)} old directories to drain")
 
     for mtime, entry_path in entries:
         entry_path = Path(entry_path)
 
         # Resume from timestamp if specified
-        if resume_from and entry_path.name > resume_from:
+        if resume_from and (entry_path.name < resume_from if reverse else entry_path.name > resume_from):
             continue
 
         stats["processed"] += 1
@@ -584,7 +626,13 @@ def drain_old_archive_dirs(resume_from: str | None = None, batch_size: int = 500
     return stats
 
 
-def process_all_db_snapshots(batch_size: int = 500, resume: str | None = None, wait_for_turn=None) -> dict[str, int]:
+def process_all_db_snapshots(
+    batch_size: int = 500,
+    resume: str | None = None,
+    wait_for_turn=None,
+    reverse: bool = False,
+    migrate: bool = False,
+) -> dict[str, int]:
     """Queue only snapshots whose indexed filesystem version is stale."""
     from archivebox.core.models import Snapshot
     from django.db.models import Q
@@ -601,21 +649,27 @@ def process_all_db_snapshots(batch_size: int = 500, resume: str | None = None, w
     }
     queryset = Snapshot.objects.filter(fs_version__in=Snapshot._FS_VERSION_MIGRATION_PATHS)
     if resume:
-        queryset = queryset.filter(timestamp__lte=resume)
+        queryset = queryset.filter(**{"timestamp__gte" if reverse else "timestamp__lte": resume})
     initial_now = timezone.now()
-    rows_to_wake = queryset.filter(Q(retry_at__isnull=True) | Q(retry_at__gt=initial_now))
-    after_id = None
+    rows_to_wake = queryset if migrate else queryset.filter(Q(retry_at__isnull=True) | Q(retry_at__gt=initial_now))
+    ordered = rows_to_wake.order_by("created_at", "id") if reverse else rows_to_wake.order_by("-created_at", "-id")
+    remaining = ordered.select_related("crawl__created_by").paged_iterator(chunk_size=batch_size)
     while True:
         if wait_for_turn:
             wait_for_turn()
-        page = rows_to_wake.filter(id__gt=after_id) if after_id is not None else rows_to_wake
-        batch = list(page.only("id", "fs_version", "modified_at").order_by("id")[:batch_size])
+        batch = list(islice(remaining, batch_size))
         if not batch:
             break
-        after_id = batch[-1].id
         now = timezone.now()
         updated = 0
         for snapshot in batch:
+            if migrate:
+                from archivebox.services.runner import run_snapshot_maintenance
+
+                if wait_for_turn:
+                    wait_for_turn()
+                run_snapshot_maintenance(str(snapshot.id))
+                continue
             updated += int(
                 snapshot.safe_update(
                     {"retry_at": now, "modified_at": now},
@@ -647,6 +701,7 @@ def process_filtered_snapshots(
     resume: str | None,
     batch_size: int,
     queue_for_archiving: bool = True,
+    reverse: bool = False,
     wait_for_turn=None,
 ) -> dict[str, Any]:
     """Process snapshots matching filters (DB query only)."""
@@ -669,6 +724,7 @@ def process_filtered_snapshots(
         before=before,
         after=after,
         resume=resume,
+        reverse=reverse,
     )
 
     total = snapshots.count()
@@ -687,6 +743,10 @@ def process_filtered_snapshots(
             stats["snapshot_ids"].append(str(snapshot.id))
             update_values = {}
             updated = 0
+            if not queue_for_archiving and snapshot.fs_migration_needed:
+                from archivebox.services.runner import run_snapshot_maintenance
+
+                run_snapshot_maintenance(str(snapshot.id))
             if queue_for_archiving:
                 update_values.update(
                     {
@@ -784,6 +844,12 @@ def print_index_stats(stats: dict[str, Any]) -> None:
 @click.option("--continuous", is_flag=True, help="Run continuously as background worker")
 @click.option("--index-only", is_flag=True, help="Backfill available search indexes from existing archived content")
 @click.option("--migrate-only", is_flag=True, help="Only migrate filesystem and update database/index state")
+@click.option("--reverse", is_flag=True, help="Process oldest first instead of newest first for explicit scans")
+@click.option(
+    "--rescan",
+    is_flag=True,
+    help="Scan all snapshot directories for orphans and metadata repairs (slow, opt-in, safe to restart)",
+)
 @snapshot_filter_options(default_filter_type="exact")
 @docstring(update.__doc__)
 def main(**kwargs):

archivebox/cli/rescan.py

@@ -0,0 +1,222 @@
+"""Explicit discovery of portable snapshots; no checkpoint or default tree walk."""
+
+import json
+import os
+import uuid
+from itertools import groupby
+from pathlib import Path
+
+from django.contrib.auth import get_user_model
+from django.core.exceptions import ValidationError
+from django.db import IntegrityError
+from rich import print
+
+from archivebox.config import CONSTANTS
+from archivebox.core.models import ArchiveResult, Snapshot
+from archivebox.crawls.models import Crawl
+from archivebox.misc.util import parse_date
+
+
+def _directories(parent: Path):
+    # Names encode dates / UUIDv7 creation time. Newest first within each
+    # subtree, with bounded memory and no traversal into plugin payloads.
+    with os.scandir(parent) as entries:
+        paths = [Path(entry.path) for entry in entries if entry.is_dir(follow_symlinks=False)]
+    yield from sorted(paths, key=lambda path: path.name, reverse=True)
+
+
+def _snapshot_directories(reverse=False):
+    dates = []
+    for user in _directories(CONSTANTS.USERS_DIR):
+        root = user / CONSTANTS.SNAPSHOTS_DIR_NAME
+        if root.is_symlink() or not root.is_dir():
+            continue
+        dates.extend(_directories(root))
+    dates.sort(key=lambda path: path.name, reverse=not reverse)
+    for _, date_group in groupby(dates, key=lambda path: path.name):
+        directories = [directory for date in date_group for domain in _directories(date) for directory in _directories(domain)]
+        yield from sorted(directories, key=lambda path: path.name, reverse=not reverse)
+
+
+def _fields(record, names):
+    return {name: record[name] for name in names.split() if name in record}
+
+
+def _dates(record):
+    return {name: parse_date(record[name]) for name in ("created_at", "modified_at") if record.get(name)}
+
+
+def _restore_dates(obj, record):
+    dates = _dates(record)
+    if dates:
+        type(obj).objects.filter(pk=obj.pk).update(**dates)
+        for name, value in dates.items():
+            setattr(obj, name, value)
+
+
+def _reconcile_directory(directory: Path) -> str:
+    snapshot_id = uuid.UUID(directory.name)
+    snapshot = Snapshot.objects.select_related("crawl__created_by").filter(pk=snapshot_id).first()
+    index = directory / CONSTANTS.JSONL_INDEX_FILENAME
+    if index.is_symlink():
+        raise ValueError("metadata must not be a symlink")
+    if not index.exists():
+        if snapshot is None or Path(snapshot.output_dir) != directory or snapshot.status != "sealed":
+            raise ValueError("missing index.jsonl; cannot reconstruct an unknown or active snapshot")
+        snapshot.write_index_jsonl(output_dir=directory)
+        snapshot.reconcile_filesystem_links()
+        return "Repaired"
+
+    records = [json.loads(line) for line in index.read_text().splitlines() if line.strip()]
+    if any(not isinstance(record, dict) for record in records):
+        raise ValueError("metadata records must be JSON objects")
+    snapshots = [record for record in records if record.get("type") == "Snapshot"]
+    if len(snapshots) != 1:
+        raise ValueError("expected exactly one Snapshot record")
+    record = snapshots[0]
+    if uuid.UUID(record["id"]) != snapshot_id:
+        raise ValueError("snapshot ID conflicts with directory name")
+    if record.get("status") != "sealed":
+        raise ValueError("snapshot is not sealed; finish the source capture first")
+    if record.get("fs_version") not in {Snapshot._fs_current_version(), *Snapshot._FS_VERSION_MIGRATION_PATHS}:
+        raise ValueError("unsupported filesystem version")
+    crawl_id = uuid.UUID(record["crawl_id"])
+    owner = directory.parents[3].name
+    crawl_records = [item for item in records if item.get("type") == "Crawl" and uuid.UUID(item["id"]) == crawl_id]
+    crawl_record = crawl_records[0] if crawl_records else {}
+    if crawl_record.get("created_by", owner) != owner:
+        raise ValueError("crawl owner conflicts with directory path")
+    crawl = Crawl.objects.select_related("created_by").filter(pk=crawl_id).first()
+    imported_crawl = crawl is None
+    if crawl and crawl.created_by.username != owner:
+        raise ValueError("existing crawl ID belongs to a different owner")
+    if snapshot and (snapshot.url != record["url"] or snapshot.crawl_id != crawl_id or snapshot.timestamp != record["timestamp"]):
+        raise ValueError("existing snapshot ID has conflicting identity; refusing to overwrite")
+    if snapshot and Path(snapshot.output_dir) != directory:
+        raise ValueError("existing snapshot has a different storage path")
+
+    # Validate identities before inserting anything. Never turn an ID collision
+    # into a new ID, and never overwrite a newer destination row.
+    # write_index_jsonl retains historical hook records before appending the
+    # canonical DB rows. The final record for each hook is authoritative.
+    results = {(item.get("plugin"), item.get("hook_name", "")): item for item in records if item.get("type") == "ArchiveResult"}.values()
+    pending = []
+    for result in results:
+        result_id = uuid.UUID(result["id"])
+        if uuid.UUID(result["snapshot_id"]) != snapshot_id:
+            raise ValueError("ArchiveResult belongs to another snapshot")
+        existing = ArchiveResult.objects.filter(pk=result_id).first()
+        identity = (snapshot_id, result["plugin"], result.get("hook_name", ""))
+        if existing:
+            if (existing.snapshot_id, existing.plugin, existing.hook_name) != identity:
+                raise ValueError("existing ArchiveResult ID has conflicting identity")
+        elif ArchiveResult.objects.filter(snapshot_id=snapshot_id, plugin=identity[1], hook_name=identity[2]).exists():
+            raise ValueError("existing hook has a different ArchiveResult ID")
+        else:
+            pending.append(result)
+
+    imported = snapshot is None
+    if imported:
+        if Snapshot.objects.filter(timestamp=record["timestamp"]).exists():
+            raise ValueError("timestamp belongs to another snapshot")
+        if Snapshot.objects.filter(crawl_id=crawl_id, url=record["url"]).exists():
+            raise ValueError("crawl URL belongs to another snapshot")
+        user, _ = get_user_model().objects.get_or_create(
+            username=owner,
+            defaults={"is_active": False, "password": "!"},
+        )
+        if crawl is None:
+            # Old exports contain only crawl_id. Keep that identity but report
+            # that absent crawl metadata cannot be reconstructed exactly.
+            if not crawl_record:
+                print(f"[yellow]    {directory}: no Crawl record; preserving ID with recovered metadata[/yellow]")
+            crawl = Crawl(
+                id=crawl_id,
+                created_by=user,
+                status="sealed",
+                retry_at=None,
+                **_fields(crawl_record, "urls max_depth config tags_str label"),
+            )
+            crawl.save(force_insert=True)
+            _restore_dates(crawl, crawl_record)
+        snapshot = Snapshot.create_from_directory(directory, crawl=crawl, preserve_identity=True)
+        if snapshot is None:
+            raise ValueError("invalid snapshot metadata")
+        snapshot.status = "sealed"
+        snapshot.retry_at = None
+        if "config" not in record:
+            snapshot.config = {"PERMISSIONS": "private"}
+        if Path(snapshot.get_storage_path_for_version(Snapshot._fs_current_version())) != directory:
+            raise ValueError("metadata does not match its owner/date/domain directory")
+        snapshot.save(force_insert=True)
+        _restore_dates(snapshot, record)
+
+    # Each insert is durable. An interruption after the Snapshot insert is not
+    # completion: the next pass still imports missing result IDs and tags.
+    for result in pending:
+        snapshot._create_archive_result_if_missing(result, {}, preserve_identity=True)
+    tags_before = set(snapshot.tags.values_list("name", flat=True))
+    snapshot._merge_tags_from_index(record)
+    tags_changed = tags_before != set(snapshot.tags.values_list("name", flat=True))
+    link = Path(snapshot.crawl.output_dir) / "snapshots" / snapshot.extract_domain_from_url(snapshot.url) / str(snapshot.id)
+    links_ok = link.is_symlink() and link.resolve() == directory.resolve()
+    if not links_ok:
+        snapshot.reconcile_filesystem_links()
+    if snapshot.fs_migration_needed:
+        snapshot.migrate_filesystem_to_current_version(source_dir=directory)
+    if imported:
+        _restore_dates(snapshot, record)
+    if imported_crawl:
+        _restore_dates(crawl, crawl_record)
+    # Leave source metadata untouched, including process/binary records that
+    # are historical evidence rather than runnable destination process state.
+    return "Imported" if imported else "Repaired" if pending or tags_changed or not links_ok else "Unchanged"
+
+
+def reconcile_known_snapshots(snapshots, wait_for_turn=None):
+    """Explicit metadata / asset repair; normal updates never call this scan."""
+    for snapshot in snapshots.filter(status="sealed").paged_iterator():
+        if wait_for_turn:
+            wait_for_turn()
+        if snapshot.fs_migration_needed:
+            snapshot.migrate_filesystem_to_current_version()
+        directory = Path(snapshot.output_dir)
+        if not directory.is_dir():
+            raise ValueError(f"Missing snapshot directory: {directory}")
+        _reconcile_directory(directory)
+        for result in snapshot.archiveresult_set.all():
+            result.update_output_metadata_from_filesystem(snapshot_dir=directory, full_scan=True)
+
+
+def rescan_snapshots(wait_for_turn=None, reverse=False, scan_legacy=None):
+    stats = dict.fromkeys(("Imported", "Repaired", "Unchanged", "Errors"), 0)
+    print("[*] Rescanning snapshot directories (O(N), orphans first; completed records are reused)...")
+    known = {str(pk) for pk in Snapshot.objects.values_list("pk", flat=True)}
+    # Two streaming passes avoid enumerating the entire archive before the
+    # first import. Rows imported before interruption move to the second pass
+    # next time; only one day's directory names are held in memory at a time.
+    for existing in (False, True):
+        if existing and scan_legacy:
+            scan_legacy()
+        for directory in _snapshot_directories(reverse=reverse):
+            if (directory.name.replace("-", "") in known) != existing:
+                continue
+            if wait_for_turn:
+                wait_for_turn()
+            try:
+                outcome = _reconcile_directory(directory)
+                if directory.name.replace("-", "") in known:
+                    for result in ArchiveResult.objects.filter(snapshot_id=uuid.UUID(directory.name)):
+                        result.update_output_metadata_from_filesystem(snapshot_dir=directory, full_scan=True)
+                stats[outcome] += 1
+                if outcome != "Unchanged":
+                    print(f"    {outcome}: {directory}")
+            except (OSError, ValueError, KeyError, TypeError, ValidationError, IntegrityError) as err:
+                stats["Errors"] += 1
+                print(f"[yellow]    Unresolved: {directory}: {err}[/yellow]")
+            if sum(stats.values()) % 100 == 0:
+                print("    " + ", ".join(f"{key}: {value}" for key, value in stats.items()))
+    print("    " + ", ".join(f"{key}: {value}" for key, value in stats.items()))
+    if stats["Errors"]:
+        raise SystemExit(1)
+    return stats

archivebox/config/common.py

@@ -83,12 +83,16 @@ def base_url_from_legacy_server_config(raw_config: Mapping[str, object]) -> str:
             return candidate
 
     listen_host = str(raw_config.get("LISTEN_HOST") or "").strip()
-    if listen_host and listen_host.lower() != "archivebox.localhost:8000":
+    if listen_host and listen_host.lower() not in {"archivebox.localhost:8000", "archivebox.localhost:5797"}:
         return normalize(listen_host)
 
     csrf_origins = [normalize(origin) for origin in str(raw_config.get("CSRF_TRUSTED_ORIGINS") or "").split(",")]
     csrf_origins = [origin for origin in csrf_origins if origin]
-    nondefault_csrf_origins = [origin for origin in csrf_origins if origin != "http://admin.archivebox.localhost:8000"]
+    nondefault_csrf_origins = [
+        origin
+        for origin in csrf_origins
+        if origin not in {"http://admin.archivebox.localhost:8000", "http://admin.archivebox.localhost:5797"}
+    ]
     if len(nondefault_csrf_origins) == 1:
         return nondefault_csrf_origins[0]
 
@@ -314,7 +318,7 @@ class ServerConfig(BaseConfigSet):
     )
 
     SECRET_KEY: str = Field(default_factory=lambda: "".join(secrets.choice("abcdefghijklmnopqrstuvwxyz0123456789_") for _ in range(50)))
-    BIND_ADDR: str = Field(default="127.0.0.1:8000")
+    BIND_ADDR: str = Field(default="127.0.0.1:5797")
     BASE_URL: str = Field(default="")
     ALLOWED_HOSTS: str = Field(default="*")
     CSRF_TRUSTED_ORIGINS: str = Field(default="")

archivebox/core/admin_snapshots.py

@@ -5,9 +5,10 @@ from functools import lru_cache
 from types import SimpleNamespace
 
 from django.contrib import admin, messages
+from django.core.exceptions import PermissionDenied
 from django.urls import path, reverse
 from django.shortcuts import get_object_or_404, redirect
-from django.http import JsonResponse, HttpResponseBadRequest, HttpResponseNotAllowed
+from django.http import JsonResponse, HttpResponseBadRequest, HttpResponseNotAllowed, Http404
 from django.utils import timezone
 from django.utils.html import format_html, format_html_join
 from django.utils.safestring import mark_safe
@@ -20,6 +21,7 @@ from archivebox.config.common import get_config
 from archivebox.misc.util import htmldecode, urldecode
 from archivebox.misc.paginators import AcceleratedPaginator
 from archivebox.misc.logging_util import printable_filesize
+from archivebox.misc.serve_static import serve_static_with_byterange_support
 from archivebox.search.admin import SearchResultsAdminMixin, SearchResultsChangeList
 from archivebox.search.views import admin_snapshot_search_stream_view
 from archivebox.core.routes_util import build_snapshot_url, build_web_url
@@ -46,6 +48,12 @@ from archivebox.core.widgets import TagEditorWidget, InlineTagEditorWidget
 GLOBAL_CONTEXT = {}
 
 SNAPSHOT_PERMISSION_META = PERMISSIONS_META
+GRID_PREVIEW_OUTPUTS = {
+    ("screenshot", "screenshot.png"),
+    ("chrome_extension_screenshot", "screenshot-1.png"),
+    ("chrome_extension_screenshot", "screenshot.png"),
+    ("favicon", "favicon.ico"),
+}
 
 
 @lru_cache(maxsize=1)
@@ -569,6 +577,11 @@ class SnapshotAdmin(SearchResultsAdminMixin, ConfigEditorMixin, BaseModelAdmin):
         custom_urls = [
             path("grid/", self.admin_site.admin_view(self.grid_view), name="grid"),
             path("search-stream/", self.admin_site.admin_view(self.search_stream_view), name="core_snapshot_search_stream"),
+            path(
+                "<path:object_id>/preview/<str:plugin>/<path:filename>",
+                self.admin_site.admin_view(self.preview_view, cacheable=True),
+                name="core_snapshot_preview",
+            ),
             path("<path:object_id>/redo-failed/", self.admin_site.admin_view(self.redo_failed_view), name="core_snapshot_redo_failed"),
             path(
                 "<path:object_id>/set-permissions/",
@@ -578,6 +591,26 @@ class SnapshotAdmin(SearchResultsAdminMixin, ConfigEditorMixin, BaseModelAdmin):
         ]
         return custom_urls + urls
 
+    def preview_view(self, request, object_id, plugin, filename):
+        """Serve grid thumbnails from the authenticated admin origin."""
+        if (plugin, filename) not in GRID_PREVIEW_OUTPUTS:
+            raise Http404("Unsupported snapshot preview")
+        snapshot = get_object_or_404(Snapshot, pk=object_id)
+        if not self.has_view_or_change_permission(request, snapshot):
+            raise PermissionDenied
+        result = snapshot.archiveresult_set.filter(plugin=plugin, status=ArchiveResult.StatusChoices.SUCCEEDED).first()
+        file_info = (result.output_files or {}).get(filename) if result else None
+        if not isinstance(file_info, dict) or int(file_info.get("size") or 0) <= 0:
+            raise Http404("Snapshot preview does not exist")
+        output_path = filename if file_info.get("root_relative") else f"{plugin}/{filename}"
+        request.archivebox_cache_policy = "private"
+        response = serve_static_with_byterange_support(request, output_path, document_root=snapshot.output_dir)
+        if response.status_code != 304 and not response.headers.get("Content-Type", "").lower().startswith("image/"):
+            raise Http404("Snapshot preview is not an image")
+        response.headers["Vary"] = "Cookie"
+        response.headers["X-Content-Type-Options"] = "nosniff"
+        return response
+
     def search_stream_view(self, request):
         return admin_snapshot_search_stream_view(self, request)
 
@@ -1046,7 +1079,10 @@ class SnapshotAdmin(SearchResultsAdminMixin, ConfigEditorMixin, BaseModelAdmin):
             return None
 
         return format_html(
-            '<img src="{}" alt="{}" class="snapshot-preview {}" decoding="async" loading="lazy" onerror="{}" data-fallbacks="{}">',
+            '<a href="{}" title="Open snapshot details">'
+            '<img src="{}" alt="{}" class="snapshot-preview {}" decoding="async" loading="lazy" onerror="{}" data-fallbacks="{}">'
+            "</a>",
+            build_web_url(f"/{obj.archive_path_from_db}/index.html", request=self.request, config=self.request.archivebox_config),
             preview["img_url"],
             preview["img_alt"],
             preview["preview_class"],

archivebox/core/asgi.py

@@ -7,13 +7,26 @@ For more information on this file, see
 https://docs.djangoproject.com/en/stable/howto/deployment/asgi/
 """
 
-from django.core.asgi import get_asgi_application
+from django.core.handlers.asgi import ASGIHandler
 
 from archivebox.config.django import setup_django
 
 setup_django(check_db=True)
 
-django_application = get_asgi_application()
+
+class ArchiveBoxASGIHandler(ASGIHandler):
+    def create_request(self, scope, body_file):
+        # ASGI already spools the complete body. Django's multipart parser
+        # otherwise treats chunked uploads without Content-Length as empty.
+        if not any(name == b"content-length" for name, _value in scope.get("headers", [])):
+            body_file.seek(0, 2)
+            length = body_file.tell()
+            body_file.seek(0)
+            scope = {**scope, "headers": [*scope.get("headers", []), (b"content-length", str(length).encode("ascii"))]}
+        return super().create_request(scope, body_file)
+
+
+django_application = ArchiveBoxASGIHandler()
 
 
 async def application(scope, receive, send):

archivebox/core/middleware.py

@@ -13,6 +13,7 @@ from django.http import HttpResponseForbidden, HttpResponseNotModified
 from django.shortcuts import redirect
 from django.utils import timezone
 from django.utils.http import http_date
+from django.utils.cache import patch_cache_control, patch_vary_headers
 
 from archivebox.config import VERSION
 from archivebox.config.common import get_config, get_request_config
@@ -123,13 +124,12 @@ def AdminCookieIsolationMiddleware(get_response):
 
 
 def CacheControlMiddleware(get_response):
-    snapshot_path_re = re.compile(r"^/[^/]+/\\d{8}/[^/]+/[0-9a-fA-F-]{8,36}/")
     static_cache_key = (get_COMMIT_HASH() or VERSION or "dev").strip()
 
     def middleware(request):
         response = get_response(request)
 
-        if request.path.startswith("/static/"):
+        if request.path.startswith("/static/") and response.status_code in (200, 304):
             rel_path = request.path[len("/static/") :]
             static_path = finders.find(rel_path)
             if static_path:
@@ -154,15 +154,17 @@ def CacheControlMiddleware(get_response):
                     response.headers["Last-Modified"] = http_date(mtime)
                 return response
 
-        if ("/archive/" in request.path or "/static/" in request.path or snapshot_path_re.match(request.path)) and not response.get(
-            "Cache-Control",
-        ):
-            config = request.__dict__.get("archivebox_config")
-            if config is None:
-                config = get_config(resolve_plugins=False)
-                request.archivebox_config = config
-            policy = "private" if config.PERMISSIONS == "private" else "public"
-            response["Cache-Control"] = f"{policy}, max-age=60, stale-while-revalidate=300"
+        # Keep artifact freshness/validators, but never let a shared cache store
+        # private or unlisted snapshots. Auth redirects and errors stay unstored.
+        snapshot_policy = getattr(request, "archivebox_cache_policy", None)
+        if snapshot_policy and response.status_code in (200, 206, 304):
+            if not response.get("Cache-Control"):
+                response["Cache-Control"] = f"{snapshot_policy}, max-age=60, stale-while-revalidate=300"
+            if snapshot_policy == "private":
+                patch_cache_control(response, private=True)
+        elif snapshot_policy or not response.get("Cache-Control"):
+            response["Cache-Control"] = "private, no-store"
+        patch_vary_headers(response, ("Cookie", "Authorization"))
         return response
 
     return middleware

archivebox/core/models.py

@@ -1394,7 +1394,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
         Uses full hostname with sanitized special chars.
 
         Examples:
-            https://example.com:8080 → example.com_8080
+            https://example.com:5797 → example.com_5797
             https://sub.example.com → sub.example.com
             file:///path → localhost
             data:text/html → data
@@ -1546,7 +1546,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
             return cls.objects.select_related("crawl__created_by").filter(url=url, timestamp=timestamp).first()
 
     @classmethod
-    def create_from_directory(cls, snapshot_dir: Path) -> Optional["Snapshot"]:
+    def create_from_directory(cls, snapshot_dir: Path, *, crawl=None, preserve_identity: bool = False) -> Optional["Snapshot"]:
         """
         Create new Snapshot from orphaned directory.
 
@@ -1618,26 +1618,25 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
             return None
 
         # Ensure uniqueness (reuses existing logic from create_or_update_from_dict)
-        timestamp = cls._ensure_unique_timestamp(url, timestamp)
+        if not preserve_identity:
+            timestamp = cls._ensure_unique_timestamp(url, timestamp)
 
         # Detect version
         fs_version = cls._detect_fs_version_from_index(data)
 
-        system_user_id = get_or_create_system_user_pk()
-        catchall_crawl, _ = Crawl.objects.get_or_create(
-            label="[migration] orphaned snapshots",
-            defaults={
-                "urls": f"# Orphaned snapshot: {url}",
-                "max_depth": 0,
-                "created_by_id": system_user_id,
-            },
-        )
-        if cls.objects.filter(crawl=catchall_crawl, url=url).exists():
+        catchall_crawl = crawl
+        if catchall_crawl is None:
+            system_user_id = get_or_create_system_user_pk()
+            catchall_crawl, _ = Crawl.objects.get_or_create(
+                label="[migration] orphaned snapshots",
+                defaults={"urls": f"# Orphaned snapshot: {url}", "max_depth": 0, "created_by_id": system_user_id},
+            )
+        if not preserve_identity and cls.objects.filter(crawl=catchall_crawl, url=url).exists():
             catchall_crawl = Crawl.objects.create(
                 label=f"[migration] orphaned snapshot {timestamp}",
                 urls=url,
                 max_depth=0,
-                created_by_id=system_user_id,
+                created_by_id=catchall_crawl.created_by_id,
             )
 
         snapshot_kwargs = {
@@ -1647,6 +1646,8 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
             "fs_version": fs_version,
             "crawl": catchall_crawl,
         }
+        if preserve_identity:
+            snapshot_kwargs.update({key: data[key] for key in ("id", "depth", "config", "notes") if key in data})
         try:
             bookmarked_at = parse_date(data.get("bookmarked_at") or timestamp)
         except (TypeError, ValueError, OSError):
@@ -1847,7 +1848,13 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
                         result_data["plugin"] = result_data.get("plugin") or result_data.get("extractor") or plugin
                         self._create_archive_result_if_missing(result_data, existing, update_existing=update_existing)
 
-    def _create_archive_result_if_missing(self, result_data: dict, existing: dict, update_existing: bool = True):
+    def _create_archive_result_if_missing(
+        self,
+        result_data: dict,
+        existing: dict,
+        update_existing: bool = True,
+        preserve_identity: bool = False,
+    ):
         """Create ArchiveResult if not already in DB."""
         from dateutil import parser
         from django.db import transaction
@@ -1925,7 +1932,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
 
         # Machine.current() can probe the host and sanitize config. Do that before
         # atomic() so the transaction below only covers the two related row writes.
-        machine = Machine.current() if cmd or pwd else None
+        machine = Machine.current() if (cmd or pwd) and not preserve_identity else None
         with transaction.atomic():
             if machine is not None:
                 process = Process.objects.create(
@@ -1941,6 +1948,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
                 )
 
             archiveresult = ArchiveResult.objects.create(
+                **({"id": result_data["id"]} if preserve_identity else {}),
                 snapshot=self,
                 plugin=plugin,
                 hook_name=hook_name,
@@ -1954,6 +1962,10 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
                 end_ts=end_ts,
                 process=process,
             )
+            if preserve_identity:
+                dates = {key: parse_date(result_data[key]) for key in ("created_at", "modified_at") if result_data.get(key)}
+                if dates:
+                    ArchiveResult.objects.filter(pk=archiveresult.pk).update(**dates)
         existing[(plugin, hook_name)] = archiveresult
 
     def write_index_json(self):
@@ -2010,7 +2022,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
         # without a corresponding DB row can be retained byte-for-byte.
         binaries_seen = set()
         processes_seen = set()
-        records = [self.to_json()]
+        records = [self.to_json(), self.crawl.to_json()]
         for ar in archive_results:
             process = ar.process_record
             if process and process.binary and process.binary_id not in binaries_seen:
@@ -2837,6 +2849,7 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
         Includes all fields needed to fully reconstruct/identify this snapshot.
         """
         from archivebox.config import VERSION
+        from archivebox.config.common import redact_sensitive_config
 
         archive_size = self.archive_size
 
@@ -2850,6 +2863,9 @@ class Snapshot(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelW
             "tags": self.tags_str(),
             "bookmarked_at": self.bookmarked_at.isoformat() if self.bookmarked_at else None,
             "created_at": self.created_at.isoformat() if self.created_at else None,
+            "modified_at": self.modified_at.isoformat() if self.modified_at else None,
+            "config": redact_sensitive_config(self.config),
+            "notes": self.notes,
             "timestamp": self.timestamp,
             "depth": self.depth,
             "status": self.status,
@@ -4009,6 +4025,8 @@ class ArchiveResult(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithNotes):
             "schema_version": VERSION,
             "id": str(self.id),
             "snapshot_id": str(self.snapshot_id),
+            "created_at": self.created_at.isoformat() if self.created_at else None,
+            "modified_at": self.modified_at.isoformat() if self.modified_at else None,
             "plugin": self.plugin,
             "hook_name": self.hook_name,
             "status": self.status,
@@ -4222,7 +4240,7 @@ class ArchiveResult(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithNotes):
     def output_file_paths(self) -> list[str]:
         return list(self.output_file_map().keys())
 
-    def update_output_metadata_from_filesystem(self, snapshot_dir: Path | None = None, save: bool = True) -> bool:
+    def update_output_metadata_from_filesystem(self, snapshot_dir: Path | None = None, save: bool = True, full_scan: bool = False) -> bool:
         from abx_dl.output_files import OutputManifest, output_file_from_path
 
         if self.plugin == "title":
@@ -4262,7 +4280,7 @@ class ArchiveResult(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithNotes):
                 add_file(snapshot_dir / raw_output, raw_output, root_relative=True)
 
         plugin_dir = snapshot_dir / self.plugin
-        if not output_files and plugin_dir.is_dir():
+        if (full_scan or not output_files) and plugin_dir.is_dir():
             output_files = OutputManifest.scan(plugin_dir, containment_root=snapshot_dir).as_mapping()
 
         if not output_files:

archivebox/core/recovery_util.py

@@ -140,6 +140,13 @@ def recover_orchestrator_state(*, include_chrome: bool = False, crawl_id: str |
             orphaned_hook_processes = orphaned_hook_processes.filter(snapshot_pwd_filter)
         else:
             orphaned_hook_processes = orphaned_hook_processes.none()
+    else:
+        # Global startup recovery must stay database-only. Process.pwd points
+        # into archive payload storage, which may be a cold FUSE mount; scanning
+        # every historical unlinked hook here blocks the sole scheduler before
+        # it can claim newly queued work. Explicit crawl recovery below remains
+        # available for the work the caller asked to resume.
+        orphaned_hook_processes = orphaned_hook_processes.none()
     for process in orphaned_hook_processes.only(
         "id",
         "pwd",

archivebox/core/routes_util.py

@@ -74,7 +74,7 @@ def derive_base_url_from_csrf(config: dict[str, Any] | None = None, **config_kwa
     had ``CSRF_TRUSTED_ORIGINS=https://archive.example.com`` set (required for
     admin login to work). On upgrade, ``BASE_URL`` is the new knob — but it
     defaults to empty, and falling through to ``BIND_ADDR`` produces an
-    unreachable URL like ``http://0.0.0.0:8000``. If the user has exactly one
+    unreachable URL like ``http://0.0.0.0:5797``. If the user has exactly one
     CSRF origin we treat it as the implicit BASE_URL so links/redirects keep
     pointing at the public hostname they already configured.
 
@@ -133,7 +133,7 @@ def canonical_base_host_for_request(request_host: str) -> str:
     """Strip role subdomains and remap loopback hostnames to ``archivebox.localhost``.
 
     Used by the banner suggestion and the in-browser pin endpoint: when the
-    user is hitting the server on raw ``localhost:9292`` or ``127.0.0.1:9292``
+    user is hitting the server on raw ``localhost:5797`` or ``127.0.0.1:5797``
     we want to suggest the wildcard-friendly ``archivebox.localhost`` family
     instead, so the eventual pinned ``BASE_URL`` plays nicely with subdomain
     routing without forcing the user to add a /etc/hosts entry.

archivebox/core/settings.py

@@ -87,6 +87,7 @@ DJANGO_OBJECT_ACTIONS_DEFAULT_HTTP_METHOD = "POST"
 MIDDLEWARE = [
     "archivebox.core.middleware.TimezoneMiddleware",
     "django.middleware.security.SecurityMiddleware",
+    "archivebox.core.middleware.CacheControlMiddleware",
     "archivebox.core.middleware.AdminCookieIsolationMiddleware",
     "django.contrib.sessions.middleware.SessionMiddleware",
     "django.middleware.common.CommonMiddleware",
@@ -97,7 +98,6 @@ MIDDLEWARE = [
     "archivebox.core.middleware.ServerSecurityModeMiddleware",
     "archivebox.core.middleware.HostRoutingMiddleware",
     "django.contrib.messages.middleware.MessageMiddleware",
-    "archivebox.core.middleware.CacheControlMiddleware",
     # Additional middlewares from plugins (if any)
 ]
 
@@ -390,7 +390,7 @@ SECURE_REFERRER_POLICY = "strict-origin-when-cross-origin"
 # get_base_url() also covers deployments that only set CSRF_TRUSTED_ORIGINS (the
 # implicit-BASE_URL fallback used on 0.7.x->0.9.x upgrades), so HTTPS hardening
 # isn't lost until BASE_URL is migrated. An explicit plain-http base (e.g. local
-# http://archivebox.localhost:8000) disables proxy HTTPS handling.
+# http://archivebox.localhost:5797) disables proxy HTTPS handling.
 BASE_URL_IS_HTTPS = get_base_url(config=CONFIG).strip().lower().startswith("https://")
 if BASE_URL_IS_HTTPS or not CONFIG.BASE_URL:
     SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")

archivebox/core/templatetags/core_tags.py

@@ -8,6 +8,7 @@ from urllib.parse import quote, urlparse
 from abx_plugins.plugins.archivewebpage.replay_preview import is_replay_target as is_archivewebpage_replay_target
 from django import template
 from django.templatetags.static import static
+from django.urls import reverse
 from django.utils import timezone
 from django.utils.html import escape
 from django.utils.safestring import mark_safe
@@ -460,9 +461,9 @@ def system_warnings_banner(context):
     """Render the top-of-page warning banner for one of the conditions below,
     in priority order (highest first):
 
-    1. ``mode="unconfigured"``— ``BASE_URL`` is empty. Security/correctness
-       issue: until it's pinned, generated URLs can echo any Host the client
-       sends, and admin/web/api routing has no canonical anchor.
+    1. ``mode="unconfigured"``— ``BASE_URL`` is empty and the operator has
+       not selected safe single-domain routing. That mode intentionally uses
+       each device's connection address, including LAN and Tailscale IPs.
     2. ``mode="base_url_mismatch"`` — the browser reached ArchiveBox through
        an origin that differs from the configured canonical ``BASE_URL``.
     3. ``mode="unsafe"``      — ``SERVER_SECURITY_MODE`` explicitly enables
@@ -489,7 +490,7 @@ def system_warnings_banner(context):
 
         config = get_config(resolve_plugins=False)
 
-    if not config.BASE_URL:
+    if not config.BASE_URL and config.SERVER_SECURITY_MODE != "safe-onedomain-nojsreplay":
         return get_setup_wizard_context(context.get("request"), config)
     mismatch = get_base_url_mismatch_context(context.get("request"), config)
     if mismatch and not _is_web_subdomain_alias(mismatch["browser_url"], mismatch["configured_base_url"]):
@@ -613,6 +614,13 @@ def snapshot_archiveresult_url(context, snapshot, plugin: str, filename: str) ->
     return ""
 
 
+@register.simple_tag(takes_context=True)
+def admin_snapshot_archiveresult_url(context, snapshot, plugin: str, filename: str) -> str:
+    if not snapshot_archiveresult_url(context, snapshot, plugin, filename):
+        return ""
+    return reverse("admin:core_snapshot_preview", args=(snapshot.pk, plugin, filename))
+
+
 @register.simple_tag(takes_context=True)
 def snapshot_index_row(context, link) -> str:
     snapshot_base = _snapshot_base_url_for_context(context, link)

archivebox/core/views.py

@@ -23,6 +23,7 @@ from django.http import Http404, HttpRequest, HttpResponse, HttpResponseForbidde
 from django.shortcuts import redirect, render
 from django.utils.decorators import method_decorator
 from django.utils.html import format_html, format_html_join
+from django.utils.cache import patch_cache_control, patch_vary_headers
 from django.utils.safestring import mark_safe
 from django.views import View
 from django.views.decorators.csrf import csrf_exempt, csrf_protect
@@ -327,6 +328,7 @@ class SnapshotView(View):
 
     @staticmethod
     def render_live_index(request, snapshot):
+        request.archivebox_cache_policy = "public" if snapshot.permissions == PERMISSIONS_PUBLIC else "private"
         return render(
             template_name="core/snapshot.html",
             request=request,
@@ -350,6 +352,7 @@ class SnapshotView(View):
             try:
                 try:
                     snapshot = Snapshot.objects.get(Q(timestamp=slug) | Q(id__startswith=slug))
+                    request.archivebox_cache_policy = "public" if snapshot.permissions == PERMISSIONS_PUBLIC else "private"
                     if not can_view_snapshot(request, snapshot):
                         return _private_snapshot_auth_redirect(request, snapshot, archivefile or "index.html")
                     canonical_base = snapshot.url_path
@@ -629,6 +632,7 @@ class SnapshotPathView(View):
                 status=404,
             )
 
+        request.archivebox_cache_policy = "public" if snapshot.permissions == PERMISSIONS_PUBLIC else "private"
         canonical_base = snapshot.url_path
         if date:
             requested_base = f"{username}/{date}/{domain or url or ''}"
@@ -891,7 +895,7 @@ def _serve_responses_path(request, responses_root: Path, rel_path: str, show_ind
     return None
 
 
-def _serve_snapshot_replay(request: HttpRequest, snapshot: Snapshot, path: str = ""):
+def _build_snapshot_replay_response(request: HttpRequest, snapshot: Snapshot, path: str = ""):
     rel_path = path or ""
     request_config = get_request_config(
         request,
@@ -899,6 +903,7 @@ def _serve_snapshot_replay(request: HttpRequest, snapshot: Snapshot, path: str =
     )
     request.archivebox_config = request_config
     request.archivebox_snapshot_url = snapshot.url
+    request.archivebox_cache_policy = "public" if snapshot.permissions == PERMISSIONS_PUBLIC else "private"
     snapshot._runtime_config = request_config
 
     if rel_path.startswith("replay/") or rel_path == "replay":
@@ -911,7 +916,7 @@ def _serve_snapshot_replay(request: HttpRequest, snapshot: Snapshot, path: str =
         # /progress.json on through to the same view used everywhere else. The caller
         # passes snapshot_id explicitly in the query string — we don't read it from the
         # subdomain (this keeps the endpoint identical across all security modes).
-        return live_progress_view(request)
+        return live_progress_view(request, authorized_snapshot=snapshot)
 
     is_directory_request = bool(path) and path.endswith("/")
     show_indexes = bool(request.GET.get("files")) or (request_config.USES_SUBDOMAIN_ROUTING and is_directory_request)
@@ -954,6 +959,20 @@ def _serve_snapshot_replay(request: HttpRequest, snapshot: Snapshot, path: str =
     raise Http404
 
 
+def _serve_snapshot_replay(request: HttpRequest, snapshot: Snapshot, path: str = ""):
+    response = _build_snapshot_replay_response(request, snapshot, path)
+    if path != "progress.json" and response.status_code in (200, 206, 304):
+        if not response.get("Cache-Control"):
+            policy = "public" if snapshot.permissions == PERMISSIONS_PUBLIC else "private"
+            response.headers["Cache-Control"] = f"{policy}, max-age=60, stale-while-revalidate=300"
+        if snapshot.permissions != PERMISSIONS_PUBLIC:
+            patch_cache_control(response, private=True)
+    else:
+        response.headers["Cache-Control"] = "private, no-store"
+    patch_vary_headers(response, ("Cookie", "Authorization"))
+    return response
+
+
 def _serve_original_domain_replay(request: HttpRequest, domain: str, path: str = ""):
     request_config = get_request_config(request, resolve_plugins=False)
     request.archivebox_config = request_config
@@ -977,6 +996,8 @@ def _serve_original_domain_replay(request: HttpRequest, domain: str, path: str =
 
     root_match = (match[0], match[1]) if match else _latest_responses_root(snapshots, domain)
     responses_root = root_match[1] if root_match else None
+    if root_match:
+        request.archivebox_cache_policy = "public" if root_match[0].permissions == PERMISSIONS_PUBLIC else "private"
     if request_config.USES_SUBDOMAIN_ROUTING:
         snapshot = root_match[0] if root_match else (snapshots[0] if requested_root_index and snapshots else None)
         if snapshot:

archivebox/crawls/models.py

@@ -467,6 +467,8 @@ class Crawl(ModelWithDeleteAfter, ModelWithOutputDir, ModelWithConfig, ModelWith
             "tags_str": self.tags_str,
             "label": self.label,
             "created_at": self.created_at.isoformat() if self.created_at else None,
+            "modified_at": self.modified_at.isoformat() if self.modified_at else None,
+            "created_by": self.created_by.username,
         }
 
     @staticmethod

archivebox/machine/detect.py

@@ -88,7 +88,7 @@ def get_vm_info():
         try:
             # Getting SMBIOS data from sysfs.
             # SMBIOS 2.8 present.
-            # argo-1        | 2024-10-01T10:40:51Z ERR  error="Incoming request ended abruptly: context canceled" connIndex=2 event=1 ingressRule=0 originService=http://archivebox:8000                                                                               │
+            # argo-1        | 2024-10-01T10:40:51Z ERR  error="Incoming request ended abruptly: context canceled" connIndex=2 event=1 ingressRule=0 originService=http://archivebox:5797                                                                               │
             # Handle 0x0100, DMI type 1, 27 bytes
             # System Information
             #         Manufacturer: DigitalOcean

archivebox/misc/serve_static.py

@@ -129,7 +129,7 @@ def _resolve_archive_path(document_root: str | Path, rel_path: str) -> tuple[Pat
 
 def _cache_policy(config=None, **config_kwargs) -> str:
     config = config or get_config(resolve_plugins=False, **config_kwargs)
-    return "private" if config.PERMISSIONS == "private" else "public"
+    return "public" if config.PERMISSIONS == "public" else "private"
 
 
 def _format_direntry_timestamp(stat_result: os.stat_result) -> str:
@@ -605,9 +605,15 @@ def _apply_transformed_html_preview_style(html_text: str) -> str:
     return f"{TRANSFORMED_HTML_PREVIEW_STYLE}\n{html_text}"
 
 
-def _set_transformed_response_headers(response, fullpath: Path, statobj: os.stat_result, encoding: str | None, config) -> None:
+def _set_transformed_response_headers(
+    response,
+    fullpath: Path,
+    statobj: os.stat_result,
+    encoding: str | None,
+    cache_policy: str,
+) -> None:
     response.headers["Last-Modified"] = http_date(statobj.st_mtime)
-    response.headers["Cache-Control"] = f"{_cache_policy(config=config)}, max-age=60, stale-while-revalidate=300"
+    response.headers["Cache-Control"] = f"{cache_policy}, max-age=60, stale-while-revalidate=300"
     response.headers["Content-Disposition"] = f'inline; filename="{fullpath.name}"'
     if encoding:
         response.headers["Content-Encoding"] = encoding
@@ -858,19 +864,22 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
     config = request.__dict__.get("archivebox_config")
     if config is None:
         config = get_config(resolve_plugins=False)
+    cache_policy = getattr(request, "archivebox_cache_policy", None) or _cache_policy(config=config)
     fullpath, path = _resolve_archive_path(document_root, path)
     if os.access(fullpath, os.R_OK) and fullpath.is_dir():
         if request.GET.get("download") == "zip" and show_indexes:
-            return _build_directory_zip_response(
+            response = _build_directory_zip_response(
                 fullpath,
                 path,
                 is_archive_replay=is_archive_replay,
                 use_async_stream=_is_asgi_request(request),
                 config=config,
             )
+            response.headers["Cache-Control"] = f"{cache_policy}, max-age=60, stale-while-revalidate=300"
+            return response
         if show_indexes:
             response = _render_directory_index(request, path, fullpath)
-            response.headers["Cache-Control"] = f"{_cache_policy(config=config)}, max-age=60, stale-while-revalidate=300"
+            response.headers["Cache-Control"] = f"{cache_policy}, max-age=60, stale-while-revalidate=300"
             response.headers["Last-Modified"] = http_date(fullpath.stat().st_mtime)
             return _apply_archive_replay_headers(
                 response,
@@ -899,7 +908,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
             if etag in inm_list or etag.strip('"') in [i.strip('"') for i in inm_list]:
                 not_modified = HttpResponseNotModified()
                 not_modified.headers["ETag"] = etag
-                not_modified.headers["Cache-Control"] = f"{_cache_policy(config=config)}, max-age=31536000, immutable"
+                not_modified.headers["Cache-Control"] = f"{cache_policy}, max-age=31536000, immutable"
                 not_modified.headers["Last-Modified"] = http_date(statobj.st_mtime)
                 return _apply_archive_replay_headers(
                     not_modified,
@@ -940,8 +949,10 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
         request.META.get("HTTP_IF_MODIFIED_SINCE"),
         statobj.st_mtime,
     ):
+        not_modified = HttpResponseNotModified()
+        not_modified.headers["Cache-Control"] = f"{cache_policy}, max-age=60, stale-while-revalidate=300"
         return _apply_archive_replay_headers(
-            HttpResponseNotModified(),
+            not_modified,
             fullpath=fullpath,
             content_type=content_type,
             is_archive_replay=is_archive_replay,
@@ -956,7 +967,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                 decoded = fullpath.read_text(encoding="utf-8", errors="replace")
                 wrapped = _render_text_preview_document(decoded, fullpath.name)
                 response = HttpResponse(wrapped, content_type="text/html; charset=utf-8")
-                _set_transformed_response_headers(response, fullpath, statobj, encoding, config)
+                _set_transformed_response_headers(response, fullpath, statobj, encoding, cache_policy)
                 return _apply_archive_replay_headers(
                     response,
                     fullpath=fullpath,
@@ -976,7 +987,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                 raw_image_url = f"{raw_image_url}?{urlencode(list(preview_query.lists()), doseq=True)}"
             wrapped = _render_image_preview_document(raw_image_url, fullpath.name)
             response = HttpResponse(wrapped, content_type="text/html; charset=utf-8")
-            _set_transformed_response_headers(response, fullpath, statobj, encoding, config)
+            _set_transformed_response_headers(response, fullpath, statobj, encoding, cache_policy)
             return _apply_archive_replay_headers(
                 response,
                 fullpath=fullpath,
@@ -1002,9 +1013,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                 last_modified=http_date(statobj.st_mtime),
                 etag=etag or "",
                 cache_control=(
-                    f"{_cache_policy(config=config)}, max-age=31536000, immutable"
-                    if etag
-                    else f"{_cache_policy(config=config)}, max-age=60, stale-while-revalidate=300"
+                    f"{cache_policy}, max-age=31536000, immutable" if etag else f"{cache_policy}, max-age=60, stale-while-revalidate=300"
                 ),
                 content_encoding=encoding or "",
             )
@@ -1042,7 +1051,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                     wrapped, _rewrite_count = _rewrite_html_image_sources_for_request(request, wrapped, document_root, rel_path)
                     wrapped = _apply_transformed_html_preview_style(wrapped)
                     response = HttpResponse(wrapped, content_type="text/html; charset=utf-8")
-                    _set_transformed_response_headers(response, fullpath, statobj, encoding, config)
+                    _set_transformed_response_headers(response, fullpath, statobj, encoding, cache_policy)
                     return _apply_archive_replay_headers(
                         response,
                         fullpath=fullpath,
@@ -1053,7 +1062,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                 if rewritten_count:
                     rewritten_html = _apply_transformed_html_preview_style(rewritten_html)
                     response = HttpResponse(rewritten_html, content_type=content_type)
-                    _set_transformed_response_headers(response, fullpath, statobj, encoding, config)
+                    _set_transformed_response_headers(response, fullpath, statobj, encoding, cache_policy)
                     return _apply_archive_replay_headers(
                         response,
                         fullpath=fullpath,
@@ -1064,7 +1073,7 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
                 if escaped_count and escaped_count > tag_count * 2:
                     decoded = _apply_transformed_html_preview_style(decoded)
                     response = HttpResponse(decoded, content_type=content_type)
-                    _set_transformed_response_headers(response, fullpath, statobj, encoding, config)
+                    _set_transformed_response_headers(response, fullpath, statobj, encoding, cache_policy)
                     return _apply_archive_replay_headers(
                         response,
                         fullpath=fullpath,
@@ -1084,13 +1093,13 @@ def serve_static_with_byterange_support(request, path, document_root=None, show_
     response.headers["Last-Modified"] = http_date(statobj.st_mtime)
     if etag:
         response.headers["ETag"] = etag
-        response.headers["Cache-Control"] = f"{_cache_policy(config=config)}, max-age=31536000, immutable"
+        response.headers["Cache-Control"] = f"{cache_policy}, max-age=31536000, immutable"
     else:
-        response.headers["Cache-Control"] = f"{_cache_policy(config=config)}, max-age=60, stale-while-revalidate=300"
+        response.headers["Cache-Control"] = f"{cache_policy}, max-age=60, stale-while-revalidate=300"
     if is_text_like:
         response.headers["Content-Disposition"] = f'inline; filename="{fullpath.name}"'
     if content_type.startswith("image/"):
-        response.headers["Cache-Control"] = "public, max-age=604800, immutable"
+        response.headers["Cache-Control"] = f"{cache_policy}, max-age=604800, immutable"
 
     # handle byte-range requests by serving chunk of file
     if stat.S_ISREG(statobj.st_mode):

archivebox/personas/admin.py

@@ -27,7 +27,7 @@ class PersonaAdmin(ConfigEditorMixin, BaseModelAdmin):
             "Persona",
             {
                 "fields": ("name", "created_by", "permissions"),
-                "classes": ("card", "persona-card-primary"),
+                "classes": ("card", "wide", "persona-card-primary"),
             },
         ),
         (
@@ -42,7 +42,7 @@ class PersonaAdmin(ConfigEditorMixin, BaseModelAdmin):
                     "import_extract_cookies",
                     "import_capture_storage",
                 ),
-                "classes": ("card", "wide"),
+                "classes": ("card", "wide", "persona-card-import"),
             },
         ),
         (

archivebox/personas/forms.py

@@ -55,7 +55,7 @@ class PersonaAdminForm(PluginConfigFormMixin, forms.ModelForm):
         label="Autodiscovered profiles",
         widget=forms.RadioSelect,
         choices=(),
-        help_text="Detected from local Chrome, Chromium, Brave, and Edge profile roots.",
+        help_text="Detected from local Chrome, Chrome Beta, Chromium, Brave, and Edge profile roots.",
     )
     import_source = forms.CharField(
         required=False,

archivebox/personas/importers.py

@@ -18,6 +18,7 @@ import subprocess
 import sys
 import tempfile
 from dataclasses import dataclass, field
+from functools import partial
 from pathlib import Path
 from typing import TYPE_CHECKING
 from urllib.parse import urlparse
@@ -31,6 +32,7 @@ if TYPE_CHECKING:
 
 BROWSER_LABELS = {
     "chrome": "Google Chrome",
+    "chrome-beta": "Google Chrome Beta",
     "chromium": "Chromium",
     "brave": "Brave",
     "edge": "Microsoft Edge",
@@ -216,24 +218,29 @@ class PersonaImportResult:
         return self.profile_copied or self.cookies_imported or self.storage_captured or self.user_agent_imported
 
 
-def get_chrome_user_data_dir() -> Path | None:
+def get_chrome_user_data_dir(*, beta: bool = False) -> Path | None:
     """Get the default Chrome user data directory for the current platform."""
     system = platform.system()
     home = Path.home()
+    app_name = "Chrome Beta" if beta else "Chrome"
 
     if system == "Darwin":
         candidates = [
-            home / "Library" / "Application Support" / "Google" / "Chrome",
+            home / "Library" / "Application Support" / "Google" / app_name,
         ]
     elif system == "Linux":
-        candidates = [
-            home / ".config" / "google-chrome",
-            home / ".config" / "chrome",
-        ]
+        candidates = (
+            [home / ".config" / "google-chrome-beta"]
+            if beta
+            else [
+                home / ".config" / "google-chrome",
+                home / ".config" / "chrome",
+            ]
+        )
     elif system == "Windows":
         local_app_data = Path(os.environ.get("LOCALAPPDATA", home / "AppData" / "Local"))
         candidates = [
-            local_app_data / "Google" / "Chrome" / "User Data",
+            local_app_data / "Google" / app_name / "User Data",
         ]
     else:
         candidates = []
@@ -320,6 +327,7 @@ def get_edge_user_data_dir() -> Path | None:
 
 BROWSER_PROFILE_FINDERS = {
     "chrome": get_chrome_user_data_dir,
+    "chrome-beta": partial(get_chrome_user_data_dir, beta=True),
     "chromium": get_chromium_user_data_dir,
     "brave": get_brave_user_data_dir,
     "edge": get_edge_user_data_dir,
@@ -354,6 +362,12 @@ def validate_persona_name(name: str) -> tuple[bool, str]:
 
 
 def discover_local_browser_profiles() -> list[PersonaImportSource]:
+    # The macOS companion exports with the host Keychain before creating personas.
+    # Raw mounts alone cannot decrypt macOS cookies in the Linux container.
+    if host_profiles := os.environ.get("ARCHIVEBOX_HOST_BROWSER_PROFILES"):
+        host_templates = discover_persona_template_profiles(Path(host_profiles))
+        host_names = {source.source_name for source in host_templates}
+        return host_templates + [source for source in discover_persona_template_profiles() if source.source_name not in host_names]
     discovered: list[PersonaImportSource] = []
 
     for browser, finder in BROWSER_PROFILE_FINDERS.items():
@@ -430,6 +444,11 @@ def discover_persona_template_profiles(personas_dir: Path | None = None) -> list
 
 def resolve_browser_import_source(browser: str, profile_dir: str | None = None) -> PersonaImportSource:
     browser = browser.lower().strip()
+    if host_profiles := os.environ.get("ARCHIVEBOX_HOST_BROWSER_PROFILES"):
+        label = BROWSER_LABELS.get(browser, browser)
+        exported = Path(host_profiles) / f"{label} - {profile_dir or 'Default'}"
+        if exported.is_dir():
+            return resolve_custom_import_source(str(exported))
     if browser not in BROWSER_PROFILE_FINDERS:
         supported = ", ".join(BROWSER_PROFILE_FINDERS)
         raise ValueError(f"Unknown browser: {browser}. Supported browsers: {supported}")
@@ -547,6 +566,7 @@ def resolve_source_browser_binary(source: PersonaImportSource) -> str:
         return str(Path(source.browser_binary).expanduser().resolve())
     names = {
         "chrome": ("Google Chrome", "google-chrome"),
+        "chrome-beta": ("Google Chrome Beta", "google-chrome-beta"),
         "chromium": ("Chromium", "chromium"),
         "brave": ("Brave Browser", "brave-browser"),
         "edge": ("Microsoft Edge", "microsoft-edge"),
@@ -561,6 +581,7 @@ def resolve_source_browser_binary(source: PersonaImportSource) -> str:
     elif platform.system() == "Windows":
         relative = {
             "chrome": "Google/Chrome/Application/chrome.exe",
+            "chrome-beta": "Google/Chrome Beta/Application/chrome.exe",
             "edge": "Microsoft/Edge/Application/msedge.exe",
             "brave": "BraveSoftware/Brave-Browser/Application/brave.exe",
             "chromium": "Chromium/Application/chrome.exe",

archivebox/plugins/forms.py

@@ -213,7 +213,7 @@ class PluginConfigFormMixin:
                     "enabled_config_key": enabled_config_key,
                     "description": str(schema.get("description") or "").strip(),
                     "source_url": f"https://github.com/ArchiveBox/abx-plugins/tree/main/abx_plugins/plugins/{plugin_name}",
-                    "docs_url": f"https://archivebox.github.io/abx-plugins/#{plugin_name}",
+                    "docs_url": f"https://plugins.archivebox.io/#{plugin_name}",
                     "required_plugins": [str(item) for item in schema.get("required_plugins") or []],
                     "required_binary_links": _build_required_binary_links(
                         schema.get("required_binaries") or [],

archivebox/plugins/views.py

@@ -17,7 +17,7 @@ from archivebox.config.common import get_live_config_url
 from archivebox.config.views import get_environment_binary_url, is_superuser
 from archivebox.plugins.discovery import BUILTIN_PLUGINS_DIR, USER_PLUGINS_DIR, discover_plugin_configs, iter_plugin_dirs
 
-ABX_PLUGINS_DOCS_BASE_URL = "https://archivebox.github.io/abx-plugins/"
+ABX_PLUGINS_DOCS_BASE_URL = "https://plugins.archivebox.io/"
 ABX_PLUGINS_GITHUB_BASE_URL = "https://github.com/ArchiveBox/abx-plugins/tree/main/abx_plugins/plugins/"
 LIVE_PLUGIN_BASE_URL = "/admin/environment/plugins/"
 

archivebox/progressmonitor/views.py

@@ -43,7 +43,7 @@ def _live_progress_plugin_names() -> tuple[frozenset[str], frozenset[str]]:
     return download_plugin_names, indexing_plugin_names
 
 
-def live_progress_view(request):
+def live_progress_view(request, *, authorized_snapshot=None):
     """Simple JSON endpoint for live progress status - used by admin progress monitor."""
     try:
         from archivebox.core.models import ArchiveResult, Snapshot
@@ -63,7 +63,10 @@ def live_progress_view(request):
             except (TypeError, ValueError):
                 return JsonResponse({"error": "Invalid snapshot_id"}, status=400)
             scoped_snapshot = Snapshot.objects.filter(id=snapshot_id_filter).select_related("crawl").first()
-            if scoped_snapshot is None or not can_view_snapshot(request, scoped_snapshot):
+            authorized_snapshot_id = str(getattr(authorized_snapshot, "id", "")).replace("-", "")
+            if scoped_snapshot is None or (
+                authorized_snapshot_id != str(scoped_snapshot.id).replace("-", "") and not can_view_snapshot(request, scoped_snapshot)
+            ):
                 return JsonResponse({"error": "Permission denied"}, status=403)
         elif crawl_id_filter:
             # Crawl-only scope still requires staff: there's no per-crawl ACL helper,
@@ -221,6 +224,8 @@ def live_progress_view(request):
 
         api_base = get_api_base_url(request=request, config=request_config) if scoped_snapshot is not None else ""
 
+        running_screencasts = {}
+
         def screencast_frame_url(crawl_id: str, crawl_dir: Path) -> str:
             frame_path = crawl_dir / "chrome_screencast" / "latest.jpg"
             try:
@@ -229,7 +234,8 @@ def live_progress_view(request):
                 return ""
             if frame_stat.st_size <= 0:
                 return ""
-            if now.timestamp() - frame_stat.st_mtime > 15:
+            started_at = running_screencasts.get(crawl_id)
+            if started_at is None or frame_stat.st_mtime < started_at.timestamp():
                 return ""
             rel = f"/api/v1/crawls/crawl/{crawl_id}/files/chrome_screencast/latest.jpg?v={frame_stat.st_mtime_ns}"
             return f"{api_base}{rel}" if api_base else rel
@@ -475,7 +481,10 @@ def live_progress_view(request):
         detailed_snapshot_ids = [snapshot["id"] for snapshot in snapshots if snapshot["status"] != Snapshot.StatusChoices.QUEUED]
         process_value_fields = ("id", "process_type", "status", "pwd", "cmd", "pid", "exit_code", "started_at", "ended_at", "modified_at")
         if active_crawl_ids or displayed_snapshot_ids:
+            relevant_process_ids = tuple(dict.fromkeys([*active_crawl_ids, *displayed_snapshot_ids]))
+            relevant_process_path_pattern = "|".join(relevant_process_ids)
             process_scope = Process.objects.filter(
+                pwd__regex=relevant_process_path_pattern,
                 machine_id=machine_id,
                 process_type__in=[
                     Process.TypeChoices.HOOK,
@@ -552,6 +561,10 @@ def live_progress_view(request):
                 snapshot_id = str(matched_snapshot["id"])
             running_worker_ids.add(str(proc["id"]))
             _plugin, _label, phase, _hook_name = process_label(proc["cmd"])
+            if _plugin == "chrome_screencast" and proc["started_at"] and Process(**proc, machine_id=machine_id).is_running:
+                # CDP sends frames only when the page changes. Keep a still
+                # frame visible for its live capture, never for a previous run.
+                running_screencasts[crawl_id] = max(proc["started_at"], running_screencasts.get(crawl_id, proc["started_at"]))
             if crawl_id and proc["pid"]:
                 crawl_process_pids.setdefault(crawl_id, proc["pid"])
             if phase == "snapshot" and snapshot_id and proc["pid"]:

archivebox/services/runner.py

@@ -1643,11 +1643,19 @@ def run_install(*, plugin_names: list[str] | None = None) -> None:
     asyncio.run(_run_install(plugin_names=plugin_names))
 
 
-def _first_due_id(queryset):
-    return queryset.order_by("retry_at", "created_at").values_list("id", flat=True).first()
+def _first_due_id(queryset, *, newest_first: bool = False):
+    ordering = ("-retry_at", "-created_at") if newest_first else ("retry_at", "created_at")
+    return queryset.order_by(*ordering).values_list("id", flat=True).first()
 
 
-def _run_due_crawl_status(status: str, *, crawl_id: str | None, lock_seconds: int, interactive_interrupts: bool) -> bool:
+def _run_due_crawl_status(
+    status: str,
+    *,
+    crawl_id: str | None,
+    lock_seconds: int,
+    interactive_interrupts: bool,
+    newest_first: bool = False,
+) -> bool:
     from archivebox.crawls.models import Crawl
 
     due_crawls = Crawl.objects.filter(
@@ -1656,7 +1664,7 @@ def _run_due_crawl_status(status: str, *, crawl_id: str | None, lock_seconds: in
     )
     if crawl_id:
         due_crawls = due_crawls.filter(id=crawl_id)
-    due_crawl_id = _first_due_id(due_crawls)
+    due_crawl_id = _first_due_id(due_crawls, newest_first=newest_first)
     if due_crawl_id is None:
         return False
     due_crawl = Crawl.objects.filter(id=due_crawl_id).first()
@@ -1670,8 +1678,15 @@ def _run_due_crawl_status(status: str, *, crawl_id: str | None, lock_seconds: in
     return True
 
 
-def _run_due_snapshot_query(queryset, *, lock_seconds: int, interactive_interrupts: bool, runtime_config) -> bool:
-    due_snapshot_id = _first_due_id(queryset)
+def _run_due_snapshot_query(
+    queryset,
+    *,
+    lock_seconds: int,
+    interactive_interrupts: bool,
+    runtime_config,
+    newest_first: bool = False,
+) -> bool:
+    due_snapshot_id = _first_due_id(queryset, newest_first=newest_first)
     return _run_due_snapshot_id(
         due_snapshot_id,
         lock_seconds=lock_seconds,
@@ -1779,20 +1794,31 @@ def run_pending_crawls(
             )
             if crawl_id:
                 active_snapshots = active_snapshots.filter(crawl_id=crawl_id)
+            # Admit new submissions between captures instead of making them
+            # wait behind every overdue child of an old, resumed crawl.
             if _run_due_snapshot_query(
-                active_snapshots,
+                active_snapshots.filter(crawl__status=Crawl.StatusChoices.QUEUED),
                 lock_seconds=60,
                 interactive_interrupts=interactive_interrupts,
                 runtime_config=runtime_config,
+                newest_first=True,
             ):
                 continue
 
-        if not maintenance_only:
             if _run_due_crawl_status(
                 Crawl.StatusChoices.QUEUED,
                 crawl_id=crawl_id,
                 lock_seconds=crawl_claim_lock_seconds,
                 interactive_interrupts=interactive_interrupts,
+                newest_first=True,
+            ):
+                continue
+
+            if _run_due_snapshot_query(
+                active_snapshots,
+                lock_seconds=60,
+                interactive_interrupts=interactive_interrupts,
+                runtime_config=runtime_config,
             ):
                 continue
 

archivebox/templates/admin/personas/persona/change_form.html

@@ -38,8 +38,8 @@
         border-bottom: 1px dashed #64748b;
     }
 
-    .field-import_mode ul,
-    .field-import_discovered_profile ul {
+    .abx-import-mode,
+    .abx-profile-picker {
         margin: 0;
         padding: 0;
         list-style: none;
@@ -47,20 +47,26 @@
         gap: 12px;
     }
 
-    .field-import_mode ul {
-        grid-template-columns: repeat(auto-fit, minmax(220px, 1fr));
+    .abx-import-mode {
+        grid-template-columns: repeat(auto-fit, minmax(min(220px, 100%), 1fr));
     }
 
-    .field-import_discovered_profile ul {
-        grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
-        max-height: 460px;
-        overflow: auto;
-        padding-right: 4px;
+    .abx-profile-picker {
+        grid-template-columns: repeat(auto-fit, minmax(min(260px, 100%), 1fr));
     }
 
-    .field-import_mode li,
-    .field-import_discovered_profile li {
+    .abx-import-mode > div,
+    .abx-profile-picker > div {
         margin: 0;
+        min-width: 0;
+    }
+
+    /* Import rows grow with their contents instead of sharing a fixed flex
+       height. Avoid overflow-x:hidden turning overflow-y:visible into auto. */
+    #content-main #persona_form .persona-card-import > .form-row,
+    #content-main #persona_form .persona-card-import fieldset > div {
+        flex: none;
+        overflow: visible;
     }
 
     .field-import_mode label,
@@ -68,7 +74,6 @@
         display: flex;
         gap: 12px;
         align-items: flex-start;
-        min-height: 100%;
         padding: 14px 16px;
         border-radius: 14px;
         border: 1px solid #dbe4ee;
@@ -167,9 +172,14 @@
         color: #991b1b;
     }
 
-    body.model-persona.change-form #content-main form fieldset.persona-card-primary {
-        flex: 1 1 520px !important;
-        max-width: calc(66.66% - 10px) !important;
+    #content-main #persona_form .field-name .flex-container {
+        max-width: 100% !important;
+    }
+
+    #persona_form .field-name input {
+        width: 100%;
+        max-width: none;
+        box-sizing: border-box;
     }
 
     body.model-persona.change-form #content-main form fieldset.persona-card-timestamps {
@@ -257,6 +267,11 @@ document.addEventListener('DOMContentLoaded', function () {
             or by pasting in a CDP url from
             <code class="persona-import-copy" data-copy-text="chrome://inspect#remote-debugging" title="Copy chrome://inspect#remote-debugging">chrome://inspect#remote-debugging</code> (e.g. <code>http://127.0.0.1:9222</code>).
         </p>
+        <p>
+            You can also import cookies into this persona from another browser using the
+            <a href="https://extension.archivebox.io/" target="_blank" rel="noopener noreferrer">ArchiveBox browser extension</a>.
+            Create the persona here, then select it in the extension's settings to sync cookies from that browser.
+        </p>
     </div>
 </section>
 {{ block.super }}
@@ -264,9 +279,9 @@ document.addEventListener('DOMContentLoaded', function () {
 
 {% block after_field_sets %}
 {{ block.super }}
-<section id="plugin-config" class="module aligned persona-plugin-config">
+<fieldset id="plugin-config" class="module aligned wide persona-plugin-config">
     <h2>Plugin Config</h2>
     <p>These typed controls update the same Persona config JSON shown above. Shared config keys stay synced across plugin sections.</p>
     {% include "plugins/plugin_config_grid.html" with plugin_groups=adminform.form.plugin_groups %}
-</section>
+</fieldset>
 {% endblock %}

archivebox/templates/admin/snapshots_grid.html

@@ -147,11 +147,21 @@
   }
   .cards .card .card-meta {
       display: grid;
-      grid-template-columns: 18px minmax(0, 1fr) auto;
+      grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr);
       gap: 8px;
       align-items: center;
       text-align: center;
   }
+  .cards .card .card-meta .action-select {
+      justify-self: start;
+  }
+  .cards .card .card-meta .card-date {
+      justify-self: center;
+      max-width: 100%;
+  }
+  .cards .card .card-meta .card-size {
+      justify-self: end;
+  }
   .cards .card .card-meta .card-date,
   .cards .card .card-meta .card-size {
       min-width: 0;
@@ -249,16 +259,16 @@
           <code class="card-url" title="{{obj.url}}">{{obj.url}}</code>
           <div class="card-media">
             <a href="{% snapshot_base_url obj %}/index.html" class="card-thumbnail">
-              {% snapshot_archiveresult_url obj 'screenshot' 'screenshot.png' as screenshot_url %}
-              {% snapshot_archiveresult_url obj 'chrome_extension_screenshot' 'screenshot-1.png' as extension_screenshot_1_url %}
-              {% snapshot_archiveresult_url obj 'chrome_extension_screenshot' 'screenshot.png' as extension_screenshot_url %}
-              {% snapshot_archiveresult_url obj 'favicon' 'favicon.ico' as favicon_url %}
+              {% admin_snapshot_archiveresult_url obj 'screenshot' 'screenshot.png' as screenshot_url %}
+              {% admin_snapshot_archiveresult_url obj 'chrome_extension_screenshot' 'screenshot-1.png' as extension_screenshot_1_url %}
+              {% admin_snapshot_archiveresult_url obj 'chrome_extension_screenshot' 'screenshot.png' as extension_screenshot_url %}
+              {% admin_snapshot_archiveresult_url obj 'favicon' 'favicon.ico' as favicon_url %}
               {% if screenshot_url or extension_screenshot_1_url or extension_screenshot_url or favicon_url %}
                 <img
                   src="{% firstof screenshot_url extension_screenshot_1_url extension_screenshot_url favicon_url %}"
                   alt="{{obj.title|default:'Not yet archived...'}}"
                   data-fallbacks="{{ extension_screenshot_1_url }},{{ extension_screenshot_url }},{{ favicon_url }}"
-                  onerror="const fallbacks=(this.dataset.fallbacks || '').split(',').filter(Boolean); if (fallbacks.length) { this.dataset.fallbacks=fallbacks.slice(1).join(','); this.src=fallbacks[0]; } else { this.onerror=null; this.closest('.card-thumbnail').classList.add('missing'); this.remove(); }"
+                  onerror="const current=this.src; const fallbacks=(this.dataset.fallbacks || '').split(',').filter(Boolean).map(candidate => new URL(candidate, document.baseURI).href).filter((candidate, index, all) => candidate !== current && all.indexOf(candidate) === index); if (fallbacks.length) { this.dataset.fallbacks=fallbacks.slice(1).join(','); this.src=fallbacks[0]; } else { this.onerror=null; this.closest('.card-thumbnail').classList.add('missing'); this.remove(); }"
                 />
               {% elif obj.status == 'started' %}
                 <img src="{% static 'spinner.gif' %}" class="loading-preview" alt="Archiving in progress"/>

archivebox/templates/core/add.html

@@ -287,7 +287,8 @@
                                 <code>daily</code> - Run once per day<br/>
                                 <code>weekly</code> - Run once per week<br/>
                                 <code>0 */6 * * *</code> - Every 6 hours (cron format)<br/>
-                                <code>0 0 * * 0</code> - Every Sunday at midnight (cron format)
+                                <code>0 0 * * 0</code> - Every Sunday at midnight (cron format)<br/>
+                                <a href="https://crontab.guru/" target="_blank" rel="noopener">Cron Format Helper</a>
                             </div>
                         </div>
 

archivebox/templates/core/setup_wizard.html

@@ -50,7 +50,7 @@
       <div class="abx-question-options" id="archivebox-setup-tls-options">
         <label class="abx-question-option"><input type="radio" name="archivebox-tls-mode" value="localhost"> <span><b>⭐ Localhost</b><small>No setup needed. HTTPS is not required for local access; all replay features remain available.</small></span></label>
         <label class="abx-question-option"><input type="radio" name="archivebox-tls-mode" value="none"> <span><b>⚠️ No TLS, private networks only</b><small>Access ArchiveBox directly on a trusted LAN, VPN, or private network without an ingress service. Public servers require HTTPS. In-browser WARC viewing will be disabled unless using <code>localhost</code> or HTTPS.</small></span></label>
-        <label class="abx-question-option"><input type="radio" name="archivebox-tls-mode" value="single"> <span><b>⚠️ Single-domain HTTPS certificate</b><small>Configure one certificate for the <code>BASE_URL</code> hostname in Cloudflare, Nginx Proxy Manager, Caddy, Traefik, Tailscale, or your platform's ingress UI, then proxy it to ArchiveBox on port <code>8000</code>. This mode is not allowed unless also using Single-domain DNS. Restart ArchiveBox after saving an HTTPS <code>BASE_URL</code> for the first time.</small></span></label>
+        <label class="abx-question-option"><input type="radio" name="archivebox-tls-mode" value="single"> <span><b>⚠️ Single-domain HTTPS certificate</b><small>Configure one certificate for the <code>BASE_URL</code> hostname in Cloudflare, Nginx Proxy Manager, Caddy, Traefik, Tailscale, or your platform's ingress UI, then proxy it to ArchiveBox on port <code>5797</code>. This mode is not allowed unless also using Single-domain DNS. Restart ArchiveBox after saving an HTTPS <code>BASE_URL</code> for the first time.</small></span></label>
         <label class="abx-question-option"><input type="radio" name="archivebox-tls-mode" value="wildcard"> <span><b>⭐ Wildcard TLS</b><small>Configure one certificate covering both the <code>BASE_URL</code> hostname and <code>*.BASE_URL</code>, normally through DNS-01. Never enable on-demand TLS or request individual certificates for snapshot subdomains. Restart ArchiveBox after saving an HTTPS <code>BASE_URL</code> for the first time.</small></span></label>
       </div>
       <div class="abx-question-status" id="archivebox-setup-tls-status">Choose how HTTPS will reach ArchiveBox.</div>

archivebox/templates/core/system_warnings_banner.html

@@ -33,17 +33,13 @@
       — the host is close to OOM. Consider stopping crawls or scaling up.
     {% elif mode == "high_load" %}
       15-min loadavg <code style="background:#000;color:#fff;padding:1px 6px;border-radius:3px;font:700 11px/1.3 ui-monospace,Menlo,Consolas,monospace;text-transform:none;">{{ load_15 }}</code>
-      exceeds <code style="background:#000;color:#fff;padding:1px 6px;border-radius:3px;font:700 11px/1.3 ui-monospace,Menlo,Consolas,monospace;text-transform:none;">{{ load_threshold }}</code>
-      ({{ cpu_count }} cores × 3) — recent CPU or I/O contention may slow crawls. This average can stay elevated after load subsides; it does not measure queue growth.
     {% elif mode == "unsafe" %}
       ArchiveBox single-domain mode — archived pages share an origin with this site
       <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Security-Modes" target="_blank" rel="noopener noreferrer" style="color:#fff;text-decoration:underline;margin-left:6px;">why?</a>
     {% elif mode == "base_url_mismatch" %}
-      Browser URL:
-      <code style="background:#000;color:#fff;padding:1px 6px;border-radius:3px;font:700 11px/1.3 ui-monospace,Menlo,Consolas,monospace;text-transform:none;">{{ browser_url }}</code>
-      · Configured BASE_URL:
-      <code style="background:#000;color:#fff;padding:1px 6px;border-radius:3px;font:700 11px/1.3 ui-monospace,Menlo,Consolas,monospace;text-transform:none;">{{ configured_base_url }}</code>
-      — GET access is allowed here, but state-changing requests are accepted only on BASE_URL.
+      Editing disabled from unrecognized URL. Visit
+      <a href="{{ configured_base_url }}" style="color:#fff;text-decoration:underline;text-transform:none;">{{ configured_base_url }}</a>
+      to make changes.
     {% else %}
       Ask an ArchiveBox superuser to finish server setup and pin the canonical URL.
     {% endif %}

archivebox/templates/static/admin.css

@@ -1987,7 +1987,7 @@ tbody .output-link:hover {opacity: 1;}
 }
 
 .model-snapshot.change-list .cards .card .card-info.card-meta {
-    grid-template-columns: 18px minmax(0, 1fr) auto;
+    grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr);
 }
 
 .model-snapshot.change-list .cards .card .card-info.card-meta > a {
@@ -2070,7 +2070,7 @@ tbody .output-link:hover {opacity: 1;}
     }
 
     .model-snapshot.change-list .cards .card .card-info.card-meta {
-        grid-template-columns: 18px minmax(0, 1fr) auto;
+        grid-template-columns: minmax(0, 1fr) auto minmax(0, 1fr);
     }
 }
 

archivebox/templates/static/setup_wizard.js

@@ -195,9 +195,11 @@
     } else {
       risks.push('⚠️ UI, API, and archive replay share one origin. CSP disables risky archived scripts, but a CSP or content-type bypass could expose the archive index, other snapshots, saved headers, admin pages, API data, and canonical-host mutations.');
     }
-    risks.push(publicIndexInput.checked
-      ? '⚠️ Anonymous visitors can enumerate public snapshot URLs and titles; saved URLs may contain private share tokens or other secrets.'
-      : '✅ Anonymous visitors cannot browse the snapshot index.');
+    if (!publicIndexInput.checked) {
+      risks.push('✅ Anonymous visitors cannot browse the snapshot index.');
+    } else {
+      risks.push('⚠️ Anonymous visitors can browse the public snapshot index; saved URLs may contain private share tokens or other secrets.');
+    }
     risks.push(!controlPlaneEnabled
       ? '✅ URL submission and other state-changing requests are disabled for everyone in this replay-only mode.'
       : (publicAddInput.checked ? '⚠️ Anonymous visitors can submit malicious or private/intranet URLs. A filtering or per-crawl configuration bypass could expose internal content or threaten the server.' : '✅ Only signed-in admins can submit new URLs.'));
@@ -234,7 +236,7 @@
     var exampleBaseUrl = desiredScheme + exampleBaseHost;
     var exampleWildcardUrl = desiredScheme + exampleWildcardHost;
     var exampleAdminUrl = desiredScheme + (dnsMode === 'wildcard' ? 'admin.' + exampleBaseHost : exampleBaseHost) + '/admin/';
-    var localhostAdminUrl = 'http://admin.archivebox.localhost:8000/admin/';
+    var localhostAdminUrl = 'http://admin.archivebox.localhost:5797/admin/';
 
     document.getElementById('archivebox-setup-hosting-status').textContent = hosting === 'localhost'
       ? '❌ Visit ' + localhostAdminUrl + ' from the same machine to continue setup.'
@@ -253,9 +255,9 @@
     document.getElementById('archivebox-setup-tls-status').textContent = tlsMode === 'localhost'
       ? '❌ Visit ' + localhostAdminUrl + ' from this machine to continue setup. No certificate is needed.'
       : (tlsMode === 'wildcard'
-        ? '❌ In your ingress provider, proxy to ArchiveBox on port 8000 and configure one browser-trusted certificate covering ' + exampleBaseHost + ' and ' + exampleWildcardHost + ', normally through DNS-01. Never enable on-demand TLS or request individual snapshot certificates. Visit ' + exampleAdminUrl + ' to continue setup.'
+        ? '❌ In your ingress provider, proxy to ArchiveBox on port 5797 and configure one browser-trusted certificate covering ' + exampleBaseHost + ' and ' + exampleWildcardHost + ', normally through DNS-01. Never enable on-demand TLS or request individual snapshot certificates. Visit ' + exampleAdminUrl + ' to continue setup.'
         : (tlsMode === 'single'
-          ? '❌ In your ingress provider, proxy to ArchiveBox on port 8000 and configure one browser-trusted certificate for ' + exampleBaseHost + '. Visit ' + exampleAdminUrl + ' to continue setup.'
+          ? '❌ In your ingress provider, proxy to ArchiveBox on port 5797 and configure one browser-trusted certificate for ' + exampleBaseHost + '. Visit ' + exampleAdminUrl + ' to continue setup.'
           : (tlsMode === 'none'
             ? '❌ Expose this ArchiveBox server directly over HTTP without a separate ingress or SSL termination service. Visit ' + exampleAdminUrl + ' to continue setup. In-browser WARC viewing will remain disabled unless browsing through localhost or HTTPS.'
             : '❌ Choose an ingress and TLS mode. Your choice will not be changed automatically.')));

archivebox/tests/conftest.py

@@ -1164,9 +1164,9 @@ def run_python_cwd(
 # Server/API Integration Helpers
 # =============================================================================
 
-API_TEST_HOST = "api.archivebox.localhost:8000"
-ADMIN_TEST_HOST = "admin.archivebox.localhost:8000"
-WEB_TEST_HOST = "web.archivebox.localhost:8000"
+API_TEST_HOST = "api.archivebox.localhost:5797"
+ADMIN_TEST_HOST = "admin.archivebox.localhost:5797"
+WEB_TEST_HOST = "web.archivebox.localhost:5797"
 
 
 @pytest.fixture
@@ -1794,8 +1794,8 @@ def real_archive_with_example(tmp_path_factory, request):
         [
             "config",
             "--set",
-            "BIND_ADDR=127.0.0.1:8000",
-            "BASE_URL=http://archivebox.localhost:8000",
+            "BIND_ADDR=127.0.0.1:5797",
+            "BASE_URL=http://archivebox.localhost:5797",
             "PUBLIC_INDEX=True",
             "PUBLIC_ADD_VIEW=True",
         ],

archivebox/tests/test_api_v1_core_archiveresult_archiveresult_id.py

@@ -67,3 +67,68 @@ def test_archiveresult_patch_upload_finalizes_queued_result(client, api_admin_us
     assert result.output_str == "output.html"
     assert snapshot.status == Snapshot.StatusChoices.SEALED
     assert snapshot.retry_at is not None
+
+
+@pytest.mark.parametrize("chunked", [False, True])
+@pytest.mark.parametrize("method", ["POST", "PATCH"])
+def test_live_archiveresult_multipart_upload(tmp_path, chunked, method):
+    import requests
+
+    from archivebox.tests.conftest import (
+        api_auth_headers,
+        cli_env,
+        create_admin_and_token,
+        get_free_port,
+        init_archive,
+        live_api_request,
+        stop_archivebox_process,
+    )
+    from archivebox.tests.test_api_v1_cli_add import start_api_server_without_runner
+
+    init_archive(tmp_path)
+    token = create_admin_and_token(tmp_path)
+    port = get_free_port()
+    server = start_api_server_without_runner(tmp_path, cli_env(port=port, server=True), port)
+    try:
+        snapshot = live_api_request(
+            port,
+            "post",
+            "/api/v1/core/snapshots",
+            api_token=token,
+            json={"url": "https://example.com/multipart", "status": "sealed"},
+        )
+        assert snapshot.status_code == 200, snapshot.text
+        result = live_api_request(
+            port,
+            "post",
+            "/api/v1/core/archiveresults",
+            api_token=token,
+            data={"snapshot_id": snapshot.json()["id"], "plugin": "dom", "status": "started"},
+        )
+        assert result.status_code == 200, result.text
+        payload = b"<html>Real multipart upload</html>"
+        body = encode_multipart(
+            BOUNDARY,
+            {
+                "files": SimpleUploadedFile("output.html", payload, content_type="text/html"),
+                "output_paths": "output.html",
+                "status": "succeeded",
+                "snapshot_id": snapshot.json()["id"],
+                "plugin": "dom",
+            },
+        )
+        path = "/api/v1/core/archiveresults" if method == "POST" else f"/api/v1/core/archiveresult/{result.json()['id']}"
+        response = requests.request(
+            method,
+            f"http://127.0.0.1:{port}{path}",
+            data=iter([body]) if chunked else body,
+            headers={**api_auth_headers(token, port=port), "Content-Type": MULTIPART_CONTENT},
+            timeout=30,
+        )
+        assert response.status_code == 200, response.text
+        assert response.json()["status"] == "succeeded"
+        files = list((tmp_path / "archive").glob("users/*/snapshots/*/*/*/dom/output.html"))
+        assert len(files) == 1
+        assert files[0].read_bytes() == payload
+    finally:
+        stop_archivebox_process(server)

archivebox/tests/test_api_v1_core_archiveresults.py

@@ -15,6 +15,79 @@ from archivebox.tests.conftest import api_client_request
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
+@pytest.mark.parametrize("plugin", [".", "..", "/./", "/../", " . ", " ../ "])
+def test_archiveresult_upload_rejects_dot_directory_names(client, api_admin_user, api_headers, plugin):
+    crawl = Crawl.objects.create(urls="https://example.com", created_by=api_admin_user)
+    snapshot = Snapshot.objects.create(url="https://example.com/invalid-upload-directory", crawl=crawl)
+    snapshot_dir = snapshot.output_dir
+    filename = "rejected-upload.txt"
+
+    response = client.post(
+        "/api/v1/core/archiveresults",
+        {
+            "snapshot_id": str(snapshot.id),
+            "plugin": plugin,
+            "files": SimpleUploadedFile(filename, b"must not be written", content_type="text/plain"),
+        },
+        **api_headers,
+    )
+
+    assert response.status_code == 400, response.content
+    assert response.json()["detail"] == "Invalid ArchiveResult plugin name"
+    assert not snapshot.archiveresult_set.exists()
+    assert not (snapshot_dir / filename).exists()
+    assert not (snapshot_dir.parent / filename).exists()
+
+
+@pytest.mark.parametrize("plugin", [".", ".."])
+def test_archiveresult_patch_rejects_existing_dot_directory_names(client, api_admin_user, api_headers, plugin):
+    crawl = Crawl.objects.create(urls="https://example.com", created_by=api_admin_user)
+    snapshot = Snapshot.objects.create(url="https://example.com/invalid-existing-directory", crawl=crawl)
+    result = ArchiveResult.objects.create(snapshot=snapshot, plugin=plugin)
+    snapshot_dir = snapshot.output_dir
+    filename = "rejected-patch.txt"
+
+    response = client.patch(
+        f"/api/v1/core/archiveresult/{result.id}",
+        encode_multipart(
+            BOUNDARY,
+            {"files": SimpleUploadedFile(filename, b"must not be written", content_type="text/plain")},
+        ),
+        content_type=MULTIPART_CONTENT,
+        **api_headers,
+    )
+
+    assert response.status_code == 400, response.content
+    assert response.json()["detail"] == "Invalid ArchiveResult plugin name"
+    result.refresh_from_db()
+    assert result.output_files == {}
+    assert not (snapshot_dir / filename).exists()
+    assert not (snapshot_dir.parent / filename).exists()
+
+
+@pytest.mark.parametrize("plugin", ["custom.files", "custom-files_v1"])
+def test_archiveresult_upload_accepts_unregistered_directory_names(client, api_admin_user, api_headers, plugin):
+    crawl = Crawl.objects.create(urls="https://example.com", created_by=api_admin_user)
+    snapshot = Snapshot.objects.create(url="https://example.com/custom-upload-directory", crawl=crawl)
+    contents = b"uploaded original file"
+
+    response = client.post(
+        "/api/v1/core/archiveresults",
+        {
+            "snapshot_id": str(snapshot.id),
+            "plugin": plugin,
+            "files": SimpleUploadedFile("original.txt", contents, content_type="text/plain"),
+        },
+        **api_headers,
+    )
+
+    assert response.status_code == 200, response.content
+    result = snapshot.archiveresult_set.get()
+    assert result.plugin == plugin
+    assert result.output_files["original.txt"]["size"] == len(contents)
+    assert (snapshot.output_dir / plugin / "original.txt").read_bytes() == contents
+
+
 def test_archiveresult_upload_upserts_by_snapshot_plugin_and_hook(client, api_admin_user, api_headers):
     crawl = Crawl.objects.create(urls="https://example.com", created_by=api_admin_user)
     snapshot = Snapshot.objects.create(url="https://example.com/unified-result", crawl=crawl)

archivebox/tests/test_api_v1_core_snapshots_rss.py

@@ -11,7 +11,7 @@ pytestmark = pytest.mark.django_db
 
 
 User = get_user_model()
-ADMIN_HOST = "admin.archivebox.localhost:8000"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
 
 
 @pytest.fixture

archivebox/tests/test_api_v1_core_tags_autocomplete.py

@@ -12,7 +12,7 @@ def test_basic_success_case_request(client, tmp_path, api_admin_user, api_token)
     response = client.get(
         "/api/v1/core/tags/autocomplete/",
         {"q": "api-basic", "api_key": api_token.token},
-        HTTP_HOST="api.archivebox.localhost:8000",
+        HTTP_HOST="api.archivebox.localhost:5797",
     )
 
     assert response.status_code == 200, response.content

archivebox/tests/test_api_v1_crawls_crawl_crawl_id.py

@@ -33,7 +33,7 @@ from .conftest import (
 
 pytestmark = pytest.mark.django_db(transaction=True)
 User = get_user_model()
-ADMIN_HOST = "admin.archivebox.localhost:8000"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
 
 
 @pytest.fixture

archivebox/tests/test_archive_result_service.py

@@ -1,5 +1,6 @@
 from pathlib import Path
 from importlib.resources import files
+import hashlib
 import json
 import os
 import shutil
@@ -189,15 +190,19 @@ def test_process_completed_projects_inline_archiveresult(tmp_path, hermetic_lib_
 
     assert result.status == ArchiveResult.StatusChoices.SUCCEEDED
     assert result.process_id == process.id
-    assert result.output_str.endswith(json.loads((snapshot_dir / "hashes" / "hashes.json").read_text())["root_hash"][:12])
+    manifest = snapshot_dir / "hashes" / "hashes.json"
+    checksum = snapshot_dir / "hashes" / "hashes.sha256"
+    assert checksum.read_text() == hashlib.sha256(manifest.read_bytes()).hexdigest() + "\n"
+    assert result.output_str.endswith(json.loads(manifest.read_text())["root_hash"][:12])
     assert result.output_files == {
         "hashes.json": {
             "extension": "json",
             "mimetype": "application/json",
-            "size": (snapshot_dir / "hashes" / "hashes.json").stat().st_size,
+            "size": manifest.stat().st_size,
         },
+        "hashes.sha256": {"extension": "sha256", "mimetype": "", "size": 65},
     }
-    assert result.output_size == (snapshot_dir / "hashes" / "hashes.json").stat().st_size
+    assert result.output_size == manifest.stat().st_size + checksum.stat().st_size
     _cleanup_machine_process_rows()
 
 

archivebox/tests/test_cli_persona.py

@@ -16,6 +16,23 @@ def test_persona_help_runs_successfully(tmp_path):
     assert "list" in result.stdout
 
 
+def test_persona_create_permissions_preserves_existing_persona(initialized_archive):
+    from archivebox.tests.conftest import run_python_cwd
+
+    result = run_archivebox_cmd(["persona", "create", "--permissions=private", "host-profile"], cwd=initialized_archive)
+    assert result.returncode == 0, result.stderr
+    existing = run_archivebox_cmd(["persona", "create", "--permissions=public", "host-profile"], cwd=initialized_archive)
+    assert existing.returncode == 0, existing.stderr
+    stdout, stderr, returncode = run_python_cwd(
+        "import os; os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'archivebox.core.settings'); "
+        "import django; django.setup(); from archivebox.personas.models import Persona; "
+        "p = Persona.objects.get(name='host-profile'); assert p.permissions == 'private'; print('private')",
+        cwd=initialized_archive,
+    )
+    assert returncode == 0, stderr
+    assert "private" in stdout
+
+
 def test_persona_import_missing_source_is_actionable_and_does_not_create(initialized_archive, tmp_path):
     result = run_archivebox_cmd(
         ["persona", "create", "--import=brave", "--source", str(tmp_path / "missing"), "broken"],

archivebox/tests/test_cli_run.py

@@ -31,6 +31,45 @@ RUN_TEST_ENV = {
 }
 
 
+@pytest.mark.django_db(transaction=True)
+def test_cli_run_retries_snapshot_after_filesystem_failure(initialized_archive):
+    from archivebox.core.models import ArchiveResult, Snapshot
+    from archivebox.tests.test_orm_helpers import use_archivebox_db
+
+    env = cli_env(live=True, PLUGINS="hashes")
+    created = run_archivebox_cmd(["snapshot", "create", create_test_url()], cwd=initialized_archive, env=env)
+    assert created.returncode == 0, created.stdout + created.stderr
+    snapshot_id = next(record["id"] for record in parse_jsonl_output(created.stdout) if record.get("type") == "Snapshot")
+    with use_archivebox_db(initialized_archive):
+        snapshot = Snapshot.objects.get(pk=snapshot_id)
+        crawl_id = str(snapshot.crawl_id)
+        directory = snapshot.output_dir
+    directory.mkdir(parents=True, exist_ok=True)
+    (directory / "input.txt").write_text("Capture survives a temporary filesystem failure")
+    broken_directory = directory / "hashes"
+    broken_directory.symlink_to("hashes")
+
+    failed = run_archivebox_cmd(["run", f"--snapshot-id={snapshot_id}"], cwd=initialized_archive, env=env, timeout=60)
+    assert failed.returncode == 1, failed.stdout + failed.stderr
+    assert "FileExistsError: [Errno 17]" in failed.stderr
+    with use_archivebox_db(initialized_archive):
+        snapshot = Snapshot.objects.get(pk=snapshot_id)
+        assert snapshot.status == Snapshot.StatusChoices.STARTED
+        assert snapshot.retry_at is not None
+        assert not ArchiveResult.objects.filter(snapshot=snapshot).exists()
+
+    broken_directory.unlink()
+    resumed = run_archivebox_cmd(["run", f"--crawl-id={crawl_id}"], cwd=initialized_archive, env=env, timeout=60)
+    assert resumed.returncode == 0, resumed.stdout + resumed.stderr
+    with use_archivebox_db(initialized_archive):
+        snapshot = Snapshot.objects.get(pk=snapshot_id)
+        assert snapshot.status == Snapshot.StatusChoices.SEALED
+        result = snapshot.archiveresult_set.get(plugin="hashes")
+        assert result.status == ArchiveResult.StatusChoices.SUCCEEDED
+        assert result.output_size > 0
+    assert (directory / "hashes").is_dir()
+
+
 def _install_real_chrome_for_test(data_dir, env, *, isolation):
     env["CHROME_ISOLATION"] = isolation
     env["CHROME_HEADLESS"] = "true"
@@ -44,6 +83,52 @@ def _install_real_chrome_for_test(data_dir, env, *, isolation):
     assert install_process.returncode == 0, install_process.stderr or install_process.stdout
 
 
+@pytest.mark.django_db(transaction=True)
+@pytest.mark.parametrize("precreate_snapshot", [False, True])
+def test_newest_crawl_runs_before_older_queued_and_started_crawls(initialized_archive, precreate_snapshot):
+    from datetime import timedelta
+
+    from django.utils import timezone
+
+    from archivebox.core.models import Snapshot
+    from archivebox.crawls.models import Crawl
+    from archivebox.tests.test_orm_helpers import use_archivebox_db
+
+    env = cli_env(live=True, PLUGINS="hashes")
+    old_url, queued_url, newest_url = create_test_url(), create_test_url(), create_test_url()
+    old = run_archivebox_cmd(["snapshot", "create", old_url], cwd=initialized_archive, env=env)
+    assert old.returncode == 0, old.stdout + old.stderr
+    with use_archivebox_db(initialized_archive):
+        snapshot = Snapshot.objects.get(url=old_url)
+        Crawl.objects.filter(pk=snapshot.crawl_id).update(status=Crawl.StatusChoices.STARTED)
+        Snapshot.objects.filter(pk=snapshot.pk).update(retry_at=timezone.now() - timedelta(days=1))
+    queued = run_archivebox_cmd(
+        ["snapshot" if precreate_snapshot else "crawl", "create", queued_url],
+        cwd=initialized_archive,
+        env=env,
+    )
+    assert queued.returncode == 0, queued.stdout + queued.stderr
+    newest = run_archivebox_cmd(
+        ["snapshot" if precreate_snapshot else "crawl", "create", newest_url],
+        cwd=initialized_archive,
+        env=env,
+    )
+    assert newest.returncode == 0, newest.stdout + newest.stderr
+
+    result = run_archivebox_cmd(["run", "--no-stdin"], cwd=initialized_archive, env=env, timeout=60)
+    assert result.returncode == 0, result.stdout + result.stderr
+    assert all(url in result.stdout for url in (old_url, queued_url, newest_url))
+    assert result.stdout.index(newest_url) < result.stdout.index(queued_url) < result.stdout.index(old_url), result.stdout
+    with use_archivebox_db(initialized_archive):
+        assert (
+            Snapshot.objects.filter(
+                url__in=[old_url, queued_url, newest_url],
+                status=Snapshot.StatusChoices.SEALED,
+            ).count()
+            == 3
+        )
+
+
 @pytest.mark.django_db(transaction=True)
 @pytest.mark.timeout(660)
 def test_cli_run_signal_cleans_real_chrome_hook_process_group(initialized_archive, recursive_test_site):
@@ -2096,7 +2181,7 @@ class TestRecoverOrchestratorStateRedFailureModes:
                 stdout="\n".join(json.dumps(record) for record in records),
             )
 
-        recovered = recover_orchestrator_state()
+        recovered = recover_orchestrator_state(crawl_id=str(crawl.id))
 
         assert recovered["archiveresults_missing_for_orphaned_hook_processes"] == 1
         assert ArchiveResult.objects.filter(snapshot=snapshot, plugin="title", hook_name=hook_name).count() == 1
@@ -2108,6 +2193,54 @@ class TestRecoverOrchestratorStateRedFailureModes:
         assert result.end_ts == newer_start + timedelta(seconds=1)
         assert result.process.started_at == newer_start
 
+    def test_global_recovery_skips_historical_hook_output_scan(self, tmp_path):
+        from datetime import timedelta
+
+        from django.utils import timezone
+
+        from archivebox.base_models.models import get_or_create_system_user_pk
+        from archivebox.core.models import ArchiveResult, Snapshot
+        from archivebox.core.recovery_util import recover_orchestrator_state
+        from archivebox.crawls.models import Crawl
+        from archivebox.machine.models import Machine, NetworkInterface, Process
+
+        future = timezone.now() + timedelta(days=1)
+        crawl = Crawl.objects.create(
+            urls="https://example.com/old\nhttps://example.com/new",
+            created_by_id=get_or_create_system_user_pk(),
+            status=Crawl.StatusChoices.STARTED,
+            retry_at=future,
+        )
+        snapshot = Snapshot.objects.create(
+            url="https://example.com/old",
+            crawl=crawl,
+            status=Snapshot.StatusChoices.STARTED,
+            retry_at=future,
+        )
+        historical_plugin_dir = tmp_path / str(snapshot.id) / "title"
+        historical_plugin_dir.mkdir(parents=True)
+        (historical_plugin_dir / "title.txt").write_text("preserved historical output")
+        Process.objects.create(
+            machine=Machine.current(refresh=True),
+            iface=NetworkInterface.current(refresh=True),
+            process_type=Process.TypeChoices.HOOK,
+            worker_type="archiveresult",
+            pwd=str(historical_plugin_dir),
+            cmd=["on_Snapshot__01_title.py"],
+            status=Process.StatusChoices.EXITED,
+            retry_at=None,
+            exit_code=0,
+            started_at=timezone.now() - timedelta(days=30),
+            ended_at=timezone.now() - timedelta(days=30) + timedelta(seconds=1),
+        )
+
+        recovered = recover_orchestrator_state()
+
+        assert recovered["snapshots_started_without_running_results"] == 1
+        assert recovered["archiveresults_missing_for_orphaned_hook_processes"] == 0
+        assert Snapshot.objects.filter(crawl=crawl, retry_at__lte=timezone.now()).count() == 1
+        assert not ArchiveResult.objects.filter(snapshot=snapshot).exists()
+
     def test_recovery_does_not_seal_queued_snapshot_waiting_for_future_retry_even_with_final_results(self):
         from datetime import timedelta
 

archivebox/tests/test_cli_server.py

@@ -351,12 +351,42 @@ def test_runtime_binary_wins_over_ambient_system_path(tmp_path):
 def test_daphne_worker_uses_default_application_close_timeout():
     from archivebox.workers.supervisord_util import SERVER_WORKER
 
-    command = SERVER_WORKER("127.0.0.1", "8000")["command"]
+    command = SERVER_WORKER("127.0.0.1", "5797")["command"]
 
     assert "daphne" in command
     assert "--application-close-timeout=0" not in command
 
 
+@pytest.mark.parametrize("in_docker,port", [(False, "5797"), (False, "8000"), (True, "9579")])
+@pytest.mark.parametrize("debug", [False, True])
+def test_port_compatibility_does_not_add_listeners_outside_docker_defaults(tmp_path, in_docker, port, debug):
+    result = subprocess.run(
+        [
+            sys.executable,
+            "-c",
+            "from archivebox.config.common import get_config;"
+            "from archivebox.workers.supervisord_util import build_server_worker_plan;"
+            "import json,sys;"
+            "config=get_config();"
+            "workers,_,_=build_server_worker_plan(config=config,host='127.0.0.1',"
+            "port=sys.argv[1],debug=sys.argv[2]=='True',reload=False,nothreading=False);"
+            "print(json.dumps([worker for worker,lazy in workers]))",
+            port,
+            str(debug),
+        ],
+        cwd=tmp_path,
+        env=cli_env(IN_DOCKER=str(in_docker), SEARCH_BACKEND_SONIC_ENABLED="False", DISPLAY="", ARCHIVEBOX_VNC_PERSONA=""),
+        capture_output=True,
+        text=True,
+        check=True,
+    )
+    workers = json.loads(result.stdout)
+    assert [worker["name"] for worker in workers] == ["worker_runserver" if debug else "worker_daphne", "worker_runner"]
+    command = shlex.split(workers[0]["command"])
+    assert "--endpoint" not in command
+    assert (f"127.0.0.1:{port}" if debug else f"--port={port}") in command
+
+
 def test_supervisord_parent_watchdog_does_not_start_another_archivebox_runtime():
     from archivebox.workers import supervisord_util
 
@@ -401,17 +431,17 @@ def test_server_worker_memory_preflight_fails_before_starting_supervisord(capsys
 def test_reload_workers_use_active_archivebox_module():
     from archivebox.workers.supervisord_util import RUNNER_WATCH_WORKER, RUNSERVER_WORKER, archivebox_cmd
 
-    runserver = RUNSERVER_WORKER("127.0.0.1", "8000", reload=True)
-    watcher = RUNNER_WATCH_WORKER("http://127.0.0.1:8000")
+    runserver = RUNSERVER_WORKER("127.0.0.1", "5797", reload=True)
+    watcher = RUNNER_WATCH_WORKER("http://127.0.0.1:5797")
 
     assert runserver["name"] == "worker_runserver"
-    assert shlex.split(runserver["command"]) == archivebox_cmd("manage", "runserver", "127.0.0.1:8000")
+    assert shlex.split(runserver["command"]) == archivebox_cmd("manage", "runserver", "127.0.0.1:5797")
     assert 'ARCHIVEBOX_RUNSERVER="1"' in runserver["environment"]
     assert 'ARCHIVEBOX_AUTORELOAD="1"' in runserver["environment"]
-    assert 'ARCHIVEBOX_RUNSERVER_BIND_URL="http://127.0.0.1:8000"' in runserver["environment"]
+    assert 'ARCHIVEBOX_RUNSERVER_BIND_URL="http://127.0.0.1:5797"' in runserver["environment"]
 
     assert watcher["name"] == "worker_runner_watch"
-    assert shlex.split(watcher["command"]) == archivebox_cmd("manage", "runner_watch", "--bind-url=http://127.0.0.1:8000")
+    assert shlex.split(watcher["command"]) == archivebox_cmd("manage", "runner_watch", "--bind-url=http://127.0.0.1:5797")
 
 
 def test_server_daemon_starts_real_plugin_owned_sonic_worker(initialized_archive, archivebox_daemon_server):

archivebox/tests/test_cli_update.py

@@ -5,8 +5,13 @@ Verify update drains old dirs, reconciles DB, and queues snapshots.
 """
 
 import pytest
+import shutil
+import signal
+import json
+import uuid
 
-from archivebox.core.models import Snapshot
+from archivebox.core.models import ArchiveResult, Snapshot
+from archivebox.config import CONSTANTS
 from archivebox.tests.conftest import run_queued_crawls, run_archivebox_cmd, cli_env
 
 from archivebox.tests.test_orm_helpers import use_archivebox_db
@@ -14,6 +19,131 @@ from archivebox.tests.test_orm_helpers import use_archivebox_db
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
+def test_rescan_imports_current_layout_and_is_idempotent(initialized_archive, tmp_path):
+    env = cli_env(disable_extractors=True)
+    run_archivebox_cmd(["add", "--index-only", "https://example.com/rescan"], cwd=initialized_archive, env=env, check=True)
+    run_queued_crawls(initialized_archive, env)
+    run_archivebox_cmd(["update", "--migrate-only"], cwd=initialized_archive, env=env, check=True)
+    with use_archivebox_db(initialized_archive):
+        original = Snapshot.objects.get(url="https://example.com/rescan")
+        original.title = "Imported snapshot"
+        original.save()
+        output = original.output_dir / "singlefile" / "singlefile.html"
+        output.parent.mkdir(parents=True, exist_ok=True)
+        output.write_text("<html><title>Portable archive fixture</title></html>")
+        result = ArchiveResult.objects.create(
+            snapshot=original,
+            plugin="singlefile",
+            hook_name="on_Snapshot__singlefile",
+            status="succeeded",
+            output_str="singlefile/singlefile.html",
+        )
+        result.update_output_metadata_from_filesystem(full_scan=True)
+        result.refresh_from_db()
+        expected_result = (result.pk, result.created_at, result.modified_at, result.output_files, result.output_size)
+        original.refresh_from_db()
+        original.write_index_jsonl()
+        index = original.output_dir / "index.jsonl"
+        historical = {**result.to_json(), "id": uuid.uuid4().hex}
+        index.write_text(json.dumps(historical) + "\n" + index.read_text())
+        snapshot_id = original.pk
+        original_fields = (original.crawl_id, original.created_at, original.modified_at, original.bookmarked_at, original.timestamp)
+        original.crawl.refresh_from_db()
+        crawl_dates = (original.crawl.created_at, original.crawl.modified_at)
+        relative_path = original.output_dir.relative_to(initialized_archive)
+
+    destination = tmp_path / "destination"
+    destination.mkdir()
+    run_archivebox_cmd(["init", "--quick"], cwd=destination, env=env, check=True)
+    shutil.copytree(initialized_archive / "archive", destination / "archive", dirs_exist_ok=True, symlinks=True)
+    result = run_archivebox_cmd(["update", "--migrate-only", "--index-only"], cwd=destination, env=env, timeout=120)
+    assert result.returncode == 0, result.stdout + result.stderr
+    with use_archivebox_db(destination):
+        assert not Snapshot.objects.filter(pk=snapshot_id).exists()
+
+    for expected in ("Imported: 1", "Unchanged: 1"):
+        result = run_archivebox_cmd(["update", "--rescan", "--migrate-only"], cwd=destination, env=env, timeout=120)
+        assert result.returncode == 0, result.stdout + result.stderr
+        assert expected in result.stdout
+        with use_archivebox_db(destination):
+            imported = Snapshot.objects.get(pk=snapshot_id)
+            assert imported.title == "Imported snapshot"
+            assert imported.status == "sealed"
+            assert imported.output_dir.relative_to(CONSTANTS.DATA_DIR) == relative_path
+            assert (destination / relative_path / "index.jsonl").is_file()
+            assert (
+                imported.crawl_id,
+                imported.created_at,
+                imported.modified_at,
+                imported.bookmarked_at,
+                imported.timestamp,
+            ) == original_fields
+            assert (imported.crawl.created_at, imported.crawl.modified_at) == crawl_dates
+            assert Snapshot.objects.count() == 1
+            imported_result = imported.archiveresult_set.get()
+            assert (
+                imported_result.pk,
+                imported_result.created_at,
+                imported_result.modified_at,
+                imported_result.output_files,
+                imported_result.output_size,
+            ) == expected_result
+            assert (destination / relative_path / "singlefile" / "singlefile.html").read_text() == output.read_text()
+
+    copied_index = destination / relative_path / "index.jsonl"
+    records = [json.loads(line) for line in copied_index.read_text().splitlines()]
+    for record in records:
+        if record.get("type") == "Snapshot":
+            record["url"] = "https://example.com/conflicting-identity"
+    conflicting_bytes = "\n".join(json.dumps(record) for record in records) + "\n"
+    copied_index.write_text(conflicting_bytes)
+    result = run_archivebox_cmd(["update", "--rescan"], cwd=destination, env=env)
+    assert result.returncode == 1, result.stdout + result.stderr
+    assert "conflicting identity" in result.stdout
+    assert copied_index.read_text() == conflicting_bytes
+    with use_archivebox_db(destination):
+        assert Snapshot.objects.get(pk=snapshot_id).url == "https://example.com/rescan"
+
+
+@pytest.mark.parametrize("reverse", [False, True])
+def test_rescan_resumes_after_real_interrupt_in_requested_order(initialized_archive, tmp_path, reverse):
+    env = cli_env(disable_extractors=True)
+    urls = [f"https://example.com/rescan-interrupt/{number}" for number in range(30)]
+    run_archivebox_cmd(["add", "--index-only", *urls], cwd=initialized_archive, env=env, check=True)
+    run_queued_crawls(initialized_archive, env)
+    run_archivebox_cmd(["update", "--migrate-only"], cwd=initialized_archive, env=env, check=True)
+    with use_archivebox_db(initialized_archive):
+        originals = list(Snapshot.objects.order_by("id" if reverse else "-id"))
+        expected = [(snapshot.pk, snapshot.created_at, snapshot.crawl_id) for snapshot in originals]
+        for snapshot in originals:
+            snapshot.write_index_jsonl()
+    destination = tmp_path / "destination"
+    destination.mkdir()
+    run_archivebox_cmd(["init", "--quick"], cwd=destination, env=env, check=True)
+    shutil.copytree(initialized_archive / "archive", destination / "archive", dirs_exist_ok=True, symlinks=True)
+    args = ["update", "--rescan", *(["--reverse"] if reverse else [])]
+    process = run_archivebox_cmd(args, cwd=destination, env=env, wait=False)
+    try:
+        for line in process.stdout:
+            if "Imported:" in line:
+                process.send_signal(signal.SIGINT)
+                break
+        stdout, stderr = process.communicate(timeout=60)
+        assert process.returncode == 130, stdout + stderr
+    finally:
+        if process.poll() is None:
+            process.terminate()
+            process.wait(timeout=60)
+    with use_archivebox_db(destination):
+        partial = list(Snapshot.objects.order_by("id" if reverse else "-id").values_list("pk", "created_at", "crawl_id"))
+        assert 0 < len(partial) < len(expected)
+        assert partial == expected[: len(partial)]
+    result = run_archivebox_cmd(args, cwd=destination, env=env, timeout=120)
+    assert result.returncode == 0, result.stdout + result.stderr
+    with use_archivebox_db(destination):
+        assert list(Snapshot.objects.order_by("id" if reverse else "-id").values_list("pk", "created_at", "crawl_id")) == expected
+
+
 def test_update_runs_successfully_on_empty_archive(initialized_archive):
     """Test that update runs without error on empty archive."""
     result = run_archivebox_cmd(
@@ -23,10 +153,9 @@ def test_update_runs_successfully_on_empty_archive(initialized_archive):
     output = result.stdout + result.stderr
 
     assert result.returncode == 0, output
-    assert "Phase 1: Draining old archive/ directories" in output
-    assert "Phase 2: Selecting database snapshots with stale filesystem versions" in output
-    assert "Updated DB rows:  0" in output
-    assert "Sealed crawls:    0" in output
+    assert "Filesystem migrations remain lazy" in output
+    assert "Phase 1" not in output
+    assert "Phase 2" not in output
 
     with use_archivebox_db(initialized_archive):
         assert Snapshot.objects.count() == 0
@@ -52,8 +181,8 @@ def test_update_reconciles_existing_snapshots(initialized_archive):
 
     output = result.stdout + result.stderr
     assert result.returncode == 0, output
-    assert "Phase 2: Selecting database snapshots with stale filesystem versions" in output
-    assert "Updated DB rows:" in output
+    assert "Filesystem migrations remain lazy" in output
+    assert "Updated DB rows:" not in output
 
     with use_archivebox_db(initialized_archive):
         assert Snapshot.objects.filter(url="https://example.com", status="sealed").count() == 1

archivebox/tests/test_cli_update_reindex_snapshots.py

@@ -35,9 +35,9 @@ def test_update_imports_orphaned_snapshots(tmp_path, initialized_archive):
         ),
     )
 
-    # Run the migration phase only; default update also runs queued crawl work.
+    # Filesystem-only discovery is explicit; default update stays lazy.
     update_process = run_archivebox_cmd(
-        ["update", "--migrate-only"],
+        ["update", "--rescan", "--migrate-only"],
         env=env,
         timeout=60,
     )

archivebox/tests/test_config_DELETE_AFTER.py

@@ -10,8 +10,8 @@ from archivebox.tests.conftest import run_archivebox_cmd, run_queued_crawls, cli
 
 pytestmark = pytest.mark.django_db(transaction=True)
 
-ADMIN_HOST = "admin.archivebox.localhost:8000"
-API_HOST = "api.archivebox.localhost:8000"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
+API_HOST = "api.archivebox.localhost:5797"
 
 
 def test_delete_after_real_cli_and_orchestrator_paths_cover_all_retained_models(tmp_path, recursive_test_site):

archivebox/tests/test_machine_models.py

@@ -620,7 +620,7 @@ class TestProcessCurrent:
         """runner_watch should be classified as a worker, not the orchestrator itself."""
         old_argv = sys.argv
         try:
-            sys.argv = ["archivebox", "manage", "runner_watch", "--bind-url=http://127.0.0.1:8000"]
+            sys.argv = ["archivebox", "manage", "runner_watch", "--bind-url=http://127.0.0.1:5797"]
             result = Process._detect_process_type()
             assert result == Process.TypeChoices.WORKER
         finally:

archivebox/tests/test_migrations_04_to_09.py

@@ -85,8 +85,11 @@ def test_oldest_django_collection_migrates_end_to_end_without_data_loss(tmp_path
     result = run_archivebox_migration_cmd(tmp_path, ["init"], timeout=90)
     assert result.returncode == 0, result.stderr
     assert "received a naive datetime" not in result.stderr
+    result = run_archivebox_migration_cmd(tmp_path, ["update"], timeout=60)
+    assert result.returncode == 0, result.stderr
+    assert all((tmp_path / "archive" / timestamp).exists() for timestamp in original_trees)
     for pass_number in (1, 2):
-        result = run_archivebox_migration_cmd(tmp_path, ["update"], timeout=180)
+        result = run_archivebox_migration_cmd(tmp_path, ["update", "--migrate-only"], timeout=180)
         assert result.returncode == 0, f"Update pass {pass_number} failed: {result.stderr}"
 
     for command in (["status"], ["list", "--json"]):

archivebox/tests/test_migrations_07_to_09.py

@@ -178,7 +178,7 @@ def test_legacy_onedomain_server_upgrade_works_in_auto_mode(archive_07):
         """[SERVER_CONFIG]
 LISTEN_HOST = archivebox.mydomain
 ALLOWED_HOSTS = archivebox.mydomain
-CSRF_TRUSTED_ORIGINS = http://admin.archivebox.localhost:8000
+CSRF_TRUSTED_ORIGINS = http://admin.archivebox.localhost:5797
 PUBLIC_INDEX = True
 PUBLIC_SNAPSHOTS = True
 PUBLIC_ADD_VIEW = False

archivebox/tests/test_migrations_08_to_09.py

@@ -720,7 +720,7 @@ def test_update_migrates_db_snapshot_when_legacy_index_missing(tmp_path):
     result = run_archivebox_migration_cmd(work_dir, ["init"], timeout=60)
     assert result.returncode == 0, f"Init failed: {result.stderr}"
     for pass_number in (1, 2):
-        result = run_archivebox_migration_cmd(work_dir, ["update"], timeout=120)
+        result = run_archivebox_migration_cmd(work_dir, ["update", "--rescan", "--migrate-only"], timeout=120)
         assert result.returncode == 0, f"Update pass {pass_number} failed: {result.stderr}"
 
     migrated_files = list((work_dir / "archive" / "users").glob("*/snapshots/*/*/*/screenshot.png"))
@@ -749,7 +749,7 @@ def test_update_recovers_orphan_with_corrupt_index_from_archive_org_url(tmp_path
 
     result = run_archivebox_migration_cmd(work_dir, ["init"], timeout=60)
     assert result.returncode == 0, f"Init failed: {result.stderr}"
-    result = run_archivebox_migration_cmd(work_dir, ["update"], timeout=120)
+    result = run_archivebox_migration_cmd(work_dir, ["update", "--rescan", "--migrate-only"], timeout=120)
     assert result.returncode == 0, f"Update failed: {result.stderr}"
 
     conn = sqlite3.connect(str(db_path))
@@ -792,7 +792,7 @@ def test_update_preserves_legacy_folder_timestamp_over_index_float_variant(tmp_p
 
     result = run_archivebox_migration_cmd(work_dir, ["init"], timeout=60)
     assert result.returncode == 0, f"Init failed: {result.stderr}"
-    result = run_archivebox_migration_cmd(work_dir, ["update"], timeout=120)
+    result = run_archivebox_migration_cmd(work_dir, ["update", "--rescan", "--migrate-only"], timeout=120)
     assert result.returncode == 0, f"Update failed: {result.stderr}"
 
     conn = sqlite3.connect(str(db_path))
@@ -834,7 +834,7 @@ def test_update_preserves_distinct_legacy_dirs_with_integer_and_float_timestamps
 
     result = run_archivebox_migration_cmd(work_dir, ["init"], timeout=60)
     assert result.returncode == 0, f"Init failed: {result.stderr}"
-    result = run_archivebox_migration_cmd(work_dir, ["update"], timeout=120)
+    result = run_archivebox_migration_cmd(work_dir, ["update", "--rescan", "--migrate-only"], timeout=120)
     assert result.returncode == 0, f"Update failed: {result.stderr}"
 
     conn = sqlite3.connect(str(db_path))
@@ -885,7 +885,7 @@ def test_update_preserves_legacy_plugin_directory_without_output_files(migration
     result = run_archivebox_migration_cmd(work_dir, ["init"], timeout=60)
     assert result.returncode == 0, f"Init failed: {result.stderr}"
     for pass_number in (1, 2):
-        result = run_archivebox_migration_cmd(work_dir, ["update"], timeout=120)
+        result = run_archivebox_migration_cmd(work_dir, ["update", "--rescan", "--migrate-only"], timeout=120)
         assert result.returncode == 0, f"Update pass {pass_number} failed: {result.stderr}"
 
     migrated_outputs = list((work_dir / "archive" / "users").glob("*/snapshots/*/*/*/media/stderr.log"))

archivebox/tests/test_opencode_agent.py

@@ -1,9 +1,11 @@
 import asyncio
+import json
 import os
 import re
 import shutil
 import socket
 from concurrent.futures import ThreadPoolExecutor
+from concurrent.futures import TimeoutError as FutureTimeoutError
 from pathlib import Path
 from types import SimpleNamespace
 from urllib.parse import parse_qs, quote, urlsplit
@@ -92,9 +94,9 @@ def installed_opencode(opencode_archive_config):
 
     config = get_config().model_dump(mode="json")
     settings = runtime._settings(config, opencode_archive_config.data_dir)
-    settings["archivebox_base_url"] = "http://admin.archivebox.localhost:8000"
-    settings["archivebox_admin_url"] = "http://admin.archivebox.localhost:8000/admin"
-    settings["archivebox_api_url"] = "http://admin.archivebox.localhost:8000/api/"
+    settings["archivebox_base_url"] = "http://admin.archivebox.localhost:5797"
+    settings["archivebox_admin_url"] = "http://admin.archivebox.localhost:5797/admin"
+    settings["archivebox_api_url"] = "http://admin.archivebox.localhost:5797/api/"
     binary, binary_env = runtime._resolve_binary(settings["binary"], settings["config"])
     version = binary.exec(
         cmd=("--version",),
@@ -359,6 +361,45 @@ def test_opencode_proxy_preserves_protocol_headers(admin_client, live_opencode):
     assert deleted.status_code == 200, deleted.content
 
 
+def test_opencode_oauth_callback_waits_for_user_and_preserves_cancellation(live_opencode):
+    from abx_plugins.plugins.opencode import runtime
+
+    settings = {**live_opencode.settings, "timeout": 1}
+    headers = {"Content-Type": "application/json"}
+    status, _, body = runtime.proxy(settings, "POST", "provider/openai/oauth/authorize", (), headers, b'{"method":0}')
+    assert status == 200
+    authorization = json.loads(body)
+    assert urlsplit(authorization["url"]).hostname == "auth.openai.com"
+    redirect = parse_qs(urlsplit(authorization["url"]).query)["redirect_uri"][0]
+    callback_origin = urlsplit(redirect)
+    assert callback_origin.hostname == "localhost"
+
+    with ThreadPoolExecutor(max_workers=1) as executor:
+        callback = executor.submit(runtime.proxy, settings, "POST", "provider/openai/oauth/callback", (), headers, b'{"method":0}')
+        try:
+            # A real pending authorization must outlive the ordinary API read
+            # timeout. No tokens or substituted provider responses are used.
+            with pytest.raises(FutureTimeoutError):
+                callback.result(timeout=2)
+        finally:
+            cancelled = requests.get(f"{callback_origin.scheme}://{callback_origin.netloc}/cancel", timeout=5)
+            assert cancelled.status_code == 200
+            assert cancelled.text == "Login cancelled"
+        status, response_headers, body = callback.result(timeout=5)
+    assert status == 500
+    assert response_headers["Content-Type"].startswith("application/json")
+    error = json.loads(body)
+    assert error["name"] == "UnknownError"
+    assert error["data"]["ref"].startswith("err_")
+    runtime._stop_owned_process()
+    # The pnpm launcher can exit before its server child. Stopping the owned
+    # process must release both listeners so another collection can authorize.
+    with pytest.raises(requests.ConnectionError):
+        requests.get(settings["origin"] + "/global/health", timeout=2)
+    with pytest.raises(requests.ConnectionError):
+        requests.get(f"{callback_origin.scheme}://{callback_origin.netloc}/cancel", timeout=2)
+
+
 def test_opencode_static_assets_cache_privately_and_revalidate(admin_client, live_opencode):
     headers = {"HTTP_HOST": ADMIN_TEST_HOST, "HTTP_SEC_FETCH_SITE": "same-origin"}
     page = admin_client.get("/admin/agent/opencode/", **headers)
@@ -392,9 +433,26 @@ def test_opencode_proxy_sse_response_is_unbuffered(admin_client, live_opencode):
     assert response.headers["Cache-Control"] == "no-store"
 
 
-def test_opencode_proxy_sse_delivers_first_event_immediately(admin_client, live_opencode):
+@pytest.mark.parametrize("path", ["event", "global/event"])
+def test_opencode_proxy_sse_delivers_first_event_immediately(admin_client, live_opencode, path):
+    from abx_plugins.plugins.opencode import runtime
+
+    # The real /event endpoint emits heartbeats indefinitely. A bounded
+    # upstream read exposes accidental buffering without hanging the test.
+    status, headers, body = runtime.proxy(
+        {**live_opencode.settings, "timeout": 1},
+        "GET",
+        path,
+        (),
+        {},
+        b"",
+    )
+    assert status == 200
+    assert not isinstance(body, bytes)
+    assert headers["Content-Type"] == "text/event-stream"
+    asyncio.run(body.aclose())
     response = admin_client.get(
-        "/admin/agent/opencode/global/event",
+        f"/admin/agent/opencode/{path}",
         HTTP_HOST=ADMIN_TEST_HOST,
         HTTP_SEC_FETCH_SITE="same-origin",
     )
@@ -440,7 +498,7 @@ def test_opencode_proxy_sse_returns_headers_before_restart_finishes(admin_client
                     (b"sec-fetch-site", b"same-origin"),
                 ],
                 "client": ("127.0.0.1", 12345),
-                "server": ("127.0.0.1", 8000),
+                "server": ("127.0.0.1", 5797),
             },
         )
         runtime._PROCESS_LOCK.acquire()

archivebox/tests/test_opencode_browser.py

@@ -20,8 +20,21 @@ from .test_server_security_browser import browser_runtime as browser_runtime
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
+@pytest.mark.parametrize(
+    "agent_server",
+    [
+        ("safe-onedomain-nojsreplay", False),
+        ("safe-subdomains-fullreplay", True),
+        ("danger-onedomain-fullreplay", False),
+        ("auto", False),
+        ("auto", True),
+    ],
+    indirect=True,
+)
 def test_agent_navigation_stays_inside_mount(agent_server, browser_runtime):
-    server_url, _, _ = agent_server
+    server_url, data_dir, _ = agent_server
+    filename = "proxy acceptance #é.txt"
+    (data_dir / filename).write_text("Proxy file read: café", encoding="utf-8")
     script = r"""
 const assert = require('node:assert/strict');
 const puppeteer = require('puppeteer');
@@ -31,6 +44,14 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
     args: ['--no-sandbox', '--disable-frame-rate-limit']});
   try {
     const page = await browser.newPage();
+    const escapedRequests = [];
+    page.on('request', request => {
+      const url = new URL(request.url());
+      if (request.frame()?.parentFrame() && url.origin === config.url &&
+          !url.pathname.startsWith('/admin/agent/opencode/')) {
+        escapedRequests.push(request.method() + ' ' + url.pathname);
+      }
+    });
     await page.setViewport({width: 1440, height: 900});
     await page.goto(config.url + '/admin/login/', {waitUntil: 'domcontentloaded'});
     await page.locator('#login-form input[name="username"]').fill('agent-browser-test');
@@ -44,7 +65,9 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
     const links = await frame.$$eval('a[href*="/session"]', nodes => nodes.map(node => node.getAttribute('href')));
     assert.ok(links.length, 'OpenCode must expose session navigation');
     for (const href of links) assert.ok(href.startsWith('/admin/agent/opencode/'), href);
+    const initialSessionUrl = frame.url();
     await frame.locator('::-p-aria(New session[role="button"])').click();
+    await frame.waitForFunction(previous => location.href !== previous, {}, initialSessionUrl);
     await frame.waitForSelector('[contenteditable="true"]');
     let navigation;
     for (const link of await frame.$$('a[href*="/session"]')) {
@@ -61,6 +84,101 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
     assert.equal((await frame.goto(sessionUrl, {waitUntil: 'domcontentloaded'})).status(), 200);
     await frame.waitForSelector('[contenteditable="true"]');
     assert.ok(!(await frame.$eval('body', node => node.innerText)).includes('Something went wrong'));
+    // Real JSON methods, encoded file paths, and both SSE subscriptions must
+    // work on the admin origin in every routing mode, without installing or
+    // authenticating a paid model provider during the test.
+    const requests = await frame.evaluate(async filename => {
+      const base = location.origin + '/admin/agent/opencode';
+      const json = async (path, method = 'GET', body) => {
+        const response = await fetch(base + path, {
+          method, headers: {'Content-Type': 'application/json'},
+          ...(body === undefined ? {} : {body: JSON.stringify(body)}),
+        });
+        if (!response.ok) throw new Error(method + ' ' + path + ': ' + response.status);
+        if (!response.headers.get('Content-Type')?.includes('application/json')) {
+          throw new Error('Non-JSON response: ' + path);
+        }
+        return response.json();
+      };
+      const providers = await json('/provider');
+      const auth = await json('/provider/auth');
+      const file = await json('/file/content?path=' + encodeURIComponent(filename));
+      const session = await json('/session', 'POST', {title: 'Proxy request verification'});
+      let renamed;
+      try {
+        renamed = await json('/session/' + session.id, 'PATCH', {title: 'Proxy: café & # symbols'});
+        const readback = await json('/session/' + session.id);
+        if (readback.title !== renamed.title) throw new Error('Session mutation was not persisted');
+      } finally { await json('/session/' + session.id, 'DELETE'); }
+      const streams = [];
+      for (const path of ['/event', '/global/event']) {
+        const controller = new AbortController();
+        const timeout = setTimeout(() => controller.abort(), 5000);
+        try {
+          const response = await fetch(base + path, {signal: controller.signal});
+          if (!response.ok || !response.headers.get('Content-Type')?.includes('text/event-stream')) {
+            throw new Error('Invalid event stream: ' + path);
+          }
+          const reader = response.body.getReader();
+          const first = await reader.read();
+          streams.push(new TextDecoder().decode(first.value));
+          await reader.cancel();
+        } finally { clearTimeout(timeout); controller.abort(); }
+      }
+      return {providers: providers.all.map(provider => provider.id), auth, file, renamed, streams};
+    }, config.filename);
+    for (const provider of ['openai', 'anthropic']) assert.ok(requests.providers.includes(provider), provider);
+    assert.ok(requests.auth.openai.some(method => method.type === 'oauth'));
+    assert.equal(requests.file.content, 'Proxy file read: café');
+    assert.equal(requests.renamed.title, 'Proxy: café & # symbols');
+    for (const event of requests.streams) assert.ok(event.includes('server.connected'), event);
+    for (const provider of ['OpenAI', 'Anthropic']) {
+      await frame.locator('[data-action="prompt-model"]').click();
+      await frame.locator('::-p-aria(' + provider + '[role="button"])').click();
+      if (provider === 'OpenAI') {
+        await frame.waitForSelector('::-p-aria(ChatGPT Pro/Plus Headless[role="button"])');
+        await frame.waitForSelector('::-p-aria(ChatGPT Pro/Plus Browser[role="button"])');
+        await frame.locator('::-p-aria(API key Browser[role="button"])').click();
+      }
+      const input = await frame.waitForSelector('::-p-aria(' + provider + ' API key[role="textbox"])');
+      assert.equal(await input.evaluate(node => node.value), '');
+      await frame.waitForSelector('::-p-aria(Continue[role="button"])');
+      await frame.locator('::-p-aria(Close[role="button"])').click();
+    }
+    // Submit a real shell command from a fresh draft. This exercises draft
+    // promotion and the shell API without needing a paid model or API key.
+    await frame.locator('::-p-aria(New session[role="button"])').click();
+    await frame.waitForFunction(() => location.pathname.endsWith('/new-session'));
+    await (await frame.waitForSelector('[contenteditable="true"]')).focus();
+    const modifier = await page.evaluate(() => navigator.platform.includes('Mac') ? 'Meta' : 'Control');
+    await page.keyboard.down(modifier);
+    await page.keyboard.down('Shift');
+    await page.keyboard.press('x');
+    await page.keyboard.up('Shift');
+    await page.keyboard.up(modifier);
+    await page.keyboard.type('printf ABX_DRAFT_SHELL_OK');
+    await frame.locator('::-p-aria(Send[role="button"])').click();
+    await frame.waitForFunction(() => /\/session\/ses_/.test(location.pathname));
+    assert.ok(new URL(frame.url()).pathname.startsWith('/admin/agent/opencode/'), frame.url());
+    await frame.waitForFunction(async () => {
+      const sessionID = location.pathname.match(/\/session\/(ses_[^/]+)/)[1];
+      const response = await fetch('/admin/agent/opencode/session/' + sessionID + '/message');
+      if (!response.ok) throw new Error('Shell message readback: ' + response.status);
+      const messages = await response.json();
+      return messages.some(message => message.parts.some(part =>
+        part.type === 'tool' && part.state.status === 'completed' &&
+        part.state.output.trim() === 'ABX_DRAFT_SHELL_OK'));
+    }, {polling: 250});
+    // The tabbed layout's own route parser must recognize mounted sessions.
+    // Otherwise Home incorrectly appears selected and clicking it does nothing.
+    const shellSessionUrl = frame.url();
+    const home = await frame.waitForSelector('[aria-label="Home"]');
+    assert.equal(await home.evaluate(node => node.getAttribute('aria-pressed')), 'false');
+    await home.click();
+    await frame.waitForFunction(() => location.pathname.replace(/\/+$/, '') === '/admin/agent/opencode');
+    assert.equal(await home.evaluate(node => node.getAttribute('aria-pressed')), 'true');
+    await home.click();
+    await frame.waitForFunction(expected => location.href === expected, {}, shellSessionUrl);
     // Exercise the actual public PTY API and native browser WebSocket. No
     // intercepted traffic or replacement server: this runs a real shell.
     const terminal = await frame.evaluate(async () => {
@@ -97,6 +215,7 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
       } finally { await fetch(base + '/pty/' + pty.id, {method: 'DELETE'}); }
     });
     assert.ok(terminal.includes('ABX_TERMINAL_OK'), terminal);
+    assert.deepEqual(escapedRequests, [], 'OpenCode requests must stay inside its proxy mount');
     assert.equal((await page.goto(config.url + '/add/', {waitUntil: 'domcontentloaded'})).status(), 200);
     console.log('AGENT_NAVIGATION_OK');
   } finally { await browser.close(); }
@@ -104,7 +223,7 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
 """
     result = subprocess.run(
         [str(browser_runtime["node_binary"]), "-e", script],
-        input=json.dumps({"chrome": str(browser_runtime["chrome_binary"]), "url": server_url}),
+        input=json.dumps({"chrome": str(browser_runtime["chrome_binary"]), "url": server_url, "filename": filename}),
         env={**os.environ, "NODE_PATH": browser_runtime["node_path"]},
         text=True,
         capture_output=True,
@@ -115,14 +234,16 @@ const config = JSON.parse(require('node:fs').readFileSync(0, 'utf8'));
 
 
 @pytest.fixture
-def agent_server(installed_opencode, browser_runtime):
+def agent_server(installed_opencode, browser_runtime, request):
     port = get_free_port()
-    url = f"http://localhost:{port}"
+    mode, subdomains = getattr(request, "param", ("safe-onedomain-nojsreplay", False))
+    base_url = f"http://archivebox.localhost:{port}" if subdomains else f"http://localhost:{port}"
+    url = f"http://admin.archivebox.localhost:{port}" if subdomains else base_url
     config = installed_opencode.config
     _set_archivebox_config(
         config.data_dir,
-        f"BASE_URL={url}",
-        "SERVER_SECURITY_MODE=safe-onedomain-nojsreplay",
+        f"BASE_URL={base_url}",
+        f"SERVER_SECURITY_MODE={mode}",
     )
     user = run_archivebox_cmd(
         [

archivebox/tests/test_search.py

@@ -32,8 +32,8 @@ from archivebox.tests.conftest import (
 pytestmark = pytest.mark.django_db(transaction=True)
 
 User = get_user_model()
-ADMIN_HOST = "admin.archivebox.localhost:8000"
-WEB_HOST = "web.archivebox.localhost:8000"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
+WEB_HOST = "web.archivebox.localhost:5797"
 
 
 @pytest.fixture

archivebox/tests/test_server_automatic_url.py

@@ -0,0 +1,61 @@
+"""Exercise automatic server addressing through real HTTP login pages."""
+
+import re
+
+import requests
+
+from .conftest import cli_env, get_free_port, run_archivebox_cmd, start_archivebox_server, stop_archivebox_process
+
+
+def test_safe_single_domain_login_does_not_require_base_url(tmp_path):
+    port = get_free_port()
+    env = cli_env(
+        disable_extractors=True,
+        BASE_URL="",
+        SERVER_SECURITY_MODE="safe-onedomain-nojsreplay",
+        ADMIN_USERNAME="automatic-url-test",
+        ADMIN_PASSWORD="Automatic-url-test-password-93!",
+        ALLOWED_HOSTS="*",
+    )
+    result = run_archivebox_cmd(["init", "--quick"], cwd=tmp_path, env=env, timeout=60)
+    assert result.returncode == 0, result.stderr or result.stdout
+    process = start_archivebox_server(tmp_path, port=port, env=env, daemonize=False, log_name="server.log")
+    try:
+        for hostname in ("localhost", "archivebox.localhost", "admin.archivebox.localhost", "192.168.1.20", "100.100.10.20"):
+            session = requests.Session()
+            response = session.get(
+                f"http://127.0.0.1:{port}/admin/login/",
+                headers={"Host": f"{hostname}:{port}"},
+                timeout=15,
+                allow_redirects=False,
+            )
+            assert response.status_code == 200, (hostname, response.status_code)
+            assert 'name="password"' in response.text, hostname
+            assert "base_url not set" not in response.text.lower(), hostname
+            assert "archivebox-setup-wizard" not in response.text, hostname
+            csrf = re.search(r'name="csrfmiddlewaretoken" value="([^"]+)"', response.text)
+            assert csrf, hostname
+            login = session.post(
+                f"http://127.0.0.1:{port}/admin/login/",
+                headers={"Host": f"{hostname}:{port}"},
+                data={
+                    "username": env["ADMIN_USERNAME"],
+                    "password": env["ADMIN_PASSWORD"],
+                    "csrfmiddlewaretoken": csrf[1],
+                    "next": "/admin/",
+                },
+                timeout=15,
+                allow_redirects=False,
+            )
+            assert login.status_code == 302, (hostname, login.status_code)
+            admin = session.get(
+                f"http://127.0.0.1:{port}/admin/",
+                headers={"Host": f"{hostname}:{port}"},
+                timeout=15,
+                allow_redirects=False,
+            )
+            assert admin.status_code == 200, (hostname, admin.status_code)
+            assert "base_url not set" not in admin.text.lower(), hostname
+            assert "archivebox-setup-wizard" not in admin.text, hostname
+    finally:
+        stop_archivebox_process(process)

archivebox/tests/test_server_security_browser.py

@@ -749,7 +749,7 @@ def test_unconfigured_public_host_superuser_can_reach_setup_wizard(tmp_path: Pat
     assert "BASE_URL" in archivebox_environment
     assert "SERVER_SECURITY_MODE" in archivebox_environment
     assert all("ARCHIVEBOX_INGRESS_BASE_URL" not in value for value in archivebox_environment)
-    assert compose["services"]["archivebox"]["ports"] == ["${ARCHIVEBOX_PORT:-8000}:8000"]
+    assert compose["services"]["archivebox"]["ports"] == ["${ARCHIVEBOX_PORT:-5797}:5797"]
     assert set(compose["services"]) == {"archivebox"}
 
 

archivebox/tests/test_ui_add_view.py

@@ -16,8 +16,8 @@ from archivebox.workers.models import RETRY_AT_MAX
 pytestmark = pytest.mark.django_db
 
 User = get_user_model()
-WEB_HOST = "web.archivebox.localhost:8000"
-ADMIN_HOST = "admin.archivebox.localhost:8000"
+WEB_HOST = "web.archivebox.localhost:5797"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
 
 
 ADD_FORM_DEFAULTS = {
@@ -99,7 +99,7 @@ def test_add_view_admin_renders_plugin_config_grid(client, admin_user):
     assert b">Source</a>" in response.content
     assert b">Docs</a>" in response.content
     assert b"https://github.com/ArchiveBox/abx-plugins/tree/main/abx_plugins/plugins/" in response.content
-    assert b"https://archivebox.github.io/abx-plugins/#" in response.content
+    assert b"https://plugins.archivebox.io/#" in response.content
     assert not any(
         field["key"].endswith("_BINARY") for group in form.plugin_groups for card in group["plugins"] for field in card["config_fields"]
     )

archivebox/tests/test_ui_admin_config_widget.py

@@ -99,8 +99,8 @@ def test_unconfigured_superuser_banner_uses_browser_assisted_setup_wizard():
         {
             "mode": "unconfigured",
             "can_configure": True,
-            "canonical_host": "archivebox.example.test:8000",
-            "suggested_base_url": "http://archivebox.example.test:8000",
+            "canonical_host": "archivebox.example.test:5797",
+            "suggested_base_url": "http://archivebox.example.test:5797",
             "machine_admin_url": "/admin/machine/machine/current/change/",
             "public_index": True,
             "public_add_view": False,
@@ -111,7 +111,7 @@ def test_unconfigured_superuser_banner_uses_browser_assisted_setup_wizard():
     assert html.count('id="archivebox-setup-wizard"') == 1
     assert 'id="archivebox-system-warning-banner"' not in html
     assert "⚠ base_url not set" not in html
-    assert "Setup access to your ArchiveBox Server: <code>archivebox.example.test:8000</code>" in html
+    assert "Setup access to your ArchiveBox Server: <code>archivebox.example.test:5797</code>" in html
     assert 'id="archivebox-setup-base-url"' in html
     assert 'id="archivebox-setup-security-mode"' in html
     assert 'id="archivebox-setup-public-index"' in html
@@ -245,9 +245,9 @@ def test_health_check_is_identifiable_across_ingress_origins():
 @pytest.mark.parametrize(
     ("request_host", "secure", "expected_display_host"),
     (
-        ("admin.archivebox.localhost:8000", False, "archivebox.localhost:8000"),
-        ("127.0.0.1:8000", False, "archivebox.localhost:8000"),
-        ("0.0.0.0:8000", False, "archivebox.localhost:8000"),
+        ("admin.archivebox.localhost:5797", False, "archivebox.localhost:5797"),
+        ("127.0.0.1:5797", False, "archivebox.localhost:5797"),
+        ("0.0.0.0:5797", False, "archivebox.localhost:5797"),
         ("archivebox.io:80", False, "archivebox.io"),
         ("archivebox.io:443", True, "archivebox.io"),
         ("192.0.2.10:443", True, "192.0.2.10"),
@@ -279,7 +279,7 @@ def test_unconfigured_banner_honors_forwarded_https_from_ingress():
     ("base_url", "request_host"),
     (
         ("", "archivebox.example.test:18443"),
-        ("http://archivebox.localhost:8000", "admin.archivebox.localhost:18010"),
+        ("http://archivebox.localhost:5797", "admin.archivebox.localhost:18010"),
     ),
 )
 def test_admin_cookie_isolation_accepts_first_run_and_external_port_mapping(base_url, request_host):
@@ -302,7 +302,7 @@ def test_unconfigured_banner_does_not_show_setup_wizard_to_non_superusers():
         {
             "mode": "unconfigured",
             "can_configure": False,
-            "suggested_base_url": "http://archivebox.example.test:8000",
+            "suggested_base_url": "http://archivebox.example.test:5797",
         },
     )
 
@@ -327,25 +327,25 @@ def test_system_warning_modes_share_one_banner(context, expected_text):
     assert expected_text in html
 
 
-def test_configured_base_url_mismatch_banner_shows_both_origins():
+def test_configured_base_url_mismatch_banner_links_to_configured_origin():
     config = get_config(include_machine=False).model_copy(update={"BASE_URL": "https://archivebox.example.test"})
-    request = RequestFactory().get("/admin/", HTTP_HOST="archivebox.internal:8000")
+    request = RequestFactory().get("/admin/", HTTP_HOST="archivebox.internal:5797")
     request.user = AnonymousUser()
 
     context = get_base_url_mismatch_context(request, config)
     assert context == {
         "mode": "base_url_mismatch",
-        "browser_url": "http://archivebox.internal:8000",
+        "browser_url": "http://archivebox.internal:5797",
         "configured_base_url": "https://archivebox.example.test",
     }
     assert system_warnings_banner({"CONFIG": config, "request": request}) == context
 
     html = render_to_string("core/system_warnings_banner.html", context)
     assert "base_url mismatch" in html
-    assert "Browser URL:" in html
-    assert "http://archivebox.internal:8000" in html
-    assert "Configured BASE_URL:" in html
-    assert "https://archivebox.example.test" in html
+    assert "Editing disabled from unrecognized URL. Visit" in html
+    assert 'href="https://archivebox.example.test"' in html
+    assert ">https://archivebox.example.test</a>" in html
+    assert "to make changes." in html
 
 
 def test_configured_base_url_accepts_its_isolated_admin_subdomain():

archivebox/tests/test_ui_admin_crawl.py

@@ -416,7 +416,7 @@ def test_snapshot_from_json_trims_markdown_suffixes_on_discovered_urls(crawl):
 
 def test_create_snapshots_from_urls_skips_invalid_and_archivebox_internal_urls(admin_user):
     crawl = Crawl.objects.create(
-        urls=("https://example.com/root\nhttp://127.0.0.1:8765/page-001.html\nnot-a-url\nhttp://admin.archivebox.localhost:8000/admin/"),
+        urls=("https://example.com/root\nhttp://127.0.0.1:8765/page-001.html\nnot-a-url\nhttp://admin.archivebox.localhost:5797/admin/"),
         created_by=admin_user,
     )
 

archivebox/tests/test_ui_admin_links.py

@@ -130,7 +130,7 @@ def _admin_post_request(path):
 def _admin_get_request(path="/"):
     from archivebox.config.common import get_config
 
-    request = RequestFactory().get(path, HTTP_HOST="admin.archivebox.localhost:8000")
+    request = RequestFactory().get(path, HTTP_HOST="admin.archivebox.localhost:5797")
     request.archivebox_config = get_config()
     return request
 
@@ -284,7 +284,7 @@ def test_admin_navigation_hides_agent_link_when_opencode_is_disabled(client, adm
     Machine.from_json({"config": {"OPENCODE_ENABLED": False}})
     client.force_login(admin_user)
 
-    response = client.get(reverse("admin:index"), HTTP_HOST="admin.archivebox.localhost:8000")
+    response = client.get(reverse("admin:index"), HTTP_HOST="admin.archivebox.localhost:5797")
 
     assert response.status_code == 200
     assert b"/admin/agent" not in response.content

archivebox/tests/test_ui_admin_persona.py

@@ -127,6 +127,10 @@ def test_persona_admin_add_view_renders_import_ui(admin_client):
     assert b"auth.json" in response.content
     assert b"Plugin Config" in response.content
     assert b'name="plugin_config__wget__WGET_TIMEOUT"' in response.content
+    assert b'<fieldset id="plugin-config" class="module aligned wide persona-plugin-config">' in response.content
+    assert b'href="https://extension.archivebox.io/"' in response.content
+    assert b"import cookies into this persona from another browser" in response.content
+    assert "wide" in response.context["adminform"].fieldsets[0][1]["classes"]
 
 
 def test_import_persona_from_source_copies_profile_without_browser_export(admin_user):

archivebox/tests/test_ui_admin_snapshot.py

@@ -13,7 +13,7 @@ import pytest
 from django.contrib.auth.models import AnonymousUser
 from django.contrib.admin.helpers import ACTION_CHECKBOX_NAME
 from django.core.paginator import UnorderedObjectListWarning
-from django.test import RequestFactory
+from django.test import Client, RequestFactory
 from django.urls import reverse
 
 from archivebox.core.middleware import ADMIN_LOGIN_HINT_COOKIE
@@ -152,6 +152,8 @@ def test_snapshot_changelist_uses_stable_ordering_without_unordered_paginator_wa
 def test_snapshot_changelist_preview_uses_prefetched_output_files(admin_client, snapshot, real_hash_projection):
     from archivebox.core.models import ArchiveResult
 
+    snapshot.title = "Preview destination test"
+    snapshot.save(update_fields=["title"])
     _process, result = real_hash_projection
     ArchiveResult.objects.filter(pk=result.pk).update(
         plugin="screenshot",
@@ -165,6 +167,103 @@ def test_snapshot_changelist_preview_uses_prefetched_output_files(admin_client,
     assert response.status_code == 200
     assert b'class="snapshot-preview' in response.content
     assert b"screenshot.png" in response.content
+    html = response.content.decode()
+    preview_cell = re.search(r'<td class="field-preview_icon">(.*?)</td>', html, re.S)
+    title_cell = re.search(r'<td class="field-title_str">(.*?)</td>', html, re.S)
+    assert preview_cell and title_cell
+    preview_link = re.search(r'<a href="([^"]+)"', preview_cell.group(1))
+    title_link = re.search(r'<a href="([^"]+)"', title_cell.group(1))
+    assert preview_link and title_link
+    assert preview_link.group(1) == title_link.group(1)
+    assert preview_link.group(1).endswith(f"/{snapshot.archive_path_from_db}/index.html")
+
+
+def test_snapshot_grid_missing_preview_is_inside_snapshot_detail_link(admin_client, snapshot):
+    snapshot.status = snapshot.StatusChoices.SEALED
+    snapshot.save(update_fields=["status"])
+
+    response = admin_client.get(reverse("admin:grid"), HTTP_HOST=ADMIN_TEST_HOST)
+
+    assert response.status_code == 200
+    html = response.content.decode()
+    thumbnail_link = re.search(
+        r'<a href="([^"]+)" class="card-thumbnail">(.*?)</a>',
+        html,
+        re.S,
+    )
+    title_link = re.search(
+        r'<div class="card-title"[^>]*>\s*<a href="([^"]+)">',
+        html,
+        re.S,
+    )
+    assert thumbnail_link and title_link
+    assert thumbnail_link.group(1) == title_link.group(1)
+    assert thumbnail_link.group(1).endswith("/index.html")
+    assert '<span class="missing-preview">No preview captured</span>' in thumbnail_link.group(2)
+
+
+def test_snapshot_grid_serves_extension_preview_from_authenticated_admin_origin(admin_client, snapshot, real_hash_projection):
+    _process, result = real_hash_projection
+    preview_dir = Path(snapshot.output_dir) / "chrome_extension_screenshot"
+    preview_dir.mkdir(parents=True, exist_ok=True)
+    preview_bytes = b"\x89PNG\r\n\x1a\nreal-extension-preview"
+    (preview_dir / "screenshot.png").write_bytes(preview_bytes)
+    result.plugin = "chrome_extension_screenshot"
+    result.status = "succeeded"
+    result.output_files = {"screenshot.png": {"size": len(preview_bytes), "mimetype": "image/png"}}
+    result.save(update_fields=["plugin", "status", "output_files"])
+
+    grid_response = admin_client.get(reverse("admin:grid"), HTTP_HOST=ADMIN_TEST_HOST)
+
+    assert grid_response.status_code == 200
+    html = grid_response.content.decode()
+    preview_url = reverse(
+        "admin:core_snapshot_preview",
+        args=(snapshot.pk, "chrome_extension_screenshot", "screenshot.png"),
+    )
+    assert f'src="{preview_url}"' in html
+    assert "new URL(candidate, document.baseURI).href" in html
+    assert "candidate !== current" in html
+
+    preview_response = admin_client.get(preview_url, HTTP_HOST=ADMIN_TEST_HOST)
+
+    assert preview_response.status_code == 200
+    assert b"".join(preview_response.streaming_content) == preview_bytes
+    assert preview_response["Content-Type"] == "image/png"
+    assert {part.strip() for part in preview_response["Cache-Control"].split(",")} == {
+        "private",
+        "max-age=604800",
+        "immutable",
+    }
+    assert {part.strip() for part in preview_response["Vary"].split(",")} >= {"Cookie", "Authorization"}
+    assert preview_response["X-Content-Type-Options"] == "nosniff"
+
+    not_modified = admin_client.get(
+        preview_url,
+        HTTP_HOST=ADMIN_TEST_HOST,
+        HTTP_IF_MODIFIED_SINCE=preview_response["Last-Modified"],
+    )
+    assert not_modified.status_code == 304
+    assert {part.strip() for part in not_modified["Cache-Control"].split(",")} == {
+        "private",
+        "max-age=60",
+        "stale-while-revalidate=300",
+    }
+    assert {part.strip() for part in not_modified["Vary"].split(",")} >= {"Cookie", "Authorization"}
+
+    denied = Client().get(preview_url, HTTP_HOST=ADMIN_TEST_HOST)
+    assert denied.status_code in (301, 302)
+    assert {part.strip() for part in denied["Cache-Control"].split(",")} == {"private", "no-store"}
+
+    disallowed_dir = Path(snapshot.output_dir) / "singlefile"
+    disallowed_dir.mkdir(parents=True, exist_ok=True)
+    (disallowed_dir / "singlefile.html").write_text("<script>document.cookie</script>", encoding="utf-8")
+    result.plugin = "singlefile"
+    result.output_files = {"singlefile.html": {"size": 32, "mimetype": "text/html"}}
+    result.save(update_fields=["plugin", "output_files"])
+    disallowed_url = reverse("admin:core_snapshot_preview", args=(snapshot.pk, "singlefile", "singlefile.html"))
+
+    assert admin_client.get(disallowed_url, HTTP_HOST=ADMIN_TEST_HOST).status_code == 404
 
 
 def test_snapshot_result_health_filter_uses_live_status_rows(admin_client, snapshot):
@@ -312,7 +411,7 @@ def test_snapshot_admin_preview_uses_extension_screenshot_when_standard_screensh
     )
 
     admin = SnapshotAdmin(Snapshot, archivebox_admin)
-    request = RequestFactory().get("/", HTTP_HOST="admin.archivebox.localhost:8000")
+    request = RequestFactory().get("/", HTTP_HOST="admin.archivebox.localhost:5797")
     request.archivebox_config = get_config()
     admin.request = request
 
@@ -411,7 +510,7 @@ class TestSnapshotProgressStats:
 
         prefetched_snapshot = Snapshot.objects.prefetch_related("archiveresult_set").get(pk=snapshot.pk)
         admin = SnapshotAdmin(Snapshot, archivebox_admin)
-        request = RequestFactory().get("/", HTTP_HOST="archivebox.localhost:8000")
+        request = RequestFactory().get("/", HTTP_HOST="archivebox.localhost:5797")
         request.resolver_match = resolve("/")
         request.archivebox_config = get_config()
         admin.request = request
@@ -777,7 +876,7 @@ class TestSnapshotProgressStats:
 
         assert embed_path in html
         assert "?preview=1" not in html
-        assert html != "http://snap-ffa4215f6d64.archivebox.localhost:8000"
+        assert html != "http://snap-ffa4215f6d64.archivebox.localhost:5797"
 
     def test_plugin_full_returns_empty_for_none_result(self):
         from archivebox.core.templatetags import core_tags
@@ -1279,13 +1378,13 @@ class TestAdminSnapshotListView:
 
         admin = SnapshotAdmin(snapshot.__class__, archivebox_admin)
 
-        request = RequestFactory().get("/", HTTP_HOST="admin.archivebox.localhost:8000")
+        request = RequestFactory().get("/", HTTP_HOST="admin.archivebox.localhost:5797")
         request.archivebox_config = get_config(overrides={"SERVER_SECURITY_MODE": "safe-subdomains-fullreplay"})
         admin.request = request
-        assert admin.get_snapshot_view_url(snapshot) == f"http://snap-{str(snapshot.pk).replace('-', '')[-12:]}.archivebox.localhost:8000"
+        assert admin.get_snapshot_view_url(snapshot) == f"http://snap-{str(snapshot.pk).replace('-', '')[-12:]}.archivebox.localhost:5797"
 
         request.archivebox_config = get_config(overrides={"SERVER_SECURITY_MODE": "safe-onedomain-nojsreplay"})
-        assert admin.get_snapshot_view_url(snapshot) == f"http://archivebox.localhost:8000/snapshot/{snapshot.pk}"
+        assert admin.get_snapshot_view_url(snapshot) == f"http://archivebox.localhost:5797/snapshot/{snapshot.pk}"
 
     def test_find_snapshots_for_url_matches_fragment_suffixed_variants(self, crawl, db):
         from archivebox.core.models import Snapshot

archivebox/tests/test_ui_config_views.py

@@ -132,7 +132,7 @@ def test_plugin_detail_view_renders_shipped_plugin_config_in_dedicated_sections(
         "source": "builtin",
     }
     assert "abx_plugins/plugins/wget" in summary_section["description"]
-    assert "https://archivebox.github.io/abx-plugins/#wget" in summary_section["description"]
+    assert "https://plugins.archivebox.io/#wget" in summary_section["description"]
 
     assert hooks_section["name"] == "Hooks"
     assert hooks_section["fields"] == {}

archivebox/tests/test_ui_live_progress.py

@@ -3,9 +3,11 @@
 import subprocess
 import uuid
 from datetime import datetime, timezone as dt_timezone
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 from importlib.resources import files
 from pathlib import Path
-from threading import Thread
+from threading import Event, Thread
+from types import SimpleNamespace
 
 import pytest
 from django.test import override_settings
@@ -19,6 +21,41 @@ from archivebox.tests.test_archive_result_service import _run_shipped_snapshot_h
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
+@pytest.fixture
+def painted_http_server():
+    """Serve a complete static page so CDP stops emitting changed frames."""
+
+    request_started = Event()
+
+    class PaintedBlockingHandler(BaseHTTPRequestHandler):
+        def do_GET(self):
+            body = b"""<!doctype html>
+                <html><head><title>Still frame</title></head>
+                <body style="background:#246;color:white"><h1>Visible page</h1></body></html>
+            """
+            self.send_response(200)
+            self.send_header("Content-Type", "text/html; charset=utf-8")
+            self.send_header("Content-Length", str(len(body)))
+            self.end_headers()
+            self.wfile.write(body)
+            self.wfile.flush()
+            request_started.set()
+
+        def log_message(self, format, *args):
+            pass
+
+    server = ThreadingHTTPServer(("127.0.0.1", 0), PaintedBlockingHandler)
+    thread = Thread(target=server.serve_forever, name="archivebox-test-painted-http-barrier")
+    thread.start()
+    yield SimpleNamespace(
+        url=f"http://127.0.0.1:{server.server_port}/",
+        request_started=request_started,
+    )
+    server.shutdown()
+    server.server_close()
+    thread.join()
+
+
 @pytest.fixture
 def real_unscoped_hook_process(tmp_path):
     from archivebox.tests.conftest import run_test_hook
@@ -329,6 +366,77 @@ class TestLiveProgressView:
         result.refresh_from_db()
         assert result.status in (ArchiveResult.StatusChoices.SUCCEEDED, ArchiveResult.StatusChoices.NORESULTS)
 
+    def test_live_progress_keeps_still_browser_frame_while_screencast_runs(
+        self,
+        client,
+        admin_user,
+        crawl,
+        snapshot,
+        painted_http_server,
+        cached_abxpkg_lib_dir,
+    ):
+        import time
+
+        from archivebox.core.models import Snapshot
+        from archivebox.crawls.models import Crawl
+        from archivebox.services.runner import run_due_snapshot
+
+        now = timezone.now()
+        Crawl.objects.filter(pk=crawl.pk).update(
+            status=Crawl.StatusChoices.STARTED,
+            retry_at=now,
+            urls=painted_http_server.url,
+            config={"PLUGINS": "chrome,chrome_screencast", "CHROME_ISOLATION": "snapshot"},
+        )
+        Snapshot.objects.filter(pk=snapshot.pk).update(
+            status=Snapshot.StatusChoices.QUEUED,
+            retry_at=now,
+            url=painted_http_server.url,
+            config={
+                "PLUGINS": "chrome,chrome_screencast",
+                "CHROME_HEADLESS": True,
+                "CHROME_SANDBOX": False,
+                "CHROME_ISOLATION": "snapshot",
+                "CHROME_TIMEOUT": 90,
+                "CHROME_PAGELOAD_TIMEOUT": 90,
+                "CHROME_DELAY_AFTER_LOAD": 30,
+                "TIMEOUT": 90,
+            },
+        )
+        snapshot.refresh_from_db()
+        frame = Path(snapshot.crawl.output_dir) / "chrome_screencast" / "latest.jpg"
+        errors = []
+
+        def run_snapshot():
+            try:
+                assert run_due_snapshot(snapshot, lock_seconds=120) is True
+            except BaseException as err:
+                errors.append(err)
+
+        runner = Thread(target=run_snapshot, name="archivebox-test-screencast-runner")
+        runner.start()
+        try:
+            assert painted_http_server.request_started.wait(timeout=60)
+            deadline = time.monotonic() + 90
+            while time.monotonic() < deadline:
+                assert errors == []
+                if frame.is_file() and time.time() - frame.stat().st_mtime > 16:
+                    break
+                time.sleep(0.1)
+            assert frame.is_file()
+            assert frame.read_bytes().startswith(b"\xff\xd8\xff")
+            assert time.time() - frame.stat().st_mtime > 16
+            client.force_login(admin_user)
+            response = client.get("/progress.json", HTTP_HOST=ADMIN_TEST_HOST)
+            assert response.status_code == 200, response.content
+            active_crawl = next(item for item in response.json()["active_crawls"] if item["id"] == str(crawl.pk))
+            assert active_crawl["screencast_url"].startswith(f"/api/v1/crawls/crawl/{crawl.pk}/files/chrome_screencast/latest.jpg")
+        finally:
+            runner.join(timeout=120)
+        assert not runner.is_alive()
+        assert errors == []
+        assert not frame.exists()
+
     def test_live_progress_hides_finished_cancelled_crawl(self, client, admin_user, crawl, snapshot):
         from archivebox.core.models import ArchiveResult, Snapshot
         from archivebox.crawls.models import Crawl
@@ -582,6 +690,60 @@ class TestLiveProgressView:
         assert active_snapshot["all_plugins"][0]["status"] == "started"
         assert active_snapshot["worker_pid"] == pid
 
+    def test_live_progress_filters_process_query_to_displayed_crawls_without_hiding_older_live_hooks(
+        self,
+        client,
+        admin_user,
+        snapshot,
+        real_snapshot_hook_projection,
+        real_unscoped_hook_process,
+    ):
+        import os
+        from datetime import timedelta
+
+        import archivebox.machine.models as machine_models
+        from archivebox.machine.models import Process
+        from django.db import connection
+        from django.test.utils import CaptureQueriesContext
+
+        process, result = real_snapshot_hook_projection
+        machine_models._CURRENT_MACHINE = process.machine
+        result.delete()
+        old_started_at = timezone.now() - timedelta(hours=1)
+        process.status = Process.StatusChoices.RUNNING
+        process.pid = os.getpid()
+        process.started_at = old_started_at
+        process.ended_at = None
+        process.save(update_fields=["status", "pid", "started_at", "ended_at", "modified_at"])
+        Process.objects.filter(pk=process.pk).update(modified_at=old_started_at)
+
+        unrelated = real_unscoped_hook_process
+        unrelated.status = Process.StatusChoices.RUNNING
+        unrelated.pid = os.getpid()
+        unrelated.ended_at = None
+        unrelated.save(update_fields=["status", "pid", "ended_at", "modified_at"])
+        client.force_login(admin_user)
+        with CaptureQueriesContext(connection) as queries:
+            response = client.get(reverse("live_progress"), HTTP_HOST=ADMIN_TEST_HOST)
+
+        assert response.status_code == 200
+        payload = response.json()
+        active_crawl = next(crawl for crawl in payload["active_crawls"] if crawl["id"] == str(snapshot.crawl_id))
+        active_snapshot = next(item for item in active_crawl["active_snapshots"] if item["id"] == str(snapshot.id))
+        assert active_snapshot["worker_pid"] == os.getpid()
+        assert payload["total_workers"] == 1
+
+        running_process_sql = next(
+            query["sql"]
+            for query in queries.captured_queries
+            if 'FROM "machine_process"' in query["sql"]
+            and '"machine_process"."process_type" IN' in query["sql"]
+            and '"machine_process"."status" = \'running\'' in query["sql"]
+        )
+        assert '"machine_process"."pwd" REGEXP' in running_process_sql
+        assert str(snapshot.id) in running_process_sql
+        assert str(unrelated.pk) not in running_process_sql
+
     def test_live_progress_merges_process_rows_with_archiveresults_when_present(
         self,
         client,

archivebox/tests/test_ui_savepagenow.py

@@ -8,7 +8,7 @@ from pathlib import Path
 from archivebox.tests.conftest import cli_env, create_test_url
 
 
-ADMIN_HOST = "admin.archivebox.localhost:8000"
+ADMIN_HOST = "admin.archivebox.localhost:5797"
 
 
 def _run_savepagenow_script(
@@ -93,7 +93,7 @@ def _run_savepagenow_not_found_script(initialized_archive: Path, request_url: st
         client = Client()
         target_url = {request_url!r}
 
-        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:8000')
+        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:5797')
         assert resp.status_code == 302, resp.status_code
         assert resp['Location'] == f'http://{ADMIN_HOST}/web/' + target_url
         assert Snapshot.objects.count() == 0
@@ -226,7 +226,7 @@ def _run_savepagenow_via_web_host_redirect_script(initialized_archive: Path, req
 
         target_url = {request_url!r}
 
-        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:8000')
+        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:5797')
         assert resp.status_code == 302, resp.status_code
         assert resp['Location'] == f'http://{ADMIN_HOST}/web/' + target_url
 
@@ -274,7 +274,7 @@ def _run_savepagenow_existing_snapshot_script(initialized_archive: Path, request
         snapshot = Snapshot.objects.create(url=stored_url, crawl=crawl)
 
         client = Client()
-        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:8000')
+        resp = client.get('/web/' + target_url, HTTP_HOST='web.archivebox.localhost:5797')
         assert resp.status_code == 302, resp.status_code
         assert resp['Location'] == f"/{{snapshot.url_path}}"
         """,
@@ -310,7 +310,7 @@ def test_web_add_creates_and_reuses_snapshot_public(initialized_archive):
         url,
         login=False,
         public_add_view=True,
-        host="web.archivebox.localhost:8000",
+        host="web.archivebox.localhost:5797",
     )
     assert result.returncode == 0, f"SavePageNow shortcut (public add) test failed.\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}"
 

archivebox/tests/test_urls.py

@@ -8,12 +8,51 @@ from pathlib import Path
 from threading import Thread
 
 import pytest
+from types import SimpleNamespace
 
 from archivebox.tests.conftest import run_archivebox_cmd
 
 REPO_ROOT = Path(__file__).resolve().parents[2]
 
 
+def test_private_snapshot_image_response_is_not_publicly_cacheable(tmp_path):
+    from django.test import RequestFactory
+
+    from archivebox.misc.serve_static import serve_static_with_byterange_support
+
+    (tmp_path / "screenshot.png").write_bytes(b"\x89PNG\r\n\x1a\nprivate")
+    request = RequestFactory().get("/snapshot/private/screenshot.png")
+    request.archivebox_config = SimpleNamespace(PERMISSIONS="public")
+    request.archivebox_cache_policy = "private"
+
+    response = serve_static_with_byterange_support(request, "screenshot.png", document_root=tmp_path)
+
+    assert response.status_code == 200
+    assert response["Content-Type"] == "image/png"
+    assert response["Cache-Control"] == "private, max-age=604800, immutable"
+
+
+@pytest.mark.django_db
+def test_private_snapshot_replay_never_returns_public_cache_headers(snapshot):
+    from django.test import RequestFactory
+
+    from archivebox.core.views import _serve_snapshot_replay
+
+    snapshot.config = {**snapshot.config, "PERMISSIONS": "private"}
+    snapshot.save(update_fields=["config"])
+    snapshot.output_dir.mkdir(parents=True, exist_ok=True)
+    (snapshot.output_dir / "screenshot.png").write_bytes(b"\x89PNG\r\n\x1a\nprivate-replay")
+    request = RequestFactory().get(f"/snapshot/{snapshot.id}/screenshot.png")
+
+    response = _serve_snapshot_replay(request, snapshot, "screenshot.png")
+
+    assert response.status_code == 200
+    assert response["Cache-Control"].startswith("private,")
+    assert "public" not in response["Cache-Control"]
+    assert "Cookie" in response["Vary"]
+    assert b"".join(response.streaming_content).endswith(b"private-replay")
+
+
 @pytest.mark.django_db
 def test_archiveresult_relpath_uses_sibling_hook_that_owns_output(admin_user):
     from archivebox.core.models import ArchiveResult, Snapshot
@@ -495,16 +534,16 @@ class TestUrlRouting:
 
             bind_host, bind_port = split_host_port(bind_addr)
             assert bind_host == "127.0.0.1"
-            assert bind_port == "8000"
+            assert bind_port == "5797"
             host_only, port = split_host_port(base_host)
             assert host_only == "archivebox.localhost"
-            assert port == "8000"
-            assert web_host == "web.archivebox.localhost:8000"
-            assert admin_host == "admin.archivebox.localhost:8000"
-            assert api_host == "api.archivebox.localhost:8000"
+            assert port == "5797"
+            assert web_host == "web.archivebox.localhost:5797"
+            assert admin_host == "admin.archivebox.localhost:5797"
+            assert api_host == "api.archivebox.localhost:5797"
             assert snapshot_subdomain == f"snap-{snapshot_id[-12:].lower()}"
-            assert snapshot_host == f"{snapshot_subdomain}.archivebox.localhost:8000"
-            assert original_host == f"{domain}.archivebox.localhost:8000"
+            assert snapshot_host == f"{snapshot_subdomain}.archivebox.localhost:5797"
+            assert original_host == f"{domain}.archivebox.localhost:5797"
             assert get_listen_subdomain(web_host) == "web"
             assert get_listen_subdomain(admin_host) == "admin"
             assert get_listen_subdomain(api_host) == "api"
@@ -562,7 +601,7 @@ class TestUrlRouting:
             )
         finally:
             reset_result = run_archivebox_cmd(
-                ["config", "--set", "BASE_URL=http://archivebox.localhost:8000"],
+                ["config", "--set", "BASE_URL=http://archivebox.localhost:5797"],
                 cwd=self.data_dir,
             )
             assert reset_result.returncode == 0, reset_result.stderr
@@ -622,6 +661,360 @@ class TestUrlRouting:
             """,
         )
 
+    def test_private_snapshot_replay_cookie_authorizes_only_its_progress(self) -> None:
+        self._run(
+            """
+            from urllib.parse import urlsplit
+
+            ensure_admin_user()
+            snapshot = get_snapshot()
+            original_config = snapshot.config
+            Snapshot.objects.filter(pk=snapshot.pk).update(config={**snapshot.config, "PERMISSIONS": "private"})
+            other_snapshot = Snapshot.objects.create(
+                url="https://other-private-progress.example/",
+                crawl=snapshot.crawl,
+                config={"PERMISSIONS": "private"},
+            )
+            snapshot_host = get_snapshot_host(str(snapshot.id))
+            admin_host = get_admin_host()
+
+            try:
+                admin_client = Client()
+                assert admin_client.login(username="testadmin", password="testpassword")
+                grant_response = admin_client.get(
+                    f"/admin/core/snapshot/replay-auth/?snapshot={snapshot.id}&next=/index.html",
+                    HTTP_HOST=admin_host,
+                )
+                assert grant_response.status_code in (301, 302)
+                grant_url = urlsplit(grant_response["Location"])
+                assert grant_url.netloc == snapshot_host
+
+                replay_client = Client()
+                auth_response = replay_client.get(
+                    f"{grant_url.path}?{grant_url.query}",
+                    HTTP_HOST=snapshot_host,
+                )
+                assert auth_response.status_code in (301, 302)
+
+                snapshot_id = str(snapshot.id).replace("-", "")
+                progress_response = replay_client.get(
+                    f"/progress.json?snapshot_id={snapshot_id}",
+                    HTTP_HOST=snapshot_host,
+                )
+                assert progress_response.status_code == 200, response_body(progress_response)[:500]
+                assert progress_response.json()["scope"]["snapshot_id"] == snapshot_id
+
+                other_snapshot_id = str(other_snapshot.id).replace("-", "")
+                rejected = replay_client.get(
+                    f"/progress.json?snapshot_id={other_snapshot_id}",
+                    HTTP_HOST=snapshot_host,
+                )
+                assert rejected.status_code == 403
+            finally:
+                other_snapshot.delete()
+                Snapshot.objects.filter(pk=snapshot.pk).update(config=original_config)
+
+            print("OK")
+            """,
+            mode="safe-subdomains-fullreplay",
+        )
+
+    def test_snapshot_payload_cache_policy_tracks_effective_permissions_and_revocation(self) -> None:
+        self._run(
+            """
+            import hashlib
+            import json
+            import shutil
+            from urllib.parse import urlsplit
+
+            from archivebox.crawls.models import Crawl
+
+            admin = ensure_admin_user()
+            snapshot = get_snapshot()
+            original_config = snapshot.config
+            created_snapshots = []
+            created_crawls = []
+            raw_name = "cache-policy.png"
+            raw_bytes = b"\\x89PNG\\r\\n\\x1a\\ncache-policy-payload"
+            raw_path = Path(snapshot.output_dir) / raw_name
+            raw_path.write_bytes(raw_bytes)
+            metadata_path = Path(snapshot.output_dir) / "index.jsonl"
+            if not metadata_path.exists():
+                metadata_path.write_text('{"type":"Snapshot"}\\n', encoding="utf-8")
+            hashes_dir = Path(snapshot.output_dir) / "hashes"
+            hashes_dir.mkdir(parents=True, exist_ok=True)
+            (hashes_dir / "hashes.json").write_text(
+                json.dumps(
+                    {
+                        "files": [
+                            {"path": raw_name, "hash": hashlib.sha256(raw_bytes).hexdigest()},
+                            {
+                                "path": "index.jsonl",
+                                "hash": hashlib.sha256(metadata_path.read_bytes()).hexdigest(),
+                            },
+                        ],
+                    },
+                ),
+                encoding="utf-8",
+            )
+            snapshot_host = get_snapshot_host(str(snapshot.id))
+
+            def cache_tokens(response):
+                return {token.strip().lower() for token in response.headers.get("Cache-Control", "").split(",") if token.strip()}
+
+            def assert_cache(response, *expected):
+                assert cache_tokens(response) == set(expected), (response.status_code, response.headers)
+
+            def assert_auth_vary(response):
+                vary = {token.strip().lower() for token in response.headers.get("Vary", "").split(",") if token.strip()}
+                assert {"cookie", "authorization"} <= vary, response.headers
+
+            try:
+                snapshot.config = {**snapshot.config, "PERMISSIONS": "public"}
+                snapshot.save(update_fields=["config"])
+                snapshot.refresh_from_db()
+                assert snapshot.permissions == "public"
+
+                public_client = Client()
+                public_index = public_client.get("/index.html", HTTP_HOST=snapshot_host)
+                assert public_index.status_code == 200
+                assert_cache(public_index, "public", "max-age=60", "stale-while-revalidate=300")
+
+                public_files = public_client.get(f"/{snapshot.url_path}/index.html?files=1", HTTP_HOST=get_web_host())
+                assert public_files.status_code == 200
+                assert raw_name.encode() in response_body(public_files)
+                assert_cache(public_files, "public", "max-age=60", "stale-while-revalidate=300")
+
+                public_metadata = public_client.get("/index.jsonl", HTTP_HOST=snapshot_host)
+                assert public_metadata.status_code == 200
+                assert response_body(public_metadata) == metadata_path.read_bytes()
+                assert_cache(public_metadata, "public", "max-age=31536000", "immutable")
+
+                public_raw = public_client.get(f"/{raw_name}", HTTP_HOST=snapshot_host)
+                assert public_raw.status_code == 200
+                assert response_body(public_raw) == raw_bytes
+                assert public_raw.headers.get("ETag")
+                assert_cache(public_raw, "public", "max-age=604800", "immutable")
+
+                public_preview = public_client.get(f"/{raw_name}?preview=1", HTTP_HOST=snapshot_host)
+                assert public_preview.status_code == 200
+                assert public_preview["Content-Type"].startswith("text/html")
+                assert_cache(public_preview, "public", "max-age=60", "stale-while-revalidate=300")
+
+                public_range = public_client.get(f"/{raw_name}", HTTP_HOST=snapshot_host, HTTP_RANGE="bytes=0-7")
+                assert public_range.status_code == 206
+                assert response_body(public_range) == raw_bytes[:8]
+                assert_cache(public_range, "public", "max-age=604800", "immutable")
+
+                public_not_modified = public_client.get(
+                    f"/{raw_name}",
+                    HTTP_HOST=snapshot_host,
+                    HTTP_IF_NONE_MATCH=public_raw["ETag"],
+                )
+                assert public_not_modified.status_code == 304
+                assert_cache(public_not_modified, "public", "max-age=31536000", "immutable")
+
+                public_progress = public_client.get(
+                    f"/progress.json?snapshot_id={str(snapshot.id).replace('-', '')}",
+                    HTTP_HOST=snapshot_host,
+                )
+                assert public_progress.status_code == 200
+                assert_cache(public_progress, "private", "no-store")
+                public_missing = public_client.get("/missing-cache-policy-output", HTTP_HOST=snapshot_host)
+                assert public_missing.status_code == 404
+                assert_cache(public_missing, "private", "no-store")
+                public_redirect = public_client.get(f"/{snapshot.url_path}/index.html", HTTP_HOST=get_admin_host())
+                assert public_redirect.status_code in (301, 302)
+                assert public_redirect["Location"].startswith(f"http://{snapshot_host}")
+                assert_cache(public_redirect, "private", "no-store")
+                legacy_redirect = public_client.get(
+                    f"/{snapshot.legacy_archive_path}/index.html",
+                    HTTP_HOST=get_web_host(),
+                )
+                assert legacy_redirect.status_code in (301, 302)
+                assert snapshot.url_path in legacy_redirect["Location"]
+                assert_cache(legacy_redirect, "private", "no-store")
+
+                static_asset = public_client.get("/static/jquery.min.js", HTTP_HOST=snapshot_host)
+                assert static_asset.status_code == 200
+                assert static_asset.headers.get("ETag")
+                assert_cache(static_asset, "public", "max-age=31536000", "immutable")
+                static_not_modified = public_client.get(
+                    "/static/jquery.min.js",
+                    HTTP_HOST=snapshot_host,
+                    HTTP_IF_NONE_MATCH=static_asset["ETag"],
+                )
+                assert static_not_modified.status_code == 304
+                assert_cache(static_not_modified, "public", "max-age=31536000", "immutable")
+
+                # Tightening permissions must still run authorization before honoring
+                # a stale validator presented in a new request.
+                snapshot.config = {**snapshot.config, "PERMISSIONS": "private"}
+                snapshot.save(update_fields=["config"])
+                snapshot.refresh_from_db()
+                assert snapshot.permissions == "private"
+                revoked = Client().get(
+                    f"/{raw_name}",
+                    HTTP_HOST=snapshot_host,
+                    HTTP_IF_NONE_MATCH=public_raw["ETag"],
+                )
+                assert revoked.status_code in (301, 302)
+                assert "/admin/core/snapshot/replay-auth/" in revoked["Location"]
+                assert_cache(revoked, "private", "no-store")
+                assert_auth_vary(revoked)
+                private_files = Client().get(f"/{snapshot.url_path}/index.html?files=1", HTTP_HOST=get_web_host())
+                assert private_files.status_code in (301, 302)
+                assert "/admin/core/snapshot/replay-auth/" in private_files["Location"]
+                assert_cache(private_files, "private", "no-store")
+                assert_auth_vary(private_files)
+
+                admin_client = Client()
+                assert admin_client.login(username=admin.username, password="testpassword")
+                grant = admin_client.get(
+                    f"/admin/core/snapshot/replay-auth/?snapshot={snapshot.id}&next=/{raw_name}",
+                    HTTP_HOST=get_admin_host(),
+                )
+                assert grant.status_code in (301, 302)
+                grant_url = urlsplit(grant["Location"])
+                replay_client = Client()
+                replay_auth = replay_client.get(f"{grant_url.path}?{grant_url.query}", HTTP_HOST=snapshot_host)
+                assert replay_auth.status_code in (301, 302)
+                assert_cache(replay_auth, "private", "no-store")
+                private_raw = replay_client.get(f"/{raw_name}", HTTP_HOST=snapshot_host)
+                assert private_raw.status_code == 200
+                assert response_body(private_raw) == raw_bytes
+                assert_cache(private_raw, "private", "max-age=604800", "immutable")
+                assert_auth_vary(private_raw)
+                private_index = replay_client.get("/index.html", HTTP_HOST=snapshot_host)
+                assert private_index.status_code == 200
+                assert_cache(private_index, "private", "max-age=60", "stale-while-revalidate=300")
+                assert_auth_vary(private_index)
+                private_zip = replay_client.get("/?files=1&download=zip", HTTP_HOST=snapshot_host)
+                assert private_zip.status_code == 200
+                assert private_zip["Content-Type"] == "application/zip"
+                assert response_body(private_zip).startswith(b"PK")
+                assert_cache(private_zip, "private", "max-age=60", "stale-while-revalidate=300")
+                private_metadata = replay_client.get("/index.jsonl", HTTP_HOST=snapshot_host)
+                assert private_metadata.status_code == 200
+                assert response_body(private_metadata) == metadata_path.read_bytes()
+                assert_cache(private_metadata, "private", "max-age=31536000", "immutable")
+                private_preview = replay_client.get(f"/{raw_name}?preview=1", HTTP_HOST=snapshot_host)
+                assert private_preview.status_code == 200
+                assert private_preview["Content-Type"].startswith("text/html")
+                assert_cache(private_preview, "private", "max-age=60", "stale-while-revalidate=300")
+                private_range = replay_client.get(f"/{raw_name}", HTTP_HOST=snapshot_host, HTTP_RANGE="bytes=0-7")
+                assert private_range.status_code == 206
+                assert response_body(private_range) == raw_bytes[:8]
+                assert_cache(private_range, "private", "max-age=604800", "immutable")
+                private_not_modified = replay_client.get(
+                    f"/{raw_name}",
+                    HTTP_HOST=snapshot_host,
+                    HTTP_IF_NONE_MATCH=private_raw["ETag"],
+                )
+                assert private_not_modified.status_code == 304
+                assert_cache(private_not_modified, "private", "max-age=31536000", "immutable")
+                assert_auth_vary(private_not_modified)
+                private_missing = replay_client.get("/missing-cache-policy-output", HTTP_HOST=snapshot_host)
+                assert private_missing.status_code == 404
+                assert_cache(private_missing, "private", "no-store")
+                private_redirect = admin_client.get(f"/{snapshot.url_path}/index.html", HTTP_HOST=get_admin_host())
+                assert private_redirect.status_code in (301, 302)
+                assert private_redirect["Location"].startswith(f"http://{snapshot_host}")
+                assert_cache(private_redirect, "private", "no-store")
+
+                snapshot.config = {**snapshot.config, "PERMISSIONS": "unlisted"}
+                snapshot.save(update_fields=["config"])
+                snapshot.refresh_from_db()
+                assert snapshot.permissions == "unlisted"
+                unlisted_raw = Client().get(f"/{raw_name}", HTTP_HOST=snapshot_host)
+                assert unlisted_raw.status_code == 200
+                assert response_body(unlisted_raw) == raw_bytes
+                assert_cache(unlisted_raw, "private", "max-age=604800", "immutable")
+                unlisted_index = Client().get("/index.html", HTTP_HOST=snapshot_host)
+                assert unlisted_index.status_code == 200
+                assert_cache(unlisted_index, "private", "max-age=60", "stale-while-revalidate=300")
+                unlisted_files = Client().get(f"/{snapshot.url_path}/index.html?files=1", HTTP_HOST=get_web_host())
+                assert unlisted_files.status_code == 200
+                assert raw_name.encode() in response_body(unlisted_files)
+                assert_cache(unlisted_files, "private", "max-age=60", "stale-while-revalidate=300")
+                unlisted_zip = Client().get(
+                    f"/{snapshot.url_path}/index.html?files=1&download=zip",
+                    HTTP_HOST=get_web_host(),
+                )
+                assert unlisted_zip.status_code == 200
+                assert unlisted_zip["Content-Type"] == "application/zip"
+                assert response_body(unlisted_zip).startswith(b"PK")
+                assert_cache(unlisted_zip, "private", "max-age=60", "stale-while-revalidate=300")
+                unlisted_metadata = Client().get("/index.jsonl", HTTP_HOST=snapshot_host)
+                assert unlisted_metadata.status_code == 200
+                assert response_body(unlisted_metadata) == metadata_path.read_bytes()
+                assert_cache(unlisted_metadata, "private", "max-age=31536000", "immutable")
+                unlisted_preview = Client().get(f"/{raw_name}?preview=1", HTTP_HOST=snapshot_host)
+                assert unlisted_preview.status_code == 200
+                assert unlisted_preview["Content-Type"].startswith("text/html")
+                assert_cache(unlisted_preview, "private", "max-age=60", "stale-while-revalidate=300")
+                unlisted_range = Client().get(f"/{raw_name}", HTTP_HOST=snapshot_host, HTTP_RANGE="bytes=0-7")
+                assert unlisted_range.status_code == 206
+                assert response_body(unlisted_range) == raw_bytes[:8]
+                assert_cache(unlisted_range, "private", "max-age=604800", "immutable")
+                unlisted_not_modified = Client().get(
+                    f"/{raw_name}",
+                    HTTP_HOST=snapshot_host,
+                    HTTP_IF_NONE_MATCH=unlisted_raw["ETag"],
+                )
+                assert unlisted_not_modified.status_code == 304
+                assert_cache(unlisted_not_modified, "private", "max-age=31536000", "immutable")
+                unlisted_missing = Client().get("/missing-cache-policy-output", HTTP_HOST=snapshot_host)
+                assert unlisted_missing.status_code == 404
+                assert_cache(unlisted_missing, "private", "no-store")
+                unlisted_redirect = Client().get(f"/{snapshot.url_path}/index.html", HTTP_HOST=get_admin_host())
+                assert unlisted_redirect.status_code in (301, 302)
+                assert unlisted_redirect["Location"].startswith(f"http://{snapshot_host}")
+                assert_cache(unlisted_redirect, "private", "no-store")
+
+                # Snapshots materialize their crawl's permission when no explicit
+                # snapshot override is supplied, and each effective permission drives caching.
+                for inherited_permission, expected_cache in (
+                    ("public", ("public", "max-age=604800", "immutable")),
+                    ("unlisted", ("private", "max-age=604800", "immutable")),
+                    ("private", ("private", "no-store")),
+                ):
+                    crawl = Crawl.objects.create(
+                        urls=f"https://inherited-{inherited_permission}.example/",
+                        created_by=admin,
+                        config={"PERMISSIONS": inherited_permission},
+                    )
+                    created_crawls.append(crawl)
+                    inherited = Snapshot.objects.create(
+                        url=f"https://inherited-{inherited_permission}.example/",
+                        crawl=crawl,
+                        config={},
+                    )
+                    created_snapshots.append(inherited)
+                    inherited.refresh_from_db()
+                    assert inherited.permissions == inherited_permission
+                    inherited.output_dir.mkdir(parents=True, exist_ok=True)
+                    (inherited.output_dir / raw_name).write_bytes(raw_bytes)
+                    inherited_host = get_snapshot_host(str(inherited.id))
+                    response = Client().get(f"/{raw_name}", HTTP_HOST=inherited_host)
+                    if inherited_permission == "private":
+                        assert response.status_code in (301, 302)
+                    else:
+                        assert response.status_code == 200
+                        assert response_body(response) == raw_bytes
+                    assert_cache(response, *expected_cache)
+
+                print("OK")
+            finally:
+                Snapshot.objects.filter(pk=snapshot.pk).update(config=original_config)
+                for snap in created_snapshots:
+                    shutil.rmtree(snap.output_dir)
+                for crawl in created_crawls:
+                    crawl.delete()
+            """,
+            mode="safe-subdomains-fullreplay",
+        )
+
     def test_admin_login_next_allows_archivebox_hosts_only(self) -> None:
         self._run(
             """
@@ -634,7 +1027,7 @@ class TestUrlRouting:
 
             allowed_next_urls = [
                 "/admin/core/snapshot/",
-                f"http://archivebox.localhost:8000/public/",
+                f"http://archivebox.localhost:5797/public/",
                 f"http://{web_host}/public/",
                 f"http://{admin_host}/admin/core/snapshot/",
                 f"http://{api_host}/api/v1/docs",
@@ -1205,7 +1598,7 @@ class TestUrlRouting:
                 mode="safe-subdomains-fullreplay",
             )
         finally:
-            self._set_config("BIND_ADDR=127.0.0.1:8000", "BASE_URL=http://archivebox.localhost:8000")
+            self._set_config("BIND_ADDR=127.0.0.1:5797", "BASE_URL=http://archivebox.localhost:5797")
 
     def test_subdomain_replay_assets_route_without_base_url(self) -> None:
         lib_dir = self.data_dir / "test-lib"
@@ -1233,7 +1626,7 @@ class TestUrlRouting:
                 env_overrides={"ABXPKG_LIB_DIR": str(lib_dir)},
             )
         finally:
-            self._set_config("BIND_ADDR=127.0.0.1:8000", "BASE_URL=http://archivebox.localhost:8000")
+            self._set_config("BIND_ADDR=127.0.0.1:5797", "BASE_URL=http://archivebox.localhost:5797")
 
     def test_subdomain_replay_assets_use_derived_chromewebstore_extensions_dir(self) -> None:
         lib_dir = self.data_dir / "test-lib"
@@ -1262,7 +1655,7 @@ class TestUrlRouting:
                 env_overrides={"ABXPKG_LIB_DIR": str(lib_dir)},
             )
         finally:
-            self._set_config("BIND_ADDR=127.0.0.1:8000", "BASE_URL=http://archivebox.localhost:8000")
+            self._set_config("BIND_ADDR=127.0.0.1:5797", "BASE_URL=http://archivebox.localhost:5797")
 
     def test_onedomain_base_url_overrides_are_preserved_for_external_links(self) -> None:
         try:
@@ -1289,7 +1682,7 @@ class TestUrlRouting:
                 mode="safe-onedomain-nojsreplay",
             )
         finally:
-            self._set_config("BASE_URL=http://archivebox.localhost:8000")
+            self._set_config("BASE_URL=http://archivebox.localhost:5797")
 
     def test_subdomain_snapshot_urls_inherit_https_archive_base_url(self) -> None:
         try:
@@ -1310,7 +1703,7 @@ class TestUrlRouting:
                 mode="safe-subdomains-fullreplay",
             )
         finally:
-            self._set_config("BASE_URL=http://archivebox.localhost:8000")
+            self._set_config("BASE_URL=http://archivebox.localhost:5797")
 
     def test_template_and_admin_links(self) -> None:
         self._run(
@@ -1343,7 +1736,7 @@ class TestUrlRouting:
 
             resp = client.get("/public/", HTTP_HOST=web_host)
             assert resp.status_code in (301, 302)
-            assert resp["Location"] == "http://admin.archivebox.localhost:8000/admin/core/snapshot/"
+            assert resp["Location"] == "http://admin.archivebox.localhost:5797/admin/core/snapshot/"
 
             resp = client.get(f"/{snapshot.url_path}/index.html", HTTP_HOST=web_host)
             assert resp.status_code == 200
@@ -1518,12 +1911,12 @@ class TestUrlRouting:
         # Exercise the same real captured files with each request-time posture,
         # including both auto branches and an HTTPS canonical reverse-proxy URL.
         for mode, base in (
-            ("auto", "http://archivebox.localhost:8000"),
-            ("auto", "http://archivebox.example:8000"),
+            ("auto", "http://archivebox.localhost:5797"),
+            ("auto", "http://archivebox.example:5797"),
             ("safe-subdomains-fullreplay", "https://archivebox.example"),
-            ("safe-onedomain-nojsreplay", "http://archivebox.example:8000"),
-            ("unsafe-onedomain-noadmin", "http://archivebox.example:8000"),
-            ("danger-onedomain-fullreplay", "http://archivebox.example:8000"),
+            ("safe-onedomain-nojsreplay", "http://archivebox.example:5797"),
+            ("unsafe-onedomain-noadmin", "http://archivebox.example:5797"),
+            ("danger-onedomain-fullreplay", "http://archivebox.example:5797"),
         ):
             self._run(
                 """

archivebox/workers/management/commands/runner_watch.py

@@ -8,7 +8,7 @@ class Command(BaseCommand):
         parser.add_argument(
             "--bind-url",
             default="",
-            help="Runserver bind URL to watch, e.g. http://127.0.0.1:8000",
+            help="Runserver bind URL to watch, e.g. http://127.0.0.1:5797",
         )
         parser.add_argument(
             "--interval",

archivebox/workers/supervisord_util.py

@@ -330,13 +330,18 @@ def SUPERVISORD_PARENT_WATCHDOG_WORKER(
     }
 
 
-SERVER_WORKER = lambda host, port: {
+SERVER_WORKER = lambda host, port, extra_port=None: {
     "name": "worker_daphne",
     "command": _shell_join(
         [
             str(resolve_env_binary("daphne")),
             f"--bind={host}",
             f"--port={port}",
+            *(
+                ["--endpoint", "tcp:port={}:interface={}".format(extra_port, host.strip("[]").replace(":", r"\:"))]
+                if extra_port is not None
+                else []
+            ),
             "archivebox.core.asgi:application",
         ],
     ),
@@ -1115,7 +1120,7 @@ def format_runtime_components(components: list[str] | tuple[str, ...]) -> str:
 def worker_runtime_component(worker_name: str) -> str | None:
     if worker_name in {"worker_runner", "worker_runner_watch"} or worker_name.startswith("worker_runner_"):
         return "orchestrator"
-    if worker_name in {"worker_daphne", "worker_runserver"}:
+    if worker_name in {"worker_daphne", "worker_runserver", "worker_runserver_compatibility"}:
         return "server"
     if worker_name == "worker_sonic":
         return "sonic"
@@ -1140,6 +1145,9 @@ def active_supervisord_runtime_components(*, supervisor=None) -> list[str]:
 
 def build_server_worker_plan(*, config, host: str, port: str, debug: bool, reload: bool, nothreading: bool, supervisor=None):
     bind_url = f"http://{host}:{port}"
+    # Docker does not expose its host port mappings to the container. Keep both
+    # image defaults reachable so existing mappings survive an image upgrade.
+    extra_port = {"5797": "8000", "8000": "5797"}.get(str(port)) if config.IN_DOCKER else None
 
     if debug:
         server_worker = RUNSERVER_WORKER(host=host, port=port, reload=reload, nothreading=nothreading)
@@ -1150,7 +1158,7 @@ def build_server_worker_plan(*, config, host: str, port: str, debug: bool, reloa
         if reload:
             log_files.insert(1, "logs/worker_runner_watch.log")
     else:
-        server_worker = SERVER_WORKER(host=host, port=port)
+        server_worker = SERVER_WORKER(host=host, port=port, extra_port=extra_port)
         bg_workers = [(RUNNER_WORKER(), False)]
         log_files = ["logs/worker_daphne.log", "logs/worker_runner.log"]
 
@@ -1178,6 +1186,12 @@ def build_server_worker_plan(*, config, host: str, port: str, debug: bool, reloa
             log_files.append(str(sonic_worker["stdout_logfile"]))
 
     workers = [(server_worker, False), *bg_workers]
+    if debug and extra_port is not None:
+        compatibility_worker = RUNSERVER_WORKER(host=host, port=extra_port, reload=reload, nothreading=nothreading)
+        compatibility_worker["name"] = "worker_runserver_compatibility"
+        compatibility_worker["stdout_logfile"] = "logs/worker_runserver_compatibility.log"
+        workers.insert(1, (compatibility_worker, False))
+        log_files.append(compatibility_worker["stdout_logfile"])
     if os.environ.get("DISPLAY") and (config.IN_DOCKER or os.environ.get("ARCHIVEBOX_VNC_PERSONA")):
         vnc_persona = os.environ.get("ARCHIVEBOX_VNC_PERSONA") or config.DEFAULT_PERSONA
         workers.append(
@@ -1478,7 +1492,7 @@ def stop_stale_sonic_processes(
 
 def start_server_workers(
     host="0.0.0.0",
-    port="8000",
+    port="5797",
     daemonize=False,
     debug=False,
     reload=False,

bin/collect_ui_screenshots.sh

@@ -5,6 +5,11 @@ set -o errtrace
 set -o nounset
 set -o pipefail
 
+# Every real runtime owner (including foreground archiving) must inherit the
+# existing screenshot-only host-load setting when it restarts shared workers.
+# Disk, memory, routing and security warnings remain enabled.
+export UI_SCREENSHOT_HIDE_HIGH_LOAD_WARNING=1
+
 REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
 DATA_DIR="${UI_SCREENSHOT_DATA_DIR:-$REPO_DIR/data}"
 OUTPUT_DIR="${UI_SCREENSHOT_OUTPUT_DIR:-$REPO_DIR/docs/screenshots}"
@@ -205,8 +210,10 @@ PYPORT
 (
     cd "$DATA_DIR"
     uv run --no-cache --project "$REPO_DIR" archivebox config --set \
-        OPENCODE_ENABLED=True "OPENCODE_PORT=$OPENCODE_PORT" TLSNOTARY_ENABLED=True
+        OPENCODE_ENABLED=True "OPENCODE_PORT=$OPENCODE_PORT" TLSNOTARY_ENABLED=True \
+        HASHES_ENABLED=True OPENTIMESTAMPS_ENABLED=True
     uv run --no-cache --project "$REPO_DIR" archivebox install opencode --binproviders=env,pnpm
+    uv run --no-cache --project "$REPO_DIR" archivebox install opentimestamps
 )
 
 echo "[*] Starting ArchiveBox on port $PORT"
@@ -214,8 +221,7 @@ echo "[*] Starting ArchiveBox on port $PORT"
     cd "$DATA_DIR"
     # Use the real server command so the runner, worker, and log views describe
     # the same persistent runtime that an operator sees.
-    UI_SCREENSHOT_HIDE_HIGH_LOAD_WARNING=1 \
-        exec uv run --no-cache --project "$REPO_DIR" archivebox server "127.0.0.1:$PORT"
+    exec uv run --no-cache --project "$REPO_DIR" archivebox server "127.0.0.1:$PORT"
 ) >"$DATA_DIR/ui-screenshot-server.log" 2>&1 &
 SERVER_PID=$!
 
@@ -245,9 +251,9 @@ fi
 stop_background_runner
 
 ABXPKG_LIB_DIR="$(uv run --no-cache --project "$REPO_DIR" abx-dl config --get ABXPKG_LIB_DIR | sed 's/^[^=]*=//; s/^"//; s/"$//')"
-SCREENSHOT_CHROME_BINARY="$ABXPKG_LIB_DIR/env/bin/chromium"
+SCREENSHOT_CHROME_BINARY="${CHROME_BINARY:-$ABXPKG_LIB_DIR/env/bin/chromium}"
 if [[ ! -x "$SCREENSHOT_CHROME_BINARY" ]]; then
-    echo "[!] abx-dl projected Chromium was not found at $SCREENSHOT_CHROME_BINARY" >&2
+    echo "[!] Screenshot Chromium was not found at $SCREENSHOT_CHROME_BINARY (set CHROME_BINARY to override)" >&2
     exit 1
 fi
 
@@ -338,7 +344,7 @@ PY
                 )
                 (
                     cd "$SETUP_DATA_DIR"
-                    BASE_URL= UI_SCREENSHOT_HIDE_HIGH_LOAD_WARNING=1 \
+                    BASE_URL= \
                         exec uv run --no-cache --project "$REPO_DIR" archivebox server "127.0.0.1:$SETUP_PORT"
                 ) >"$SETUP_DATA_DIR/ui-screenshot-setup-wizard-server.log" 2>&1 &
                 SETUP_SERVER_PID=$!
@@ -631,17 +637,19 @@ PY
             SCREENSHOT_HEIGHT=1000 \
             node "$REPO_DIR/bin/take_screenshot.js" "$LIVE_SNAPSHOT_VIEW_URL" "$CAPTURE_ROOT/snapshot-output-discovery.png" >"$SNAPSHOT_DISCOVERY_REPORT"
         SNAPSHOT_OUTPUT_PLUGINS="$(UI_SCREENSHOT_DISCOVERY_REPORT="$SNAPSHOT_DISCOVERY_REPORT" uv run --no-cache --project "$REPO_DIR" python -c \
-            'import json, os; from urllib.parse import urlsplit; report=json.load(open(os.environ["UI_SCREENSHOT_DISCOVERY_REPORT"])); print("\n".join("{}\t{}".format(output["plugin"], "wait-frame-text:Verified · signature and archived response match" if output["plugin"] == "tlsnotary" else "wait-replay:Nick Sweeting" if urlsplit(output["previewUrl"]).path.endswith(".wacz") else "") for output in report["checks"]["snapshotOutputs"]))')"
+            'import json, os; from urllib.parse import urlsplit; report=json.load(open(os.environ["UI_SCREENSHOT_DISCOVERY_REPORT"])); print("\n".join("{}\t{}".format(output["plugin"], "wait-frame-text:Verified · signature and archived response match" if output["plugin"] == "tlsnotary" else "wait-frame-text:Calendar submission saved" if output["plugin"] == "opentimestamps" else "wait-replay:Nick Sweeting" if urlsplit(output["previewUrl"]).path.endswith(".wacz") else "") for output in report["checks"]["snapshotOutputs"]))')"
         if [[ -z "$SNAPSHOT_OUTPUT_PLUGINS" ]]; then
             echo "[!] The Sweeting.me snapshot detail page exposed no selectable outputs" >&2
             exit 1
         fi
-        if ! printf '%s\n' "$SNAPSHOT_OUTPUT_PLUGINS" | cut -f1 | grep -qx tlsnotary; then
-            echo "[!] Live capture did not expose the required TLSNotary verifier output" >&2
-            tail -200 "$CAPTURE_ROOT/sweeting-live-capture.log" >&2
-            cat "$SNAPSHOT_DISCOVERY_REPORT" >&2
-            exit 1
-        fi
+        for required_plugin in tlsnotary opentimestamps; do
+            if ! printf '%s\n' "$SNAPSHOT_OUTPUT_PLUGINS" | cut -f1 | grep -qx "$required_plugin"; then
+                echo "[!] Live capture did not expose the required $required_plugin output" >&2
+                tail -200 "$CAPTURE_ROOT/sweeting-live-capture.log" >&2
+                cat "$SNAPSHOT_DISCOVERY_REPORT" >&2
+                exit 1
+            fi
+        done
         while IFS=$'\t' read -r plugin_name output_capture_mode; do
             [[ -z "$plugin_name" ]] && continue
             VIEWS+=("Snapshot View ($plugin_name)|$LIVE_SNAPSHOT_VIEW_URL#$plugin_name|/|archivebox/templates/core/snapshot.html|$output_capture_mode")

bin/deploy_dev_demo.sh

@@ -111,7 +111,7 @@ if [[ -n "$DEPLOY_EXPECT_VERSION" && "$VERSION_OUTPUT" != *"ArchiveBox v${DEPLOY
 fi
 
 echo "[+] Health check:"
-"${COMPOSE[@]}" exec -T "$DEPLOY_SERVICE" /opt/archivebox/lib/env/bin/curl -fsS --max-time 10 --connect-timeout 2 -H 'Host: admin.archivebox.io' http://127.0.0.1:8000/health/ </dev/null
+"${COMPOSE[@]}" exec -T "$DEPLOY_SERVICE" /opt/archivebox/lib/env/bin/curl -fsS --max-time 10 --connect-timeout 2 -H 'Host: admin.archivebox.io' http://127.0.0.1:5797/health/ </dev/null
 REMOTE
 
 echo "[√] Demo deploy finished."

bin/docker_entrypoint.sh

@@ -230,11 +230,14 @@ assert_writable_dir "$DATA_DIR/logs"
 assert_writable_dir "$DATA_DIR/archive"
 assert_writable_dir "$PERSONAS_DIR/Default/chrome_profile"
 
-# check if novnc x11 $DISPLAY is available
-export DISPLAY="${DISPLAY:-"novnc:0.0"}"
-if ! xdpyinfo > /dev/null 2>&1; then
-    # cant connect to x11 display, unset it so that chrome doesn't try to connect to it and hang indefinitely
-    unset DISPLAY
+# Preserve an explicit DISPLAY (including an empty value to disable X11).
+# Only probe the implicit noVNC default: a missing probe or unavailable server
+# must not silently discard the user's display configuration.
+if [[ -z "${DISPLAY+x}" ]]; then
+    export DISPLAY="novnc:0.0"
+    if ! xdpyinfo > /dev/null 2>&1; then
+        unset DISPLAY
+    fi
 fi
 
 # Active browser processes do not survive container restarts, but their lock
@@ -250,6 +253,9 @@ find /tmp "$TMP_DIR" -maxdepth 1 -type d -name "archivebox-chrome-profile.*" -mm
     
 
 ensure_dir "/home/$ARCHIVEBOX_USER"
+# A nested read-only browser-profile mount can create this parent as root.
+# OpenCode's installer needs to create a sibling config directory as archivebox.
+ensure_dir "${XDG_CONFIG_HOME:-/home/$ARCHIVEBOX_USER/.config}"
 ensure_small_runtime_tree "$ABXBUS_CACHE_DIR"
 ensure_small_runtime_tree "$ABXBUS_CACHE_DIR/semaphores"
 ensure_small_runtime_tree "$UV_CACHE_DIR"
@@ -365,7 +371,7 @@ else
     # e.g. "docker run archivebox help"
     #      "docker run archivebox add --depth=1 https://example.com"
     #      "docker run archivebox manage createsupseruser"
-    #      "docker run archivebox server 0.0.0.0:8000"
+    #      "docker run archivebox server 0.0.0.0:5797"
     if [[ "$(id -u)" == "0" ]]; then
         exec setpriv --reuid="$ARCHIVEBOX_USER" --regid="$ARCHIVEBOX_USER" --init-groups "$ARCHIVEBOX_BIN_PATH" "$@"
     else

bin/generate_ui_screenshot_gallery.py

@@ -32,6 +32,7 @@ REQUIRED_VIEW_NAMES = {
     "Snapshot admin detail",
     "Snapshot View (capture in progress)",
     "Snapshot View (header collapsed)",
+    "Snapshot View (opentimestamps)",
     "Snapshot files",
     "Archive results",
     "Archive result detail",
@@ -128,7 +129,8 @@ def render_gallery(gallery_header: str, gallery_sections: str) -> str:
     return (
         '<!doctype html><html lang="en"><head><meta charset="utf-8">'
         '<meta name="viewport" content="width=device-width, initial-scale=1">'
-        "<title>ArchiveBox UI Screenshots</title><style>"
+        + (REPO_DIR / "bin/templates/screenshots-head.html").read_text(encoding="utf-8")
+        + "<style>"
         ":root{--ink:#28252a;--muted:#716c74;--accent:#9b2854;--paper:#fcfaf7;--line:#e9e2e3}"
         "*{box-sizing:border-box}body{margin:0;font:16px/1.75 system-ui,sans-serif;"
         "background:var(--paper);color:var(--ink)}.screenshot-content{max-width:1100px;margin:0 auto;padding:28px}"
@@ -145,7 +147,7 @@ def render_gallery(gallery_header: str, gallery_sections: str) -> str:
         "figcaption{font-size:12px;color:var(--muted);text-align:center;margin:0 0 12px}"
         "img{display:block;width:100%;height:auto;border-radius:8px}code{overflow-wrap:anywhere}"
         "@media(max-width:550px){.screenshot-content{padding:20px}figure{padding:8px;border-radius:12px}.profile-switch button{padding:9px 16px}}"
-        '</style><link rel="stylesheet" href="../site-chrome.css"></head><body>'
+        "</style></head><body>"
         + (REPO_DIR / "bin/templates/screenshots-header.html").read_text(encoding="utf-8")
         + '<div class="screenshot-content">'
         + gallery_header

bin/restore_site_screenshots.py

@@ -1,83 +1,70 @@
-#!/usr/bin/env python3
-"""Restore validated capture output without running application CI."""
+#!/usr/bin/env -S uv run --no-project python
+"""Restore validated screenshots, keeping capture and website revisions separate."""
 
 import argparse
-from concurrent.futures import ThreadPoolExecutor
 import hashlib
+import importlib
 import json
 import os
-from pathlib import Path, PurePosixPath
-import subprocess
-from urllib.error import HTTPError
-from urllib.request import urlopen
+import sys
+from concurrent.futures import ThreadPoolExecutor
+from pathlib import Path
 
-parser = argparse.ArgumentParser(description=__doc__)
-parser.add_argument("kind", choices=["archivebox", "ios", "extension"])
-parser.add_argument("destination", type=Path)
-args = parser.parse_args()
-args.destination.mkdir(parents=True, exist_ok=True)
-# Only successful default-branch capture runs are eligible, never PR artifacts.
-if os.environ.get("GH_TOKEN"):
-    repo = os.environ["GITHUB_REPOSITORY"]
-    branch = "dev" if args.kind == "archivebox" else "main"
-    runs = json.loads(
-        subprocess.check_output(
-            ["gh", "api", f"repos/{repo}/actions/workflows/screenshots.yml/runs?branch={branch}&status=success&per_page=100"],
-        ),
-    )
-    for run in runs["workflow_runs"]:
-        if run["event"] not in ("push", "workflow_dispatch"):
-            continue
-        artifacts = json.loads(subprocess.check_output(["gh", "api", f"repos/{repo}/actions/runs/{run['id']}/artifacts"]))
-        if any(a["name"] == "site-screenshots" and not a["expired"] for a in artifacts["artifacts"]):
-            subprocess.run(
-                ["gh", "run", "download", str(run["id"]), "--repo", repo, "--name", "site-screenshots", "--dir", str(args.destination)],
-                check=True,
-            )
-            print(f"Restored validated captures from run {run['id']} ({run['head_sha']})")
-            raise SystemExit(0)
-# Published captures remain available even after Actions artifacts expire.
-base, manifest_name = {
-    "archivebox": ("https://archivebox.io/screenshots/", "build.json"),
-    "ios": ("https://archivebox.github.io/ios-archivebox/screenshots/", "manifest.json"),
-    "extension": ("https://archivebox.github.io/archivebox-browser-extension/screenshots/", "manifest.json"),
-}[args.kind]
+PAGES = Path(__file__).resolve().parents[1] / ".github/pages"
+sys.path.insert(0, str(PAGES))
+artifacts = importlib.import_module("artifacts")
+
+# Repository-specific capture contract.
+REPO = "ArchiveBox/ArchiveBox"
+BRANCH = "dev"
+BASE = "https://archivebox.io/screenshots/"
+MANIFEST = "build.json"
 
 
-def fetch(name):
-    path = PurePosixPath(name)
-    if path.is_absolute() or ".." in path.parts or ":" in name or "\\" in name:
-        raise ValueError(f"Unsafe capture path: {name}")
-    with urlopen(base + name, timeout=60) as response:
-        data = response.read()
-    expected = hashes.get(name)
-    if expected and hashlib.sha256(data).hexdigest() != expected:
-        raise ValueError(f"Capture checksum mismatch: {name}")
-    target = args.destination / name
-    target.parent.mkdir(parents=True, exist_ok=True)
-    target.write_bytes(data)
+def validate(destination, run=None):
+    manifest = json.loads((destination / MANIFEST).read_text())
+    if run and manifest["revision"] != run["head_sha"]:
+        raise ValueError("Screenshot revision differs from the original capture run")
+    if not manifest.get("files") or len(manifest["files"]) != manifest["capture_count"]:
+        raise ValueError("Incomplete screenshot manifest")
+    for name, expected in manifest["files"].items():
+        image = destination / artifacts.relative_path(name)
+        if hashlib.sha256(image.read_bytes()).hexdigest() != expected:
+            raise ValueError(f"Screenshot checksum mismatch: {name}")
+    if not (destination / "index.html").is_file():
+        raise ValueError("Missing gallery content for the current renderer")
+    return manifest
 
 
-try:
-    with urlopen(base + manifest_name, timeout=60) as response:
-        raw = response.read()
-except HTTPError as error:
-    if error.code == 404 and args.kind == "ios":
-        print("No complete native gallery published yet; use the curated README screenshots.")
-        raise SystemExit(0)
-    raise
-manifest = json.loads(raw)
-hashes = {}
-if args.kind == "archivebox":
-    hashes = manifest["files"]
-    names = [*hashes, "index.html"]
-elif args.kind == "extension":
-    names = [image["file"] for capture in manifest["screenshots"] for image in capture["images"]]
-else:
-    if manifest.get("complete") is not True:
-        raise ValueError("Published native gallery must be complete")
-    names = [capture["path"] for capture in manifest["captures"]]
-with ThreadPoolExecutor(max_workers=8) as pool:
-    list(pool.map(fetch, names))
-(args.destination / manifest_name).write_bytes(raw)
-print(f"Restored {len(names)} published capture files, retaining revision {manifest['revision']}")
+def main():
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("destination", type=Path)
+    args = parser.parse_args()
+    args.destination.mkdir(parents=True, exist_ok=True)
+    if os.environ.get("GH_TOKEN"):
+        for run in artifacts.runs(REPO, "screenshots.yml", BRANCH):
+            if "site-screenshots" not in artifacts.names(REPO, run):
+                continue
+            artifacts.download(REPO, run, "site-screenshots", args.destination)
+            validate(args.destination, run)
+            print(f"Restored validated screenshots from {run['id']} ({run['head_sha']})")
+            return
+    raw = artifacts.fetch(BASE, MANIFEST)
+    manifest = json.loads(raw)
+    names = [*manifest["files"], "index.html"]
+
+    def restore(name):
+        target = args.destination / artifacts.relative_path(name)
+        data = artifacts.fetch(BASE, name)
+        target.parent.mkdir(parents=True, exist_ok=True)
+        target.write_bytes(data)
+
+    with ThreadPoolExecutor(max_workers=8) as pool:
+        list(pool.map(restore, names))
+    (args.destination / MANIFEST).write_bytes(raw)
+    validate(args.destination)
+    print(f"Restored {len(names)} published files from {manifest['revision']}")
+
+
+if __name__ == "__main__":
+    main()

bin/setup.sh

@@ -189,9 +189,9 @@ docker_run_archivebox_install() {
 
 docker_run_archivebox_server() {
     if [ -n "$ARCHIVEBOX_PLATFORM" ]; then
-        "$DOCKER_BINARY" run --platform "$ARCHIVEBOX_PLATFORM" -v "$PWD":/data -d -p 8000:8000 --name=archivebox "$ARCHIVEBOX_IMAGE"
+        "$DOCKER_BINARY" run --platform "$ARCHIVEBOX_PLATFORM" -v "$PWD":/data -d -p 5797:5797 --name=archivebox "$ARCHIVEBOX_IMAGE"
     else
-        "$DOCKER_BINARY" run -v "$PWD":/data -d -p 8000:8000 --name=archivebox "$ARCHIVEBOX_IMAGE"
+        "$DOCKER_BINARY" run -v "$PWD":/data -d -p 5797:5797 --name=archivebox "$ARCHIVEBOX_IMAGE"
     fi
 }
 
@@ -204,8 +204,8 @@ docker_compose_run_archivebox() {
 }
 
 wait_for_archivebox() {
-    url="http://127.0.0.1:8000/health/"
-    host_header="admin.archivebox.localhost:8000"
+    url="http://127.0.0.1:5797/health/"
+    host_header="admin.archivebox.localhost:5797"
     attempts=60
     attempt=1
 
@@ -224,7 +224,7 @@ wait_for_archivebox() {
 
 open_archivebox() {
     if [ -n "$OPEN_BINARY" ] && [ -t 1 ]; then
-        "$OPEN_BINARY" "http://admin.archivebox.localhost:8000/admin/" || true
+        "$OPEN_BINARY" "http://admin.archivebox.localhost:5797/admin/" || true
     fi
 }
 
@@ -410,13 +410,13 @@ if [ "$DOCKER_IMAGE_READY" = "true" ] && "$DOCKER_BINARY" compose version > /dev
     wait_for_archivebox
     open_archivebox
     echo
-    echo "[√] Server started on http://127.0.0.1:8000 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
+    echo "[√] Server started on http://127.0.0.1:5797 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
     echo "    cd $ARCHIVEBOX_HOME_DIR"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME compose ps"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME compose down"
     echo "    ${FOLLOWUP_SUDO}env ARCHIVEBOX_IMAGE=$ARCHIVEBOX_IMAGE $CONTAINER_ENGINE_NAME compose pull"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME compose up"
-    echo "    Open http://localhost:8000/admin/ to create the first admin and finish web setup."
+    echo "    Open http://localhost:5797/admin/ to create the first admin and finish web setup."
     echo "    (When running remotely, replace localhost with this server's IP address or hostname.)"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME compose exec archivebox archivebox add 'https://example.com'"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME compose exec archivebox archivebox list"
@@ -438,13 +438,13 @@ elif [ "$DOCKER_IMAGE_READY" = "true" ]; then
     wait_for_archivebox
     open_archivebox
     echo
-    echo "[√] Server started on http://127.0.0.1:8000 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
+    echo "[√] Server started on http://127.0.0.1:5797 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
     echo "    cd $ARCHIVEBOX_DATA_DIR"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME ps --filter name=archivebox"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME rm -f archivebox"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME pull $ARCHIVEBOX_IMAGE"
-    echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME run $DOCKER_PLATFORM_ARGS -v $PWD:/data -d -p 8000:8000 --name=archivebox $ARCHIVEBOX_IMAGE"
-    echo "    Open http://localhost:8000/admin/ to create the first admin and finish web setup."
+    echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME run $DOCKER_PLATFORM_ARGS -v $PWD:/data -d -p 5797:5797 --name=archivebox $ARCHIVEBOX_IMAGE"
+    echo "    Open http://localhost:5797/admin/ to create the first admin and finish web setup."
     echo "    (When running remotely, replace localhost with this server's IP address or hostname.)"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME run $DOCKER_PLATFORM_ARGS -v $PWD:/data -it $ARCHIVEBOX_IMAGE add 'https://example.com'"
     echo "    ${FOLLOWUP_SUDO}$CONTAINER_ENGINE_NAME run $DOCKER_PLATFORM_ARGS -v $PWD:/data -it $ARCHIVEBOX_IMAGE list"
@@ -501,14 +501,14 @@ select_archivebox_lib_dir
 # init shows version output at the end too
 echo
 echo "[+] Starting ArchiveBox server using: archivebox server --daemonize..."
-"$ARCHIVEBOX_BINARY" server --daemonize 0.0.0.0:8000
+"$ARCHIVEBOX_BINARY" server --daemonize 0.0.0.0:5797
 wait_for_archivebox
 open_archivebox
 echo
-echo "[√] Server started on http://127.0.0.1:8000 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
+echo "[√] Server started on http://127.0.0.1:5797 and data directory initialized in $ARCHIVEBOX_DATA_DIR. Usage:"
 echo "    cd $ARCHIVEBOX_DATA_DIR                            # see your data dir"
-echo "    ${FOLLOWUP_SUDO}archivebox server 0.0.0.0:8000                     # start server process"
-echo "    Open http://localhost:8000/admin/ to create the first admin and finish web setup."
+echo "    ${FOLLOWUP_SUDO}archivebox server 0.0.0.0:5797                     # start server process"
+echo "    Open http://localhost:5797/admin/ to create the first admin and finish web setup."
 echo "    (When running remotely, replace localhost with this server's IP address or hostname.)"
 echo "    ps aux | grep archivebox                           # see server process pid"
 echo "    ${FOLLOWUP_SUDO}pkill -f 'archivebox server'                       # stop the server"