Merge pull request #125 from W1lkins/fix-symlinks

Fix the setup/archive symlinks
add old locations too
2018-12-21 14:42:45 -08:00 · 2018-12-21 22:41:30 +00:00 · 2018-12-21 22:10:25 +00:00 · 2018-12-21 11:39:37 -08:00 · 2018-12-21 20:39:06 +01:00 · 2018-12-20 16:13:55 +01:00
47 changed files with 3440 additions and 847 deletions
@@ -0,0 +1,6 @@
+archiver/output
+archiver/tests
+__pycache__
+.DS_Store
+venv
+.venv
@@ -1,92 +1,14 @@
-# Pocket archive output folder
-pocket/
+# OS cruft
+.DS_Store
+._*

-# Byte-compiled / optimized / DLL files
+# python
 __pycache__/
-*.py[cod]
-*$py.class
+archiver/venv

-# C extensions
-*.so
+# vim
+.swp*

-# Distribution / packaging
-.Python
-env/
-build/
-develop-eggs/
-dist/
-downloads/
-eggs/
-.eggs/
-lib/
-lib64/
-parts/
-sdist/
-var/
-*.egg-info/
-.installed.cfg
-*.egg
-
-# PyInstaller
-#  Usually these files are written by a python script from a template
-#  before PyInstaller builds the exe, so as to inject date/other infos into it.
-*.manifest
-*.spec
-
-# Installer logs
-pip-log.txt
-pip-delete-this-directory.txt
-
-# Unit test / coverage reports
-htmlcov/
-.tox/
-.coverage
-.coverage.*
-.cache
-nosetests.xml
-coverage.xml
-*,cover
-.hypothesis/
-
-# Translations
-*.mo
-*.pot
-
-# Django stuff:
-*.log
-local_settings.py
-
-# Flask stuff:
-instance/
-.webassets-cache
-
-# Scrapy stuff:
-.scrapy
-
-# Sphinx documentation
-docs/_build/
-
-# PyBuilder
-target/
-
-# IPython Notebook
-.ipynb_checkpoints
-
-# pyenv
-.python-version
-
-# celery beat schedule file
-celerybeat-schedule
-
-# dotenv
-.env
-
-# virtualenv
-venv/
-ENV/
-
-# Spyder project settings
-.spyderproject
-
-# Rope project settings
-.ropeproject
+# output artifacts
+output
+output/
@@ -1,46 +0,0 @@
-# Contributor Covenant Code of Conduct
-
-## Our Pledge
-
-In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
-
-## Our Standards
-
-Examples of behavior that contributes to creating a positive environment include:
-
-* Using welcoming and inclusive language
-* Being respectful of differing viewpoints and experiences
-* Gracefully accepting constructive criticism
-* Focusing on what is best for the community
-* Showing empathy towards other community members
-
-Examples of unacceptable behavior by participants include:
-
-* The use of sexualized language or imagery and unwelcome sexual attention or advances
-* Trolling, insulting/derogatory comments, and personal or political attacks
-* Public or private harassment
-* Publishing others' private information, such as a physical or electronic address, without explicit permission
-* Other conduct which could reasonably be considered inappropriate in a professional setting
-
-## Our Responsibilities
-
-Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
-
-Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
-
-## Scope
-
-This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
-
-## Enforcement
-
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at git@nicksweeting.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
-
-Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
-
-## Attribution
-
-This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]
-
-[homepage]: http://contributor-covenant.org
-[version]: http://contributor-covenant.org/version/1/4/
@@ -0,0 +1,15 @@
+# Donate
+
+Right now I'm working on this project in my spare time and accepting the occational PR,   
+if you want me to dedicate more time to it, donate to support development!
+
+ - Ethereum: 0x5B0F85FFc44fD759C2d97f0BE4681279966f3832
+ - Bitcoin: https://shapeshift.io/ BTC -> to address above (ETH)
+ - Paypal: https://www.paypal.me/NicholasSweeting/25
+
+The eventual goal is to support one or two developers full-time on this project via donations.   
+With more engineering power this can become a distributed archive service with a nice UI,   
+like the Way-Back machine but hosted by everyone!
+
+If have any questions or want to sponsor this project long-term, contact me at  
+bookmark-archiver@sweeting.me
@@ -0,0 +1,48 @@
+FROM debian:stretch
+LABEL maintainer="Nick Sweeting <bookmark-archiver@sweeting.me>"
+
+RUN apt-get update \
+    && apt-get install -qy git wget gnupg2 libgconf-2-4 python3 python3-pip \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install latest chrome package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
+RUN apt-get update && apt-get install -y curl --no-install-recommends \
+    && wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
+    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
+    && apt-get update \
+    && apt-get install -y chromium fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst ttf-freefont \
+      --no-install-recommends \
+    && rm -rf /var/lib/apt/lists/* \
+    && rm -rf /src/*.deb \
+    && ln -s /usr/bin/chromium /usr/bin/chromium-browser
+
+# It might be a good idea to use dumb-init to help prevent zombie chrome processes.
+# ADD https://github.com/Yelp/dumb-init/releases/download/v1.2.0/dumb-init_1.2.0_amd64 /usr/local/bin/dumb-init
+# RUN chmod +x /usr/local/bin/dumb-init
+
+RUN git clone https://github.com/pirate/ArchiveBox /home/chromeuser/app \
+    && pip3 install -r /home/chromeuser/app/archiver/requirements.txt
+
+# Add user so we area strong, independent chrome that don't need --no-sandbox.
+RUN groupadd -r chromeuser && useradd -r -g chromeuser -G audio,video chromeuser \
+    && mkdir -p /home/chromeuser/app/archiver/output \
+    && chown -R chromeuser:chromeuser /home/chromeuser/app/archiver/output \
+    && chown -R chromeuser:chromeuser /home/chromeuser
+
+VOLUME /home/chromeuser/app/archiver/output
+
+ENV LANG=en_US.UTF-8 \
+    LANGUAGE=en_US:en \
+    LC_ALL=en_US.UTF-8 \
+    PYTHONIOENCODING=UTF-8 \
+    CHROME_SANDBOX=False \
+    OUTPUT_DIR=/home/chromeuser/app/archiver/output
+
+# Run everything from here on out as non-privileged user
+USER chromeuser
+WORKDIR /home/chromeuser/app
+
+# ENTRYPOINT ["dumb-init", "--"]
+# CMD ["/home/chromeuser/app/archive"]
+
+ENTRYPOINT ["python3", "-u", "/home/chromeuser/app/archiver/archive.py"]
@@ -1,64 +1,132 @@
-# Bookmark Archiver <img src="https://getpocket.com/favicon.ico" height="22px"/> <img src="https://pinboard.in/favicon.ico" height="22px"/> <img src="https://nicksweeting.com/images/bookmarks.png" height="22px"/> [![Twitter URL](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/thesquashSH)
+# ArchiveBox: Open source local web archiving <img src="https://nicksweeting.com/images/archive.png" height="22px"/>  [![Github Stars](https://img.shields.io/github/stars/pirate/bookmark-archiver.svg)](https://github.com/pirate/ArchiveBox) [![Twitter URL](https://img.shields.io/twitter/url/http/shields.io.svg?style=social)](https://twitter.com/thesquashSH)

-▶️ [Quickstart](#quickstart) | [Details](#details) | [Configuration](#configuration) | [Manual Setup](#manual-setup) | [Troubleshooting](#troubleshooting) | [Changelog](#changelog)
+ ### (Recently [renamed](https://github.com/pirate/ArchiveBox/issues/108) from `Bookmark Archiver`)

-Save an archived copy of all websites you star (the actual *content* of the sites, not just the list of bookmarks).
+    "Your own personal Way-Back Machine"

-**Supports: Browser Bookmarks (Chrome, Firefox, Safari, IE, Opera), Pocket, Pinboard, Shaarli, Delicious, Instapaper, Unmark.it, and more!**
+▶️ [Quickstart](#quickstart) | [Details](#details) | [Configuration](#configuration) | [Manual Setup](#manual-setup) | [Troubleshooting](#troubleshooting) | [Demo](https://archive.sweeting.me) | [Source](https://github.com/pirate/ArchiveBox/tree/master) | [Changelog](#changelog) | [Donate](https://github.com/pirate/ArchiveBox/blob/master/DONATE.md)

-Outputs browsable static html archives of each site, a PDF, a screenshot, and a link to a copy on archive.org, all indexed in a nice html file.  
-(Your own personal Way-Back Machine) [DEMO: sweeting.me/pocket](https://home.sweeting.me/pocket)
+---

-![](screenshot.png)
+Save an archived copy of the websites you visit (the actual *content* of each site, not just the list of links).  Can archive entire browsing history, or just links matching a filter or bookmarks list.
+
+ArchiveBox can import links from:
+
+ - <img src="https://nicksweeting.com/images/bookmarks.png" height="22px"/> Browser history or bookmarks (Chrome, Firefox, Safari, IE, Opera)
+ - <img src="https://getpocket.com/favicon.ico" height="22px"/> Pocket
+ - <img src="https://pinboard.in/favicon.ico" height="22px"/> Pinboard
+ - <img src="https://nicksweeting.com/images/rss.svg" height="22px"/> RSS or plain text lists
+ - Shaarli, Delicious, Instapaper, Reddit Saved Posts, Wallabag, Unmark.it, and more!
+
+For each site, it outputs (configurable):
+
+- Browsable static HTML archive (wget)
+- PDF (Chrome headless)
+- Screenshot (Chrome headless)
+- HTML after 2s of JS running (Chrome headless)
+- Favicon
+- Submits URL to archive.org
+- Index summary pages: index.html & index.json
+
+The archiving is additive, so you can schedule `./archive` to run regularly and pull new links into the index.
+All the saved content is static and indexed with json files, so it lives forever & is easily parseable, it requires no always-running backend.
+
+[DEMO: archive.sweeting.me](https://archive.sweeting.me)
+
+<img src="https://i.imgur.com/q3Oz9wN.png" width="75%" alt="Desktop Screenshot" align="top"><img src="https://i.imgur.com/TG0fGVo.png" width="25%" alt="Mobile Screenshot" align="top"><br/>

 ## Quickstart

-**1. Get your bookmarks:**
+**1. Get your list of URLs:**

-Follow the links here to find instructions for exporting bookmarks from each service.
+Follow the links here to find instructions for exporting a list of URLs from each service.

 - [Pocket](https://getpocket.com/export)
 - [Pinboard](https://pinboard.in/export/)
 - [Instapaper](https://www.instapaper.com/user/export)
- - [Shaarli](http://sebsauvage.net/wiki/lib/exe/fetch.php?media=php:php_shaarli:shaarli_cap16_dragbutton.png)
+ - [Reddit Saved Posts](https://github.com/csu/export-saved-reddit)
+ - [Shaarli](https://shaarli.readthedocs.io/en/master/guides/backup-restore-import-export/#export-links-as)
 - [Unmark.it](http://help.unmark.it/import-export)
+ - [Wallabag](https://doc.wallabag.org/en/user/import/wallabagv2.html)
 - [Chrome Bookmarks](https://support.google.com/chrome/answer/96816?hl=en)
 - [Firefox Bookmarks](https://support.mozilla.org/en-US/kb/export-firefox-bookmarks-to-backup-or-transfer)
 - [Safari Bookmarks](http://i.imgur.com/AtcvUZA.png)
 - [Opera Bookmarks](http://help.opera.com/Windows/12.10/en/importexport.html)
 - [Internet Explorer Bookmarks](https://support.microsoft.com/en-us/help/211089/how-to-import-and-export-the-internet-explorer-favorites-folder-to-a-32-bit-version-of-windows)
+ - Chrome History: `./bin/export-browser-history --chrome`
+ - Firefox History: `./bin/export-browser-history --firefox`
+ - Other File or URL: (e.g. RSS feed) pass as second argument in the next step

 (If any of these links are broken, please submit an issue and I'll fix it)

 **2. Create your archive:**

 ```bash
-git clone https://github.com/pirate/bookmark-archiver
-cd bookmark-archiver/
-./setup.sh
-./archive.py ~/Downloads/bookmark_export.html   # replace this path with the path to your bookmarks export file
+git clone https://github.com/pirate/ArchiveBox
+cd ArchiveBox/
+./setup                                         # install all dependencies
+
+# add a list of links from a file
+./archive ~/Downloads/bookmark_export.html      # replace with the path to your export file or URL from step 1
+
+# OR add a list of links from remote URL
+./archive "https://getpocket.com/users/yourusername/feed/all"  # url to an RSS, html, or json links file
+
+# OR add all the links from your browser history
+./bin/export-browser-history --chrome           # works with --firefox as well, can take path to SQLite history db
+./archive output/sources/chrome_history.json
+
+# OR just continue archiving the existing links in the index
+./archive   # at any point if you just want to continue archiving where you left off, without adding any new links
 ```

-You can open `service/index.html` to view your archive.  (favicons will appear next to each title once it has finished downloading)
+**3. Done!**

+You can open `output/index.html` to view your archive.  (favicons will appear next to each title once it has finished downloading)
+
+If you want to host your archive somewhere to share it with other people, see the [Publishing Your Archive](#publishing-your-archive) section below.
+
+**4. (Optional) Schedule it to run every day**
+
+You can import links from any local file path or feed url by changing the second argument to `archive.py`.
+ArchiveBox will ignore links that are imported multiple times, it will keep the earliest version that it's seen.
+This means you can add multiple cron jobs to pull links from several different feeds or files each day,
+it will keep the index up-to-date without duplicate links.
+
+This example archives a pocket RSS feed and an export file every 24 hours, and saves the output to a logfile.
+```bash
+0 24 * * * yourusername /opt/ArchiveBox/archive https://getpocket.com/users/yourusername/feed/all > /var/log/archivebox_rss.log
+0 24 * * * yourusername /opt/ArchiveBox/archive /home/darth-vader/Desktop/bookmarks.html > /var/log/archivebox_firefox.log
+```
+(Add the above lines to `/etc/crontab`)
+
+**Next Steps**  
+  
 If you have any trouble, see the [Troubleshooting](#troubleshooting) section at the bottom.  
+If you'd like to customize options, see the [Configuration](#configuration) section.  

-If you want something easier than running programs in the command-line, take a look at [Pocket Premium](https://getpocket.com/premium) (yay Mozilla!) and [Pinboard Pro](https://pinboard.in/upgrade/), which both offer easy-to-use bookmark archiving with full-text-search.
+If you want something easier than running programs in the command-line, take a look at [Pocket Premium](https://getpocket.com/premium) (yay Mozilla!) and [Pinboard Pro](https://pinboard.in/upgrade/) (yay independent developer!).  Both offer easy-to-use bookmark archiving with full-text-search and other features.

 ## Details

-`archive.py` is a script that takes a [Pocket-format](https://getpocket.com/export), [Pinboard-format](https://pinboard.in/export/), or [Netscape-format](https://msdn.microsoft.com/en-us/library/aa753582(v=vs.85).aspx) bookmark export file, and turns it into a browsable archive that you can store locally or host online.
+`archive.py` is a script that takes a [Pocket-format](https://getpocket.com/export), [JSON-format](https://pinboard.in/export/), [Netscape-format](https://msdn.microsoft.com/en-us/library/aa753582(v=vs.85).aspx), or RSS-formatted list of links, and downloads a clone of each linked website to turn into a browsable archive that you can store locally or host online.

-The archiver produces a folder like `pocket/` containing an `index.html`, and archived copies of all the sites,
-organized by starred timestamp.  It's Powered by the [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) Chromium and good 'ol `wget`.
-NEW: Also submits each link to save on archive.org!
+The archiver produces an output folder `output/` containing an `index.html`, `index.json`, and archived copies of all the sites,
+organized by timestamp bookmarked.  It's Powered by [headless](https://developers.google.com/web/updates/2017/04/headless-chrome) Chromium and good 'ol `wget`.

 For each sites it saves:

 - wget of site, e.g. `en.wikipedia.org/wiki/Example.html` with .html appended if not present
- - `sreenshot.png` 1440x900 screenshot of site using headless chrome
 - `output.pdf` Printed PDF of site using headless chrome
+ - `screenshot.png` 1440x900 screenshot of site using headless chrome
+ - `output.html` DOM Dump of the HTML after rendering using headless chrome
 - `archive.org.txt` A link to the saved site on archive.org
+ - `audio/` and `video/` for sites like youtube, soundcloud, etc. (using youtube-dl) (WIP)
+ - `code/` clone of any repository for github, bitbucket, or gitlab links (WIP)
+ - `index.json` JSON index containing link info and archive details
+ - `index.html` HTML index containing link info and archive details (optional fancy or simple index)
+
+Wget doesn't work on sites you need to be logged into, but chrome headless does, see the [Configuration](#configuration)* section for `CHROME_USER_DATA_DIR`.

 **Large Exports & Estimated Runtime:** 

@@ -67,68 +135,92 @@ Those numbers are from running it single-threaded on my i5 machine with 50mbps d

 You can run it in parallel by using the `resume` feature, or by manually splitting export.html into multiple files:
 ```bash
-./archive.py export.html bookmarks 1498800000 &  # third argument is timestamp to resume downloading from
-./archive.py export.html bookmarks 1498810000 &
-./archive.py export.html bookmarks 1498820000 &
-./archive.py export.html bookmarks 1498830000 &
+./archive export.html 1498800000 &  # second argument is timestamp to resume downloading from
+./archive export.html 1498810000 &
+./archive export.html 1498820000 &
+./archive export.html 1498830000 &
 ```
 Users have reported running it with 50k+ bookmarks with success (though it will take more RAM while running).

+If you already imported a huge list of bookmarks and want to import only new
+bookmarks, you can use the `ONLY_NEW` environment variable. This is useful if
+you want to import a bookmark dump periodically and want to skip broken links
+which are already in the index.
+
 ## Configuration

-You can tweak parameters via environment variables, or by editing `archive.py` directly:
+You can tweak parameters via environment variables, or by editing `config.py` directly:
 ```bash
-env CHROME_BINARY=google-chrome-stable RESOLUTION=1440,900 FETCH_PDF=False ./archive.py ~/Downloads/bookmarks_export.html
+env CHROME_BINARY=google-chrome-stable RESOLUTION=1440,900 FETCH_PDF=False ./archive ~/Downloads/bookmarks_export.html
 ```

- - Archive methods: `FETCH_WGET`, `FETCH_PDF`, `FETCH_SCREENSHOT`, `FETCH_FAVICON`, `SUBMIT_ARCHIVE_DOT_ORG` values: [`True`]/`False`
- - Archive images/css/js: `FETCH_WGET_REQUISITES` values: [`True`]/`False`  (True is highly recommended)
- - Screenshot: `RESOLUTION` values: [`1440,900`]/`1024,768`/`...`
- - Outputted Files: `ARCHIVE_PERMISSIONS` values: [`755`]/`644`/`...`
- - Path to Chrome: `CHROME_BINARY` values: [`chromium-browser`]/`/usr/local/bin/chromium-browser`/`...`
- - Path to wget: `WGET_BINARY` values: [`wget`]/`/usr/local/bin/wget`/`...`
- - Download timemout: `TIMEOUT` values: [`60`]/`30`/`...`
+**Shell Options:**
+ - colorize console ouput: `USE_COLOR` value: [`True`]/`False`
+ - show progress bar: `SHOW_PROGRESS` value: [`True`]/`False`
+ - archive permissions: `OUTPUT_PERMISSIONS` values: [`755`]/`644`/`...`

- (See defaults & more at the top of `archive.py`)
+**Dependency Options:**
+ - path to Chrome: `CHROME_BINARY` values: [`chromium-browser`]/`/usr/local/bin/google-chrome`/`...`
+ - path to wget: `WGET_BINARY` values: [`wget`]/`/usr/local/bin/wget`/`...`

-You can also tweak the outputted html index in `index_template.html`.  It just uses python
-format strings (not a proper templating engine like jinja2), which is why the CSS is double-bracketed `{{...}}`.
+**Archive Options:**
+ - maximum allowed download time per link: `TIMEOUT` values: [`60`]/`30`/`...`
+ - import only new links: `ONLY_NEW` values `True`/[`False`]
+ - archive methods (values: [`True`]/`False`):
+   - fetch page with wget: `FETCH_WGET`
+   - fetch images/css/js with wget: `FETCH_WGET_REQUISITES` (True is highly recommended)
+   - print page as PDF: `FETCH_PDF`
+   - fetch a screenshot of the page: `FETCH_SCREENSHOT`
+   - fetch a DOM dump of the page: `FETCH_DOM`
+   - fetch a favicon for the page: `FETCH_FAVICON`
+   - submit the page to archive.org: `SUBMIT_ARCHIVE_DOT_ORG` 
+ - screenshot: `RESOLUTION` values: [`1440,900`]/`1024,768`/`...`
+ - user agent: `WGET_USER_AGENT` values: [`Wget/1.19.1`]/`"Mozilla/5.0 ..."`/`...`
+ - chrome profile: `CHROME_USER_DATA_DIR` values: [`~/Library/Application\ Support/Google/Chrome/Default`]/`/tmp/chrome-profile`/`...`
+    To capture sites that require a user to be logged in, you must specify a path to a chrome profile (which loads the cookies needed for the user to be logged in).  If you don't have an existing chrome profile, create one with `chromium-browser --disable-gpu --user-data-dir=/tmp/chrome-profile`, and log into the sites you need.  Then set `CHROME_USER_DATA_DIR=/tmp/chrome-profile` to make ArchiveBox use that profile.
+ - output directory: `OUTPUT_DIR` values: [`$REPO_DIR/output`]/`/srv/www/bookmarks`/`...` Optionally output the archives to an alternative directory.
+
+ (See defaults & more at the top of `config.py`)
+
+To tweak the outputted html index file's look and feel, just edit the HTML files in `archiver/templates/`.
+
+The chrome/chromium dependency is _optional_ and only required for screenshots, PDF, and DOM dump output, it can be safely ignored if those three methods are disabled.

 ## Publishing Your Archive

-The archive produced by `./archive.py` is suitable for serving on any provider that 
-can host static html (e.g. github pages!).
+The archive produced by `./archive` is suitable for serving on any provider that can host static html (e.g. github pages!).

-You can also serve it from a home server or VPS by uploading the archive folder 
-to your web directory, e.g. `/var/www/pocket` and configuring your webserver.
+You can also serve it from a home server or VPS by uploading the outputted `output` folder to your web directory, e.g. `/var/www/ArchiveBox` and configuring your webserver.

 Here's a sample nginx configuration that works to serve archive folders:

 ```nginx
-location /pocket/ {
-    alias       /var/www/pocket/;
+location / {
+    alias       /path/to/ArchiveBox/output/;
    index       index.html;
-    autoindex   on;                         # see directory listing upon clicking "The Files" links
+    autoindex   on;               # see directory listing upon clicking "The Files" links
    try_files   $uri $uri/ =404;
 }
 ```

 Make sure you're not running any content as CGI or PHP, you only want to serve static files!

-Urls look like: `https://sweeting.me/archive/archive/1493350273/en.wikipedia.org/wiki/Dining_philosophers_problem`
+Urls look like: `https://archive.example.com/archive/1493350273/en.wikipedia.org/wiki/Dining_philosophers_problem.html`

 **Security WARNING & Content Disclaimer**

-Hosting other people's site content has security implications for other sites on the same domain, make sure you understand
-the dangers of hosting other people's CSS & JS files [on a shared domain](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy).  It's best to put this on a domain/subdomain
-of its own to slightly mitigate [CSRF attacks](https://en.wikipedia.org/wiki/Cross-site_request_forgery).
+Re-hosting other people's content has security implications for any other sites sharing your hosting domain.  Make sure you understand
+the dangers of hosting unknown archived CSS & JS files [on your shared domain](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy).
+Due to the security risk of serving some malicious JS you archived by accident, it's best to put this on a domain or subdomain
+of its own to keep cookies separate and slightly mitigate [CSRF attacks](https://en.wikipedia.org/wiki/Cross-site_request_forgery) and other nastiness.

-You may also want to blacklist your archive in `/robots.txt` if you don't want to be publicly assosciated
-with all the links you archive via search engine results.
+You may also want to blacklist your archive in `/robots.txt` if you don't want to be publicly assosciated with all the links you archive via search engine results.

 Be aware that some sites you archive may not allow you to rehost their content publicly for copyright reasons,
 it's up to you to host responsibly and respond to takedown requests appropriately.

+Please modify the `FOOTER_INFO` config variable to add your contact info to the footer of your index.
+
 ## Info & Motivation

 This is basically an open-source version of [Pocket Premium](https://getpocket.com/premium) (which you should consider paying for!).
@@ -139,15 +231,16 @@ PDFs & Screenshots of dynamic sites in addition to static html, something archiv

 Now I can rest soundly knowing important articles and resources I like wont dissapear off the internet.

-My published archive as an example: [sweeting.me/pocket](https://home.sweeting.me/pocket).
+My published archive as an example: [archive.sweeting.me](https://archive.sweeting.me).

 ## Manual Setup

-If you don't like `sudo` running random setup scripts off the internet (which you shouldn't), you can follow these manual setup instructions.
+If you don't like running random setup scripts off the internet (:+1:), you can follow these manual setup instructions.

-**1. Install dependencies:** `chromium >= 59`,` wget >= 1.16`, `python3 >= 3.5`  (google-chrome >= v59 also works well)
+**1. Install dependencies:** `chromium >= 59`,` wget >= 1.16`, `python3 >= 3.5`  (`google-chrome >= v59` works fine as well)

 If you already have Google Chrome installed, or wish to use that instead of Chromium, follow the [Google Chrome Instructions](#google-chrome-instructions).
+
 ```bash
 # On Mac:
 brew cask install chromium  # If you already have Google Chrome/Chromium in /Applications/, skip this command
@@ -173,11 +266,11 @@ Follow the instruction links above in the "Quickstart" section to download your

 **3. Run the archive script:**

-1. Clone this repo `git clone https://github.com/pirate/bookmark-archiver`
-3. `cd bookmark-archiver/`
-4. `./archive.py ~/Downloads/bookmarks_export.html`
+1. Clone this repo `git clone https://github.com/pirate/ArchiveBox`
+3. `cd ArchiveBox/`
+4. `./archive ~/Downloads/bookmarks_export.html`

-You may optionally specify a third argument to `archive.py export.html [pocket|pinboard|bookmarks]` to enforce the use of a specific link parser.
+You may optionally specify a second argument to `archive.py export.html 153242424324` to resume the archive update at a specific timestamp.

 If you have any trouble, see the [Troubleshooting](#troubleshooting) section at the bottom.

@@ -208,7 +301,7 @@ apt update; apt install google-chrome-beta python3 wget
 2. Set the environment variable `CHROME_BINARY` to `google-chrome` before running:

 ```bash
-env CHROME_BINARY=google-chrome ./archive.py ~/Downloads/bookmarks_export.html
+env CHROME_BINARY=google-chrome ./archive ~/Downloads/bookmarks_export.html
 ```
 If you're having any trouble trying to set up Google Chrome or Chromium, see the Troubleshooting section below.

@@ -229,27 +322,40 @@ If you still need help, [the official Python docs](https://docs.python.org/3.6/u

 `archive.py` depends on being able to access a `chromium-browser`/`google-chrome` executable.  The executable used
 defaults to `chromium-browser` but can be manually specified with the environment variable `CHROME_BINARY`:
+
 ```bash
-env CHROME_BINARY=/usr/local/bin/chromium-browser ./archive.py ~/Downloads/bookmarks_export.html
+env CHROME_BINARY=/usr/local/bin/chromium-browser ./archive ~/Downloads/bookmarks_export.html
 ```

 1. Test to make sure you have Chrome on your `$PATH` with:
+
 ```bash
 which chromium-browser || which google-chrome
 ```
 If no executable is displayed, follow the setup instructions to install and link one of them.

 2. If a path is displayed, the next step is to check that it's runnable:
+
 ```bash
 chromium-browser --version || google-chrome --version
 ```
 If no version is displayed, try the setup instructions again, or confirm that you have permission to access chrome.

-3. If a version is displayed and it's `>=59`, make sure `archive.py` is running the right one:
+3. If a version is displayed and it's `<59`, upgrade it:
+
 ```bash
-env CHROME_BINARY=/path/from/step/1/chromium-browser ./archive.py bookmarks_export.html   # replace the path with the one you got from step 1
+apt upgrade chromium-browser -y
+# OR
+brew cask upgrade chromium-browser
 ```

+4. If a version is displayed and it's `>=59`, make sure `archive.py` is running the right one:
+
+```bash
+env CHROME_BINARY=/path/from/step/1/chromium-browser ./archive bookmarks_export.html   # replace the path with the one you got from step 1
+```
+
+
 **Wget & Curl:**

 If you're missing `wget` or `curl`, simply install them using `apt` or your package manager of choice.
@@ -263,7 +369,7 @@ a bug in versions `<=1.19.1_1` that caused wget to fail for perfectly valid site

 **No links parsed from export file:**

-Please open an [issue](https://github.com/pirate/bookmark-archiver/issues) with a description of where you got the export, and
+Please open an [issue](https://github.com/pirate/ArchiveBox/issues) with a description of where you got the export, and
 preferrably your export file attached (you can redact the links).  We'll fix the parser to support your format.

 **Lots of skipped sites:**
@@ -272,64 +378,111 @@ If you ran the archiver once, it wont re-download sites subsequent times, it wil
 If you haven't already run it, make sure you have a working internet connection and that the parsed URLs look correct.
 You can check the `archive.py` output or `index.html` to see what links it's downloading.

-If you're still having issues, try deleting or moving the `service/archive` folder and running `archive.py` again.
+If you're still having issues, try deleting or moving the `output/archive` folder (back it up first!) and running `./archive` again.

 **Lots of errors:**

 Make sure you have all the dependencies installed and that you're able to visit the links from your browser normally.
-Open an [issue](https://github.com/pirate/bookmark-archiver/issues) with a description of the errors if you're still having problems.
+Open an [issue](https://github.com/pirate/ArchiveBox/issues) with a description of the errors if you're still having problems.

 **Lots of broken links from the index:**

 Not all sites can be effectively archived with each method, that's why it's best to use a combination of `wget`, PDFs, and screenshots.
-If it seems like more than 10-20% of sites in the archive are broken, open an [issue](https://github.com/pirate/bookmark-archiver/issues)
+If it seems like more than 10-20% of sites in the archive are broken, open an [issue](https://github.com/pirate/ArchiveBox/issues)
 with some of the URLs that failed to be archived and I'll investigate.

+**Removing unwanted links from the index:**
+
+If you accidentally added lots of unwanted links into index and they slow down your archiving, you can use the `bin/purge` script to remove them from your index, which removes everything matching python regexes you pass into it. E.g: `bin/purge -r 'amazon\.com' -r 'google\.com'`. It would prompt before removing links from index, but for extra safety you might want to back up `index.json` first (or put in undex version control).
+
 ### Hosting the Archive

 If you're having issues trying to host the archive via nginx, make sure you already have nginx running with SSL.
-If you don't, google around, there are plenty of tutorials to help get that set up.  Open an [issue](https://github.com/pirate/bookmark-archiver/issues)
+If you don't, google around, there are plenty of tutorials to help get that set up.  Open an [issue](https://github.com/pirate/ArchiveBox/issues)
 if you have problem with a particular nginx config.

-## TODO
-
- - body text extraction using [fathom](https://hacks.mozilla.org/2017/04/fathom-a-framework-for-understanding-web-pages/)
- - auto-tagging based on important extracted words
- - audio & video archiving with `youtube-dl`
- - full-text indexing with elasticsearch
- - video closed-caption downloading for full-text indexing video content
- - automatic text summaries of article with summarization library
- - feature image extraction
- - http support (from my https-only domain)
- - try wgetting dead sites from archive.org (https://github.com/hartator/wayback-machine-downloader)
-
-**Live Updating:** (coming soon... maybe...)
-
-It's possible to pull links via the pocket API or public pocket RSS feeds instead of downloading an html export.
-Once I write a script to do that, we can stick this in `cron` and have it auto-update on it's own.
-
-For now you just have to download `ril_export.html` and run `archive.py` each time it updates. The script
-will run fast subsequent times because it only downloads new links that haven't been archived already.

 ## Links

+**Similar Projects:**
+ - [Reminiscence](https://github.com/kanishka-linux/reminiscence/) extremely similar to BA, uses a Django backend + UI and provides auto tagging and summary features with NLTK
+ - [Memex by Worldbrain.io](https://github.com/WorldBrain/Memex) a browser extension that saves all your history and does full-text search
+ - [Hypothes.is](https://web.hypothes.is/) a web/pdf/ebook annotation tool that also archives content
+ - [Perkeep](https://perkeep.org/) "Perkeep lets you permanently keep your stuff, for life."
+ - [Fetching.io](http://fetching.io/) A personal search engine/archiver that lets you search through all archived websites that you've bookmarked
+ - [Shaarchiver](https://github.com/nodiscc/shaarchiver) very similar project that archives Firefox, Shaarli, or Delicious bookmarks and all linked media, generating a markdown/HTML index
+ - [Webrecorder.io](https://webrecorder.io/) Save full browsing sessions and archive all the content
+ - [Wallabag](https://wallabag.org) Save articles you read locally or on your phone
+
+**Discussions:**
 - [Hacker News Discussion](https://news.ycombinator.com/item?id=14272133)
 - [Reddit r/selfhosted Discussion](https://www.reddit.com/r/selfhosted/comments/69eoi3/pocket_stream_archive_your_own_personal_wayback/)
 - [Reddit r/datahoarder Discussion #1](https://www.reddit.com/r/DataHoarder/comments/69e6i9/archive_a_browseable_copy_of_your_saved_pocket/)
 - [Reddit r/datahoarder Discussion #2](https://www.reddit.com/r/DataHoarder/comments/6kepv6/bookmarkarchiver_now_supports_archiving_all_major/)
- - https://wallabag.org + https://github.com/wallabag/wallabag
- - https://webrecorder.io/
- - https://github.com/ikreymer/webarchiveplayer#auto-load-warcs
- - [Shaarchiver](https://github.com/nodiscc/shaarchiver) very similar project that archives Firefox, Shaarli, or Delicious bookmarks and all linked media, generating a markdown/HTML index

+
+**Tools/Other:**
+ - https://github.com/ikreymer/webarchiveplayer#auto-load-warcs
+ - [Sheetsee-Pocket](http://jlord.us/sheetsee-pocket/) project that provides a pretty auto-updating index of your Pocket links (without archiving them)
+ - [Pocket -> IFTTT -> Dropbox](https://christopher.su/2013/saving-pocket-links-file-day-dropbox-ifttt-launchd/) Post by Christopher Su on his Pocket saving IFTTT recipie
+
+
+## Roadmap
+
+[*Official Roadmap*](https://github.com/pirate/ArchiveBox/issues/120).
+
+If you feel like contributing a PR, some of these tasks are pretty easy.  Feel free to open an issue if you need help getting started in any way!
+
+**Major upcoming changes:**
+
+ - finalize python packaging to allow installing via pip and importing individual componenets
+ - add an optional web GUI for managing sources, adding new links, and viewing the archive
+
+**Minor upcoming changes:**
+ - download closed-captions text from youtube videos
+ - body text extraction using [fathom](https://hacks.mozilla.org/2017/04/fathom-a-framework-for-understanding-web-pages/)
+ - auto-tagging based on important extracted words
+ - audio & video archiving with `youtube-dl`
+ - full-text indexing with elasticsearch/elasticlunr/ag
+ - video closed-caption downloading on Youtube for full-text indexing of video content
+ - automatic text summaries of article with nlp summarization library
+ - featured image extraction
+ - http support (from my https-only domain)
+ - try wgetting dead sites from archive.org (https://github.com/hartator/wayback-machine-downloader)
+ 
+ 
 ## Changelog

+ - [renamed](https://github.com/pirate/ArchiveBox/issues/108) from **Bookmark Archiver** -> **ArchiveBox**
+ - v0.1.0 released
+ - support for browser history exporting added with `./bin/export-browser-history`
+ - support for chrome `--dump-dom` to output full page HTML after JS executes
+ - v0.0.3 released
+ - support for chrome `--user-data-dir` to archive sites that need logins
+ - fancy individual html & json indexes for each link
+ - smartly append new links to existing index instead of overwriting 
+  - v0.0.2 released
+ - proper HTML templating instead of format strings (thanks to https://github.com/bardisty!)
+ - refactored into separate files, wip audio & video archiving
+ - v0.0.1 released
+ - Index links now work without nginx url rewrites, archive can now be hosted on github pages
 - added setup.sh script & docstrings & help commands
 - made Chromium the default instead of Google Chrome (yay free software)
- - added [env-variable](https://github.com/pirate/bookmark-archiver/pull/25) configuration (thanks to https://github.com/hannah98!)
+ - added [env-variable](https://github.com/pirate/ArchiveBox/pull/25) configuration (thanks to https://github.com/hannah98!)
 - renamed from **Pocket Archive Stream** -> **Bookmark Archiver**
- - added [Netscape-format](https://github.com/pirate/bookmark-archiver/pull/20) export support (thanks to https://github.com/ilvar!)
- - added [Pinboard-format](https://github.com/pirate/bookmark-archiver/pull/7) export support (thanks to https://github.com/sconeyard!)
+ - added [Netscape-format](https://github.com/pirate/ArchiveBox/pull/20) export support (thanks to https://github.com/ilvar!)
+ - added [Pinboard-format](https://github.com/pirate/ArchiveBox/pull/7) export support (thanks to https://github.com/sconeyard!)
 - front-page of HN, oops! apparently I have users to support now :grin:?
 - added Pocket-format export support
- - created Pocket Archive Stream 2017/05/05
+ - v0.0.0 released: created Pocket Archive Stream 2017/05/05
+ 
+ 
+ ## Donations
+ 
+ This project can really flourish with some more engineering effort, but unless it can support   
+ me financially I'm unlikely to be able to take it to the next level alone.  It's already pretty   
+ functional and robust, but it really deserves to be taken to the next level with a few more   
+ talented engineers.  If you want to help sponsor this project long-term or just say thanks or suggest changes, contact  
+ me at bookmark-archiver@sweeting.me.
+   
+ [Grants / Donations](https://github.com/pirate/ArchiveBox/blob/master/DONATE.md)
@@ -0,0 +1 @@
+bin/archivebox
@@ -1,523 +0,0 @@
-#!/usr/bin/env python3
-# Bookmark Archiver
-# Nick Sweeting 2017 | MIT License
-# https://github.com/pirate/bookmark-archiver
-
-import re
-import os
-import sys
-import json
-import time
-
-from datetime import datetime
-from subprocess import run, PIPE, DEVNULL
-
-__DESCRIPTION__ = 'Bookmark Archiver: Create a browsable html archive of a list of links.'
-__DOCUMENTATION__ = 'https://github.com/pirate/bookmark-archiver'
-
-### SETTINGS
-
-INDEX_TEMPLATE = 'index_template.html'
-
-# os.getenv('VARIABLE', 'DEFAULT') gets the value of environment
-# variable "VARIABLE" and if it is not set, sets it to 'DEFAULT'
-
-# for boolean values, check to see if the string is 'true', and
-# if so, the python variable will be True
-
-FETCH_WGET =             os.getenv('FETCH_WGET',             'True'             ).lower() == 'true'
-FETCH_WGET_REQUISITES =  os.getenv('FETCH_WGET_REQUISITES',  'True'             ).lower() == 'true'
-FETCH_PDF =              os.getenv('FETCH_PDF',              'True'             ).lower() == 'true'
-FETCH_SCREENSHOT =       os.getenv('FETCH_SCREENSHOT',       'True'             ).lower() == 'true'
-FETCH_FAVICON =          os.getenv('FETCH_FAVICON',          'True'             ).lower() == 'true'
-SUBMIT_ARCHIVE_DOT_ORG = os.getenv('SUBMIT_ARCHIVE_DOT_ORG', 'True'             ).lower() == 'true'
-RESOLUTION =             os.getenv('RESOLUTION',             '1440,900'         )
-ARCHIVE_PERMISSIONS =    os.getenv('ARCHIVE_PERMISSIONS',    '755'              )
-CHROME_BINARY =          os.getenv('CHROME_BINARY',          'chromium-browser' )  # change to google-chrome browser if using google-chrome
-WGET_BINARY =            os.getenv('WGET_BINARY',            'wget'             )
-TIMEOUT =                int(os.getenv('TIMEOUT',            '60'))
-
-
-def check_dependencies():
-    print('[*] Checking Dependencies:')
-    if FETCH_PDF or FETCH_SCREENSHOT:
-        if run(['which', CHROME_BINARY]).returncode:
-            print('[X] Missing dependency: {}'.format(CHROME_BINARY))
-            print('    See https://github.com/pirate/bookmark-archiver for help.')
-            raise SystemExit(1)
-
-        # parse chrome --version e.g. Google Chrome 61.0.3114.0 canary / Chromium 59.0.3029.110 built on Ubuntu, running on Ubuntu 16.04
-        result = run([CHROME_BINARY, '--version'], stdout=PIPE)
-        version = result.stdout.decode('utf-8').replace('Google Chrome ', '').replace('Chromium ', '').split(' ', 1)[0].split('.', 1)[0]  # TODO: regex might be better
-        if int(version) < 59:
-            print('[X] Chrome version must be 59 or greater for headless PDF and screenshot saving')
-            print('    See https://github.com/pirate/bookmark-archiver for help.')
-            raise SystemExit(1)
-
-    if FETCH_WGET:
-        if run(['which', 'wget']).returncode:
-            print('[X] Missing dependency: wget')
-            print('    See https://github.com/pirate/bookmark-archiver for help.')
-            raise SystemExit(1)
-
-    if FETCH_FAVICON or SUBMIT_ARCHIVE_DOT_ORG:
-        if run(['which', 'curl']).returncode:
-            print('[X] Missing dependency: curl')
-            print('    See https://github.com/pirate/bookmark-archiver for help.')
-            raise SystemExit(1)
-
-
-### PARSING READER LIST EXPORTS
-
-def get_link_type(link):
-    """Certain types of links need to be handled specially, this figures out when that's the case"""
-
-    if link['base_url'].endswith('.pdf'):
-        return 'PDF'
-    elif link['base_url'].rsplit('.', 1) in ('pdf', 'png', 'jpg', 'jpeg', 'svg', 'bmp', 'gif', 'tiff', 'webp'):
-        return 'image'
-    elif 'wikipedia.org' in link['domain']:
-        return 'wiki'
-    elif 'youtube.com' in link['domain']:
-        return 'youtube'
-    return None
-
-def parse_pocket_export(html_file):
-    """Parse Pocket-format bookmarks export files (produced by getpocket.com/export/)"""
-
-    html_file.seek(0)
-    pattern = re.compile("^\\s*<li><a href=\"(.+)\" time_added=\"(\\d+)\" tags=\"(.*)\">(.+)</a></li>", re.UNICODE)   # see sample input in ./example_ril_export.html
-    for line in html_file:
-        match = pattern.search(line)
-        if match:
-            fixed_url = match.group(1).replace('http://www.readability.com/read?url=', '')           # remove old readability prefixes to get original url
-            without_scheme = fixed_url.replace('http://', '').replace('https://', '')
-            info = {
-                'url': fixed_url,
-                'domain': without_scheme.split('/')[0],    # without pathname
-                'base_url': without_scheme.split('?')[0],  # without query args
-                'time': datetime.fromtimestamp(int(match.group(2))).strftime('%Y-%m-%d %H:%M'),
-                'timestamp': match.group(2),
-                'tags': match.group(3),
-                'title': match.group(4).replace(' — Readability', '').replace('http://www.readability.com/read?url=', '') or without_scheme,
-            }
-            info['type'] = get_link_type(info)
-            yield info
-
-def parse_json_export(json_file):
-    """Parse JSON-format bookmarks export files (produced by pinboard.in/export/)"""
-
-    json_file.seek(0)
-    json_content = json.load(json_file)
-    for line in json_content:
-        if line:
-            erg = line
-            info = {
-                'url': erg['href'],
-                'domain': erg['href'].replace('http://', '').replace('https://', '').split('/')[0],
-                'base_url': erg['href'].replace('https://', '').replace('http://', '').split('?')[0],
-                'time': datetime.fromtimestamp(int(time.mktime(time.strptime(erg['time'].split(',')[0], '%Y-%m-%dT%H:%M:%SZ')))),
-                'timestamp': str(int(time.mktime(time.strptime(erg['time'].split(',')[0], '%Y-%m-%dT%H:%M:%SZ')))),
-                'tags': erg['tags'],
-                'title': erg['description'].replace(' — Readability', ''),
-            }
-            info['type'] = get_link_type(info)
-            yield info
-
-def parse_bookmarks_export(html_file):
-    """Parse netscape-format bookmarks export files (produced by all browsers)"""
-
-    html_file.seek(0)
-    pattern = re.compile("<a href=\"(.+?)\" add_date=\"(\\d+)\"[^>]*>(.+)</a>", re.UNICODE | re.IGNORECASE)
-    for line in html_file:
-        match = pattern.search(line)
-        if match:
-            url = match.group(1)
-            secs = match.group(2)
-            dt = datetime.fromtimestamp(int(secs))
-
-            info = {
-                'url': url,
-                'domain': url.replace('http://', '').replace('https://', '').split('/')[0],
-                'base_url': url.replace('https://', '').replace('http://', '').split('?')[0],
-                'time': dt,
-                'timestamp': secs,
-                'tags': "",
-                'title': match.group(3),
-            }
-
-            info['type'] = get_link_type(info)
-            yield info
-
-
-### ACHIVING FUNCTIONS
-
-def chmod_file(path, cwd='.', permissions=ARCHIVE_PERMISSIONS):
-    if not os.path.exists(os.path.join(cwd, path)):
-        raise Exception('Failed to chmod: {} does not exist (did the previous step fail?)'.format(path))
-
-    chmod_result = run(['chmod', '-R', ARCHIVE_PERMISSIONS, path], cwd=cwd, stdout=DEVNULL, stderr=PIPE, timeout=5)
-    if chmod_result.returncode == 1:
-        print('     ', chmod_result.stderr.decode())
-        raise Exception('Failed to chmod {}/{}'.format(cwd, path))
-
-def fetch_wget(out_dir, link, overwrite=False):
-    """download full site using wget"""
-
-    domain = link['base_url'].split('/', 1)[0]
-    if not os.path.exists('{}/{}'.format(out_dir, domain)) or overwrite:
-        print('    - Downloading Full Site')
-        CMD = [
-            *'wget --timestamping --adjust-extension --no-parent'.split(' '),
-            *(('--page-requisites', '--convert-links') if FETCH_WGET_REQUISITES else ()),
-            link['url'],
-        ]
-        try:
-            result = run(CMD, stdout=PIPE, stderr=PIPE, cwd=out_dir, timeout=TIMEOUT)  # dom.html
-            if result.returncode > 0:
-                print('     ', result.stderr.decode().split('\n')[-1])
-                raise Exception('Failed to wget download')
-            chmod_file(domain, cwd=out_dir)
-        except Exception as e:
-            print('       Run to see full output:', 'cd {}; {}'.format(out_dir, ' '.join(CMD)))
-            print('       Failed: {} {}'.format(e.__class__.__name__, e))
-    else:
-        print('    √ Skipping site download')
-
-def fetch_pdf(out_dir, link, overwrite=False):
-    """print PDF of site to file using chrome --headless"""
-
-    if (not os.path.exists('{}/output.pdf'.format(out_dir)) or overwrite) and link['type'] not in ('PDF', 'image'):
-        print('    - Printing PDF')
-        CMD = [
-            CHROME_BINARY,
-            *'--headless --disable-gpu --print-to-pdf'.split(' '),
-            link['url']
-        ]
-        try:
-            result = run(CMD, stdout=DEVNULL, stderr=PIPE, cwd=out_dir, timeout=TIMEOUT)  # output.pdf
-            if result.returncode:
-                print('     ', result.stderr.decode())
-                raise Exception('Failed to print PDF')
-            chmod_file('output.pdf', cwd=out_dir)
-        except Exception as e:
-            print('       Run to see full output:', 'cd {}; {}'.format(out_dir, ' '.join(CMD)))
-            print('       Failed: {} {}'.format(e.__class__.__name__, e))
-    else:
-        print('    √ Skipping PDF print')
-
-def fetch_screenshot(out_dir, link, overwrite=False):
-    """take screenshot of site using chrome --headless"""
-
-    if (not os.path.exists('{}/screenshot.png'.format(out_dir)) or overwrite) and link['type'] not in ('PDF', 'image'):
-        print('    - Snapping Screenshot')
-        CMD = [
-            CHROME_BINARY,
-            *'--headless --disable-gpu --screenshot'.split(' '),
-            '--window-size={}'.format(RESOLUTION),
-            link['url']
-        ]
-        try:
-            result = run(CMD, stdout=DEVNULL, stderr=DEVNULL, cwd=out_dir, timeout=TIMEOUT)  # sreenshot.png
-            if result.returncode:
-                print('     ', result.stderr.decode())
-                raise Exception('Failed to take screenshot')
-            chmod_file('screenshot.png', cwd=out_dir)
-        except Exception as e:
-            print('       Run to see full output:', 'cd {}; {}'.format(out_dir, ' '.join(CMD)))
-            print('       Failed: {} {}'.format(e.__class__.__name__, e))
-    else:
-        print('    √ Skipping screenshot')
-
-def archive_dot_org(out_dir, link, overwrite=False):
-    """submit site to archive.org for archiving via their service, save returned archive url"""
-    if (not os.path.exists('{}/archive.org.txt'.format(out_dir)) or overwrite):
-        print('    - Submitting to archive.org')
-        submit_url = 'https://web.archive.org/save/{}'.format(link['url'].split('?', 1)[0])
-
-        success = False
-        CMD = ['curl', '-I', submit_url]
-        try:
-            result = run(CMD, stdout=PIPE, stderr=DEVNULL, cwd=out_dir, timeout=TIMEOUT)  # archive.org
-            headers = result.stdout.splitlines()
-            content_location = [h for h in headers if b'Content-Location: ' in h]
-            if content_location:
-                archive_path = content_location[0].split(b'Content-Location: ', 1)[-1].decode('utf-8')
-                saved_url = 'https://web.archive.org{}'.format(archive_path)
-                success = True
-            else:
-                raise Exception('Failed to find "Content-Location" URL header in Archive.org response.')
-        except Exception as e:
-            print('       Visit url to see output:', ' '.join(CMD))
-            print('       Failed: {} {}'.format(e.__class__.__name__, e))
-
-        if success:
-            with open('{}/archive.org.txt'.format(out_dir), 'w') as f:
-                f.write(saved_url)
-            chmod_file('archive.org.txt', cwd=out_dir)
-
-    else:
-        print('    √ Skipping archive.org')
-
-def fetch_favicon(out_dir, link, overwrite=False):
-    """download site favicon from google's favicon api"""
-
-    if not os.path.exists('{}/favicon.ico'.format(out_dir)) or overwrite:
-        print('    - Fetching Favicon')
-        CMD = 'curl https://www.google.com/s2/favicons?domain={domain}'.format(**link).split(' ')
-        fout = open('{}/favicon.ico'.format(out_dir), 'w')
-        try:
-            run([*CMD], stdout=fout, stderr=DEVNULL, cwd=out_dir, timeout=TIMEOUT)  # favicon.ico
-            chmod_file('favicon.ico', cwd=out_dir)
-        except Exception as e:
-            print('       Run to see full output:', ' '.join(CMD))
-            print('       Failed: {} {}'.format(e.__class__.__name__, e))
-        fout.close()
-    else:
-        print('    √ Skipping favicon')
-
-
-### ORCHESTRATION
-
-def next_uniq_timestamp(used_timestamps, timestamp):
-    """resolve duplicate timestamps by appending a decimal 1234, 1234 -> 1234.1, 1234.2"""
-
-    if timestamp not in used_timestamps:
-        return timestamp
-
-    if '.' in timestamp:
-        timestamp, nonce = timestamp.split('.')
-        nonce = int(nonce)
-    else:
-        nonce = 1
-
-    new_timestamp = '{}.{}'.format(timestamp, nonce)
-
-    while new_timestamp in used_timestamps:
-        nonce += 1
-        new_timestamp = '{}.{}'.format(timestamp, nonce)
-
-    return new_timestamp
-
-def uniquefied_links(links):
-    """uniqueify link timestamps by de-duping using url, returns links sorted most recent -> oldest
-
-    needed because firefox will produce exports where many links share the same timestamp, this func
-    ensures that all non-duplicate links have monotonically increasing timestamps
-    """
-
-    links = list(reversed(sorted(links, key=lambda l: (l['timestamp'], l['url']))))
-    seen_timestamps = {}
-
-    for link in links:
-        t = link['timestamp']
-        if t in seen_timestamps:
-            if link['url'] == seen_timestamps[t]['url']:
-                # don't create new unique timestamp if link is the same
-                continue
-            else:
-                # resolve duplicate timstamp by appending a decimal
-                link['timestamp'] = next_uniq_timestamp(seen_timestamps, link['timestamp'])
-        seen_timestamps[link['timestamp']] = link
-
-    return links
-
-def valid_links(links):
-    """remove chrome://, about:// or other schemed links that cant be archived"""
-    return (link for link in links if link['url'].startswith('http') or link['url'].startswith('ftp'))
-
-def calculate_archive_url(link):
-    """calculate the path to the wgetted html file, since wget may adjust some paths
-    to be different than the base_url path
-    See docs on wget --adjust-extension."""
-
-    split_url = link['url'].split('#', 1)
-
-    if re.search(".+\\.[Hh][Tt][Mm][Ll]?$", split_url[0], re.I | re.M):
-        # already ends in .html
-        return link['base_url']
-    else:
-        # .html needs to be appended
-        without_scheme = split_url[0].split('://', 1)[-1]
-        if without_scheme.endswith('/'):
-            return '#'.join([without_scheme + 'index.html', *split_url[1:]])
-        return '#'.join([without_scheme + '.html', *split_url[1:]])
-
-
-
-def dump_index(links, service):
-    """create index.html file for a given list of links and service"""
-
-    with open(INDEX_TEMPLATE, 'r') as f:
-        index_html = f.read()
-
-    # TODO: refactor this out into index_template.html
-    link_html = """\
-    <tr>
-        <td>{time}</td>
-        <td><a href="archive/{timestamp}/{archive_url}" style="font-size:1.4em;text-decoration:none;color:black;" title="{title}">
-            <img src="archive/{timestamp}/favicon.ico">
-            {title} <small style="background-color: #eee;border-radius:4px; float:right">{tags}</small>
-        </td>
-        <td style="text-align:center"><a href="archive/{timestamp}/" title="Files">📂</a></td>
-        <td style="text-align:center"><a href="{pdf_link}" title="PDF">📄</a></td>
-        <td style="text-align:center"><a href="{screenshot_link}" title="Screenshot">🖼</a></td>
-        <td style="text-align:center"><a href="https://web.archive.org/web/{base_url}" title="Archive.org">🏛</a></td>
-        <td>🔗 <img src="https://www.google.com/s2/favicons?domain={domain}" height="16px"> <a href="{url}">{url}</a></td>
-    </tr>"""
-
-    def get_template_vars(link):
-        # since we dont screenshot or PDF links that are images or PDFs, change those links to point to the wget'ed file
-        link_info = {**link}
-
-        # append .html to archive links that dont have it, since wget appends .html to everything
-        link_info['archive_url'] = calculate_archive_url(link)
-
-        # add link type to title
-        if link['type']:
-            link_info.update({'title': '{title} ({type})'.format(**link)})
-
-        # PDF and images link to wgetted version, since we dont re-screenshot/pdf them
-        if link['type'] in ('PDF', 'image'):
-            link_info.update({
-                'pdf_link': 'archive/{timestamp}/{base_url}'.format(**link),
-                'screenshot_link': 'archive/{timestamp}/{base_url}'.format(**link),
-            })
-        else:
-            link_info.update({
-                'pdf_link': 'archive/{timestamp}/output.pdf'.format(**link),
-                'screenshot_link': 'archive/{timestamp}/screenshot.png'.format(**link)
-            })
-        return link_info
-
-    article_rows = '\n'.join(
-        link_html.format(**get_template_vars(link)) for link in links
-    )
-
-    template_vars = (datetime.now().strftime('%Y-%m-%d %H:%M'), article_rows)
-
-    with open(''.join((service, '/index.html')), 'w') as f:
-        f.write(index_html.format(*template_vars))
-
-
-def dump_website(link, service, overwrite=False):
-    """download the DOM, PDF, and a screenshot into a folder named after the link's timestamp"""
-
-    print('[+] [{timestamp} ({time})] "{title}": {base_url}'.format(**link))
-
-    out_dir = ''.join((service, '/archive/{timestamp}')).format(**link)
-    if not os.path.exists(out_dir):
-        os.makedirs(out_dir)
-
-    run(['chmod', ARCHIVE_PERMISSIONS, out_dir], timeout=5)
-
-    if link['type']:
-        print('    i Type: {}'.format(link['type']))
-
-    if not (link['url'].startswith('http') or link['url'].startswith('ftp')):
-        print('    X Skipping: invalid link.')
-        return
-
-    if FETCH_WGET:
-        fetch_wget(out_dir, link, overwrite=overwrite)
-
-    if FETCH_PDF:
-        fetch_pdf(out_dir, link, overwrite=overwrite)
-
-    if FETCH_SCREENSHOT:
-        fetch_screenshot(out_dir, link, overwrite=overwrite)
-
-    if SUBMIT_ARCHIVE_DOT_ORG:
-        archive_dot_org(out_dir, link, overwrite=overwrite)
-
-    if FETCH_FAVICON:
-        fetch_favicon(out_dir, link, overwrite=overwrite)
-
-
-def create_archive(export_file, service=None, resume=None):
-    """update or create index.html and download archive of all links"""
-
-    with open(export_file, 'r', encoding='utf-8') as f:
-        print('[+] [{}] Starting archive from {} export file.'.format(
-            datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
-            export_file
-        ))
-
-        # if specific service was passed via command line
-        if service == "pocket":
-            links = parse_pocket_export(f)
-        elif service == "pinboard":
-            links = parse_json_export(f)
-        elif service == "bookmarks":
-            links = parse_bookmarks_export(f)
-        else:
-            # otherwise try all parsers until one works
-            try:
-                links = list(parse_json_export(f))
-                service = 'pinboard'
-            except Exception:
-                links = list(parse_pocket_export(f))
-                if links:
-                    service = 'pocket'
-                else:
-                    links = list(parse_bookmarks_export(f))
-                    service = 'bookmarks'
-
-        links = valid_links(links)              # remove chrome://, about:, mailto: etc.
-        links = uniquefied_links(links)         # fix duplicate timestamps, returns sorted list
-
-        if resume:
-            try:
-                links = [
-                    link
-                    for link in links
-                    if float(link['timestamp']) >= float(resume)
-                ]
-            except TypeError:
-                print('Resume value and all timestamp values must be valid numbers.')
-
-    if not links:
-        print('[X] No links found in {}, is it a {} export file?'.format(export_file, service))
-        raise SystemExit(1)
-
-    if not os.path.exists(service):
-        os.makedirs(service)
-
-    if not os.path.exists(''.join((service, '/archive'))):
-        os.makedirs(''.join((service, '/archive')))
-
-    dump_index(links, service)
-
-    run(['chmod', '-R', ARCHIVE_PERMISSIONS, service], timeout=30)
-
-    print('[*] [{}] Created archive index with {} links.'.format(
-        datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
-        len(links),
-    ))
-
-    check_dependencies()
-
-    for link in links:
-        dump_website(link, service)
-
-    print('[√] [{}] Archive update complete.'.format(datetime.now()))
-
-
-
-if __name__ == '__main__':
-    argc = len(sys.argv)
-
-    if argc < 2 or sys.argv[1] in ('-h', '--help', 'help'):
-        print(__DESCRIPTION__)
-        print("Documentation:     {}".format(__DOCUMENTATION__))
-        print("")
-        print("Usage:")
-        print("    ./archive.py ~/Downloads/bookmarks_export.html")
-        print("")
-        raise SystemExit(0)
-
-    export_file = sys.argv[1]                                       # path to export file
-    export_type = sys.argv[2] if argc > 2 else None                 # select export_type for file format select
-    resume_from = sys.argv[3] if argc > 3 else None                 # timestamp to resume dowloading from
-
-    create_archive(export_file, service=export_type, resume=resume_from)
@@ -0,0 +1,181 @@
+#!/usr/bin/env python3
+# ArchiveBox
+# Nick Sweeting 2017 | MIT License
+# https://github.com/pirate/ArchiveBox
+
+import os
+import sys
+
+from datetime import datetime
+from subprocess import run
+
+from parse import parse_links
+from links import (
+    new_links,
+    validate_links
+)
+from archive_methods import archive_links, _RESULTS_TOTALS
+from index import (
+    write_links_index,
+    write_link_index,
+    parse_json_links_index,
+    parse_json_link_index,
+)
+from config import (
+    ONLY_NEW,
+    OUTPUT_PERMISSIONS,
+    OUTPUT_DIR,
+    ANSI,
+    TIMEOUT,
+    GIT_SHA,
+)
+from util import (
+    download_url,
+    progress,
+    cleanup_archive,
+    pretty_path,
+    migrate_data,
+)
+
+__AUTHOR__ = 'Nick Sweeting <git@nicksweeting.com>'
+__VERSION__ = GIT_SHA
+__DESCRIPTION__ = 'ArchiveBox: Create a browsable html archive of a list of links.'
+__DOCUMENTATION__ = 'https://github.com/pirate/ArchiveBox'
+
+def print_help():
+    print(__DESCRIPTION__)
+    print("Documentation:     {}\n".format(__DOCUMENTATION__))
+    print("Usage:")
+    print("    ./bin/archivebox ~/Downloads/bookmarks_export.html\n")
+
+
+def merge_links(archive_path=OUTPUT_DIR, import_path=None, only_new=False):
+    """get new links from file and optionally append them to links in existing archive"""
+    all_links = []
+    if import_path:
+        # parse and validate the import file
+        raw_links = parse_links(import_path)
+        all_links = validate_links(raw_links)
+
+    # merge existing links in archive_path and new links
+    existing_links = []
+    if archive_path:
+        existing_links = parse_json_links_index(archive_path)
+        all_links = validate_links(existing_links + all_links)
+    
+    num_new_links = len(all_links) - len(existing_links)
+    if num_new_links and not only_new:
+        print('[{green}+{reset}] [{}] Adding {} new links from {} to {}/index.json'.format(
+            datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+            num_new_links,
+            pretty_path(import_path),
+            pretty_path(archive_path),
+            **ANSI,
+        ))
+    # else:
+    #     print('[*] [{}] No new links added to {}/index.json{}'.format(
+    #         datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+    #         archive_path,
+    #         ' from {}'.format(import_path) if import_path else '',
+    #         **ANSI,
+    #     ))
+
+    if only_new:
+        return new_links(all_links, existing_links)
+
+    return all_links
+
+def update_archive(archive_path, links, source=None, resume=None, append=True):
+    """update or create index.html+json given a path to an export file containing new links"""
+
+    start_ts = datetime.now().timestamp()
+
+    if resume:
+        print('{green}[▶] [{}] Resuming archive downloading from {}...{reset}'.format(
+             datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+             resume,
+             **ANSI,
+        ))
+    else:
+        print('{green}[▶] [{}] Updating files for {} links in archive...{reset}'.format(
+             datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+             len(links),
+             **ANSI,
+        ))
+
+    # loop over links and archive them
+    archive_links(archive_path, links, source=source, resume=resume)
+
+    # print timing information & summary
+    end_ts = datetime.now().timestamp()
+    seconds = end_ts - start_ts
+    if seconds > 60:
+        duration = '{0:.2f} min'.format(seconds / 60, 2)
+    else:
+        duration = '{0:.2f} sec'.format(seconds, 2)
+
+    print('{}[√] [{}] Update of {} links complete ({}){}'.format(
+        ANSI['green'],
+        datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+        len(links),
+        duration,
+        ANSI['reset'],
+    ))
+    print('    - {} entries skipped'.format(_RESULTS_TOTALS['skipped']))
+    print('    - {} entries updated'.format(_RESULTS_TOTALS['succeded']))
+    print('    - {} errors'.format(_RESULTS_TOTALS['failed']))
+
+
+if __name__ == '__main__':
+    argc = len(sys.argv)
+
+    if set(sys.argv).intersection(('-h', '--help', 'help')):
+        print_help()
+        raise SystemExit(0)
+
+    migrate_data()
+
+    source = sys.argv[1] if argc > 1 else None  # path of links file to import
+    resume = sys.argv[2] if argc > 2 else None  # timestamp to resume dowloading from
+   
+    if argc == 1:
+        source, resume = None, None
+    elif argc == 2:
+        if all(d.isdigit() for d in sys.argv[1].split('.')):
+            # argv[1] is a resume timestamp
+            source, resume = None, sys.argv[1]
+        else:
+            # argv[1] is a path to a file to import
+            source, resume = sys.argv[1].strip(), None
+    elif argc == 3:
+        source, resume = sys.argv[1].strip(), sys.argv[2]
+    else:
+        print_help()
+        raise SystemExit(1)
+
+    # See if archive folder already exists
+    for out_dir in (OUTPUT_DIR, 'bookmarks', 'pocket', 'pinboard', 'html'):
+        if os.path.exists(out_dir):
+            break
+    else:
+        out_dir = OUTPUT_DIR
+
+    # Step 0: Download url to local file (only happens if a URL is specified instead of local path) 
+    if source and any(source.startswith(s) for s in ('http://', 'https://', 'ftp://')):
+        source = download_url(source)
+
+    # Step 1: Parse the links and dedupe them with existing archive
+    links = merge_links(archive_path=out_dir, import_path=source, only_new=False)
+    new_links = merge_links(archive_path=out_dir, import_path=source, only_new=True)
+
+    # Step 2: Write new index
+    write_links_index(out_dir=out_dir, links=links)
+
+    # Step 3: Verify folder structure is 1:1 with index
+    # cleanup_archive(out_dir, links)
+
+    # Step 4: Run the archive methods for each link
+    if ONLY_NEW:
+        update_archive(out_dir, new_links, source=source, resume=resume, append=True)
+    else:
+        update_archive(out_dir, links, source=source, resume=resume, append=True)
@@ -0,0 +1,509 @@
+import os
+import sys
+
+from functools import wraps
+from collections import defaultdict
+from datetime import datetime
+from subprocess import run, PIPE, DEVNULL
+
+from peekable import Peekable
+
+from index import wget_output_path, parse_json_link_index, write_link_index
+from links import links_after_timestamp
+from config import (
+    CHROME_BINARY,
+    FETCH_WGET,
+    FETCH_WGET_REQUISITES,
+    FETCH_PDF,
+    FETCH_SCREENSHOT,
+    FETCH_DOM,
+    RESOLUTION,
+    CHECK_SSL_VALIDITY,
+    SUBMIT_ARCHIVE_DOT_ORG,
+    FETCH_AUDIO,
+    FETCH_VIDEO,
+    FETCH_FAVICON,
+    WGET_USER_AGENT,
+    CHROME_USER_DATA_DIR,
+    CHROME_SANDBOX,
+    TIMEOUT,
+    ANSI,
+    ARCHIVE_DIR,
+)
+from util import (
+    check_dependencies,
+    progress,
+    chmod_file,
+    pretty_path,
+)
+
+
+_RESULTS_TOTALS = {   # globals are bad, mmkay
+    'skipped': 0,
+    'succeded': 0,
+    'failed': 0,
+}
+
+def archive_links(archive_path, links, source=None, resume=None):
+    check_dependencies()
+
+    to_archive = Peekable(links_after_timestamp(links, resume))
+    idx, link = 0, to_archive.peek(0)
+
+    try:
+        for idx, link in enumerate(to_archive):
+            link_dir = os.path.join(ARCHIVE_DIR, link['timestamp'])
+            archive_link(link_dir, link)
+    
+    except (KeyboardInterrupt, SystemExit, Exception) as e:
+        print('{lightyellow}[X] [{now}] Downloading paused on link {timestamp} ({idx}/{total}){reset}'.format(
+            **ANSI,
+            now=datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+            idx=idx+1,
+            timestamp=link['timestamp'],
+            total=len(links),
+        ))
+        print('    Continue where you left off by running:')
+        print('        {} {}'.format(
+            pretty_path(sys.argv[0]),
+            link['timestamp'],
+        ))
+        if not isinstance(e, KeyboardInterrupt):
+            raise e
+        raise SystemExit(1)
+
+
+def archive_link(link_dir, link, overwrite=True):
+    """download the DOM, PDF, and a screenshot into a folder named after the link's timestamp"""
+
+    update_existing = os.path.exists(link_dir)
+    if update_existing:
+        link = {
+            **parse_json_link_index(link_dir),
+            **link,
+        }
+    else:
+        os.makedirs(link_dir)
+    
+    log_link_archive(link_dir, link, update_existing)
+
+    if FETCH_WGET:
+        link = fetch_wget(link_dir, link, overwrite=overwrite)
+
+    if FETCH_PDF:
+        link = fetch_pdf(link_dir, link, overwrite=overwrite)
+
+    if FETCH_SCREENSHOT:
+        link = fetch_screenshot(link_dir, link, overwrite=overwrite)
+
+    if FETCH_DOM:
+        link = fetch_dom(link_dir, link, overwrite=overwrite)
+
+    if SUBMIT_ARCHIVE_DOT_ORG:
+        link = archive_dot_org(link_dir, link, overwrite=overwrite)
+
+    # if FETCH_AUDIO:
+    #     link = fetch_audio(link_dir, link, overwrite=overwrite)
+
+    # if FETCH_VIDEO:
+    #     link = fetch_video(link_dir, link, overwrite=overwrite)
+
+    if FETCH_FAVICON:
+        link = fetch_favicon(link_dir, link, overwrite=overwrite)
+
+    write_link_index(link_dir, link)
+    # print()
+    
+    return link
+
+def log_link_archive(link_dir, link, update_existing):
+    print('[{symbol_color}{symbol}{reset}] [{now}] "{title}"\n    {blue}{url}{reset}'.format(
+        symbol='*' if update_existing else '+',
+        symbol_color=ANSI['black' if update_existing else 'green'],
+        now=datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+        **link,
+        **ANSI,
+    ))
+
+    print('    > {}{}'.format(pretty_path(link_dir), '' if update_existing else ' (new)'))
+    if link['type']:
+        print('      i {}'.format(link['type']))
+
+
+
+def attach_result_to_link(method):
+    """
+    Instead of returning a result={output:'...', status:'success'} object,
+    attach that result to the links's history & latest fields, then return
+    the updated link object.
+    """
+    def decorator(fetch_func):
+        @wraps(fetch_func)
+        def timed_fetch_func(link_dir, link, overwrite=False, **kwargs):
+            # initialize methods and history json field on link
+            link['latest'] = link.get('latest') or {}
+            link['latest'][method] = link['latest'].get(method) or None
+            link['history'] = link.get('history') or {}
+            link['history'][method] = link['history'].get(method) or []
+
+            start_ts = datetime.now().timestamp()
+
+            # if a valid method output is already present, dont run the fetch function
+            if link['latest'][method] and not overwrite:
+                print('      √ {}'.format(method))
+                result = None
+            else:
+                print('      > {}'.format(method))
+                result = fetch_func(link_dir, link, **kwargs)
+
+            end_ts = datetime.now().timestamp()
+            duration = str(end_ts * 1000 - start_ts * 1000).split('.')[0]
+
+            # append a history item recording fail/success
+            history_entry = {
+                'timestamp': str(start_ts).split('.')[0],
+            }
+            if result is None:
+                history_entry['status'] = 'skipped'
+            elif isinstance(result.get('output'), Exception):
+                history_entry['status'] = 'failed'
+                history_entry['duration'] = duration
+                history_entry.update(result or {})
+                link['history'][method].append(history_entry)
+            else:
+                history_entry['status'] = 'succeded'
+                history_entry['duration'] = duration
+                history_entry.update(result or {})
+                link['history'][method].append(history_entry)
+                link['latest'][method] = result['output']
+
+            _RESULTS_TOTALS[history_entry['status']] += 1
+            
+            return link
+        return timed_fetch_func
+    return decorator
+
+
+@attach_result_to_link('wget')
+def fetch_wget(link_dir, link, requisites=FETCH_WGET_REQUISITES, timeout=TIMEOUT):
+    """download full site using wget"""
+
+    domain_dir = os.path.join(link_dir, link['domain'])
+    existing_file = wget_output_path(link)
+    if os.path.exists(domain_dir) and existing_file:
+        return {'output': existing_file, 'status': 'skipped'}
+
+    CMD = [
+        # WGET CLI Docs: https://www.gnu.org/software/wget/manual/wget.html
+        *'wget -N -E -np -x -H -k -K -S --restrict-file-names=unix'.split(' '),
+        *(('-p',) if FETCH_WGET_REQUISITES else ()),
+        *(('--user-agent={}'.format(WGET_USER_AGENT),) if WGET_USER_AGENT else ()),
+        *((() if CHECK_SSL_VALIDITY else ('--no-check-certificate',))),
+        link['url'],
+    ]
+    end = progress(timeout, prefix='      ')
+    try:
+        result = run(CMD, stdout=PIPE, stderr=PIPE, cwd=link_dir, timeout=timeout + 1)  # index.html
+        end()
+        output = wget_output_path(link, look_in=domain_dir)
+
+        # Check for common failure cases
+        if result.returncode > 0:
+            print('        got wget response code {}:'.format(result.returncode))
+            if result.returncode != 8:
+                print('\n'.join('          ' + line for line in (result.stderr or result.stdout).decode().rsplit('\n', 10)[-10:] if line.strip()))
+            if b'403: Forbidden' in result.stderr:
+                raise Exception('403 Forbidden (try changing WGET_USER_AGENT)')
+            if b'404: Not Found' in result.stderr:
+                raise Exception('404 Not Found')
+            if b'ERROR 500: Internal Server Error' in result.stderr:
+                raise Exception('500 Internal Server Error')
+            if result.returncode == 4:
+                raise Exception('Failed wget download')
+    except Exception as e:
+        end()
+        print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+
+
+@attach_result_to_link('pdf')
+def fetch_pdf(link_dir, link, timeout=TIMEOUT, user_data_dir=CHROME_USER_DATA_DIR):
+    """print PDF of site to file using chrome --headless"""
+
+    if link['type'] in ('PDF', 'image'):
+        return {'output': wget_output_path(link)}
+    
+    if os.path.exists(os.path.join(link_dir, 'output.pdf')):
+        return {'output': 'output.pdf', 'status': 'skipped'}
+
+    CMD = [
+        *chrome_headless(user_data_dir=user_data_dir),
+        '--print-to-pdf',
+        link['url']
+    ]
+    end = progress(timeout, prefix='      ')
+    try:
+        result = run(CMD, stdout=PIPE, stderr=PIPE, cwd=link_dir, timeout=timeout + 1)  # output.pdf
+        end()
+        if result.returncode:
+            print('     ', (result.stderr or result.stdout).decode())
+            raise Exception('Failed to print PDF')
+        chmod_file('output.pdf', cwd=link_dir)
+        output = 'output.pdf'
+    except Exception as e:
+        end()
+        print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+
+@attach_result_to_link('screenshot')
+def fetch_screenshot(link_dir, link, timeout=TIMEOUT, user_data_dir=CHROME_USER_DATA_DIR, resolution=RESOLUTION):
+    """take screenshot of site using chrome --headless"""
+
+    if link['type'] in ('PDF', 'image'):
+        return {'output': wget_output_path(link)}
+    
+    if os.path.exists(os.path.join(link_dir, 'screenshot.png')):
+        return {'output': 'screenshot.png', 'status': 'skipped'}
+
+    CMD = [
+        *chrome_headless(user_data_dir=user_data_dir),
+        '--screenshot',
+        '--window-size={}'.format(resolution),
+        '--hide-scrollbars',
+        # '--full-page',   # TODO: make this actually work using ./bin/screenshot fullPage: true
+        link['url'],
+    ]
+    end = progress(timeout, prefix='      ')
+    try:
+        result = run(CMD, stdout=PIPE, stderr=PIPE, cwd=link_dir, timeout=timeout + 1)  # sreenshot.png
+        end()
+        if result.returncode:
+            print('     ', (result.stderr or result.stdout).decode())
+            raise Exception('Failed to take screenshot')
+        chmod_file('screenshot.png', cwd=link_dir)
+        output = 'screenshot.png'
+    except Exception as e:
+        end()
+        print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+    
+@attach_result_to_link('dom')
+def fetch_dom(link_dir, link, timeout=TIMEOUT, user_data_dir=CHROME_USER_DATA_DIR):
+    """print HTML of site to file using chrome --dump-html"""
+
+    if link['type'] in ('PDF', 'image'):
+        return {'output': wget_output_path(link)}
+    
+    output_path = os.path.join(link_dir, 'output.html')
+
+    if os.path.exists(output_path):
+        return {'output': 'output.html', 'status': 'skipped'}
+
+    CMD = [
+        *chrome_headless(user_data_dir=user_data_dir),
+        '--dump-dom',
+        link['url']
+    ]
+    end = progress(timeout, prefix='      ')
+    try:
+        with open(output_path, 'w+') as f:
+            result = run(CMD, stdout=f, stderr=PIPE, cwd=link_dir, timeout=timeout + 1)  # output.html
+        end()
+        if result.returncode:
+            print('     ', (result.stderr).decode())
+            raise Exception('Failed to fetch DOM')
+        chmod_file('output.html', cwd=link_dir)
+        output = 'output.html'
+    except Exception as e:
+        end()
+        print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+
+@attach_result_to_link('archive_org')
+def archive_dot_org(link_dir, link, timeout=TIMEOUT):
+    """submit site to archive.org for archiving via their service, save returned archive url"""
+
+    path = os.path.join(link_dir, 'archive.org.txt')
+    if os.path.exists(path):
+        archive_org_url = open(path, 'r').read().strip()
+        return {'output': archive_org_url, 'status': 'skipped'}
+
+    submit_url = 'https://web.archive.org/save/{}'.format(link['url'])
+
+    success = False
+    CMD = ['curl', '-L', '-I', '-X', 'GET', submit_url]
+    end = progress(timeout, prefix='      ')
+    try:
+        result = run(CMD, stdout=PIPE, stderr=DEVNULL, cwd=link_dir, timeout=timeout + 1)  # archive.org.txt
+        end()
+
+        # Parse archive.org response headers
+        headers = defaultdict(list)
+
+        # lowercase all the header names and store in dict
+        for header in result.stdout.splitlines():
+            if b':' not in header or not header.strip():
+                continue
+            name, val = header.decode().split(':', 1)
+            headers[name.lower().strip()].append(val.strip())
+
+        # Get successful archive url in "content-location" header or any errors
+        content_location = headers['content-location']
+        errors = headers['x-archive-wayback-runtime-error']
+
+        if content_location:
+            saved_url = 'https://web.archive.org{}'.format(content_location[0])
+            success = True
+        elif len(errors) == 1 and 'RobotAccessControlException' in errors[0]:
+            output = submit_url
+            # raise Exception('Archive.org denied by {}/robots.txt'.format(link['domain']))
+        elif errors:
+            raise Exception(', '.join(errors))
+        else:
+            raise Exception('Failed to find "content-location" URL header in Archive.org response.')
+    except Exception as e:
+        end()
+        print('        Visit url to see output:', ' '.join(CMD))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    if success:
+        with open(os.path.join(link_dir, 'archive.org.txt'), 'w', encoding='utf-8') as f:
+            f.write(saved_url)
+        chmod_file('archive.org.txt', cwd=link_dir)
+        output = saved_url
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+
+@attach_result_to_link('favicon')
+def fetch_favicon(link_dir, link, timeout=TIMEOUT):
+    """download site favicon from google's favicon api"""
+
+    if os.path.exists(os.path.join(link_dir, 'favicon.ico')):
+        return {'output': 'favicon.ico', 'status': 'skipped'}
+
+    CMD = ['curl', 'https://www.google.com/s2/favicons?domain={domain}'.format(**link)]
+    fout = open('{}/favicon.ico'.format(link_dir), 'w')
+    end = progress(timeout, prefix='      ')
+    try:
+        run(CMD, stdout=fout, stderr=DEVNULL, cwd=link_dir, timeout=timeout + 1)  # favicon.ico
+        fout.close()
+        end()
+        chmod_file('favicon.ico', cwd=link_dir)
+        output = 'favicon.ico'
+    except Exception as e:
+        fout.close()
+        end()
+        print('        Run to see full output:', ' '.join(CMD))
+        print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+        output = e
+
+    return {
+        'cmd': CMD,
+        'output': output,
+    }
+
+# @attach_result_to_link('audio')
+# def fetch_audio(link_dir, link, timeout=TIMEOUT):
+#     """Download audio rip using youtube-dl"""
+
+#     if link['type'] not in ('soundcloud',)\
+#        and 'audio' not in link['tags']:
+#         return
+
+#     path = os.path.join(link_dir, 'audio')
+
+#     if not os.path.exists(path) or overwrite:
+#         print('    - Downloading audio')
+#         CMD = [
+#             "youtube-dl -x --audio-format mp3 --audio-quality 0 -o '%(title)s.%(ext)s'",
+#             link['url'],
+#         ]
+#         end = progress(timeout, prefix='      ')
+#         try:
+#             result = run(CMD, stdout=DEVNULL, stderr=DEVNULL, cwd=link_dir, timeout=timeout + 1)  # audio/audio.mp3
+#             end()
+#             if result.returncode:
+#                 print('     ', result.stderr.decode())
+#                 raise Exception('Failed to download audio')
+#             chmod_file('audio.mp3', cwd=link_dir)
+#             return 'audio.mp3'
+#         except Exception as e:
+#             end()
+#             print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+#             print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+#             raise
+#     else:
+#         print('    √ Skipping audio download')
+
+# @attach_result_to_link('video')
+# def fetch_video(link_dir, link, timeout=TIMEOUT):
+#     """Download video rip using youtube-dl"""
+
+#     if link['type'] not in ('youtube', 'youku', 'vimeo')\
+#        and 'video' not in link['tags']:
+#         return
+
+#     path = os.path.join(link_dir, 'video')
+
+#     if not os.path.exists(path) or overwrite:
+#         print('    - Downloading video')
+#         CMD = [
+#             "youtube-dl -x --video-format mp4 --audio-quality 0 -o '%(title)s.%(ext)s'",
+#             link['url'],
+#         ]
+#         end = progress(timeout, prefix='      ')
+#         try:
+#             result = run(CMD, stdout=DEVNULL, stderr=DEVNULL, cwd=link_dir, timeout=timeout + 1)  # video/movie.mp4
+#             end()
+#             if result.returncode:
+#                 print('     ', result.stderr.decode())
+#                 raise Exception('Failed to download video')
+#             chmod_file('video.mp4', cwd=link_dir)
+#             return 'video.mp4'
+#         except Exception as e:
+#             end()
+#             print('        Run to see full output:', 'cd {}; {}'.format(link_dir, ' '.join(CMD)))
+#             print('        {}Failed: {} {}{}'.format(ANSI['red'], e.__class__.__name__, e, ANSI['reset']))
+#             raise
+#     else:
+#         print('    √ Skipping video download')
+
+
+def chrome_headless(binary=CHROME_BINARY, user_data_dir=CHROME_USER_DATA_DIR):
+    args = [binary, '--headless']  # '--disable-gpu'
+    if not CHROME_SANDBOX:
+        args.append('--no-sandbox')
+    default_profile = os.path.expanduser('~/Library/Application Support/Google/Chrome/Default')
+    if user_data_dir:
+        args.append('--user-data-dir={}'.format(user_data_dir))
+    elif os.path.exists(default_profile):
+        args.append('--user-data-dir={}'.format(default_profile))
+    return args
@@ -0,0 +1,84 @@
+import os
+import sys
+import shutil
+
+from subprocess import run, PIPE
+
+# ******************************************************************************
+# * TO SET YOUR CONFIGURATION, EDIT THE VALUES BELOW, or use the 'env' command *
+# * e.g.                                                                       *
+# * env USE_COLOR=True CHROME_BINARY=google-chrome ./archive.py export.html    *
+# ******************************************************************************
+
+IS_TTY = sys.stdout.isatty()
+USE_COLOR =              os.getenv('USE_COLOR',              str(IS_TTY)        ).lower() == 'true'
+SHOW_PROGRESS =          os.getenv('SHOW_PROGRESS',          str(IS_TTY)        ).lower() == 'true'
+ONLY_NEW =               os.getenv('ONLY_NEW',               'False'            ).lower() == 'true'
+FETCH_WGET =             os.getenv('FETCH_WGET',             'True'             ).lower() == 'true'
+FETCH_WGET_REQUISITES =  os.getenv('FETCH_WGET_REQUISITES',  'True'             ).lower() == 'true'
+FETCH_AUDIO =            os.getenv('FETCH_AUDIO',            'False'            ).lower() == 'true'
+FETCH_VIDEO =            os.getenv('FETCH_VIDEO',            'False'            ).lower() == 'true'
+FETCH_PDF =              os.getenv('FETCH_PDF',              'True'             ).lower() == 'true'
+FETCH_SCREENSHOT =       os.getenv('FETCH_SCREENSHOT',       'True'             ).lower() == 'true'
+FETCH_DOM =              os.getenv('FETCH_DOM',              'True'             ).lower() == 'true'
+FETCH_FAVICON =          os.getenv('FETCH_FAVICON',          'True'             ).lower() == 'true'
+SUBMIT_ARCHIVE_DOT_ORG = os.getenv('SUBMIT_ARCHIVE_DOT_ORG', 'True'             ).lower() == 'true'
+RESOLUTION =             os.getenv('RESOLUTION',             '1440,1200'        )
+CHECK_SSL_VALIDITY =     os.getenv('CHECK_SSL_VALIDITY',     'True'             ).lower() == 'true'
+OUTPUT_PERMISSIONS =     os.getenv('OUTPUT_PERMISSIONS',     '755'              )
+CHROME_BINARY =          os.getenv('CHROME_BINARY',          'chromium-browser' )  # change to google-chrome browser if using google-chrome
+WGET_BINARY =            os.getenv('WGET_BINARY',            'wget'             )
+WGET_USER_AGENT =        os.getenv('WGET_USER_AGENT',        'ArchiveBox')
+CHROME_USER_DATA_DIR =   os.getenv('CHROME_USER_DATA_DIR',    None)
+TIMEOUT =                int(os.getenv('TIMEOUT',            '60'))
+FOOTER_INFO =            os.getenv('FOOTER_INFO',            'Content is hosted for personal archiving purposes only.  Contact server owner for any takedown requests.',)
+
+### Paths
+REPO_DIR = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..'))
+
+OUTPUT_DIR = os.getenv('OUTPUT_DIR', os.path.join(REPO_DIR, 'output'))
+ARCHIVE_DIR = os.path.join(OUTPUT_DIR, 'archive')
+SOURCES_DIR = os.path.join(OUTPUT_DIR, 'sources')
+
+PYTHON_PATH = os.path.join(REPO_DIR, 'archiver')
+TEMPLATES_DIR = os.path.join(PYTHON_PATH, 'templates')
+
+# ******************************************************************************
+# ********************** Do not edit below this point **************************
+# ******************************************************************************
+
+CHROME_SANDBOX =        os.getenv('CHROME_SANDBOX',         'True'             ).lower() == 'true'
+
+### Terminal Configuration
+TERM_WIDTH = shutil.get_terminal_size((100, 10)).columns
+ANSI = {
+    'reset': '\033[00;00m',
+    'lightblue': '\033[01;30m',
+    'lightyellow': '\033[01;33m',
+    'lightred': '\033[01;35m',
+    'red': '\033[01;31m',
+    'green': '\033[01;32m',
+    'blue': '\033[01;34m',
+    'white': '\033[01;37m',
+    'black': '\033[01;30m',
+}
+if not USE_COLOR:
+    # dont show colors if USE_COLOR is False
+    ANSI = {k: '' for k in ANSI.keys()}
+
+### Confirm Environment Setup
+try:
+    GIT_SHA = run(["git", "rev-list", "-1", "HEAD", "./"], stdout=PIPE, cwd=REPO_DIR).stdout.strip().decode()
+except Exception:
+    GIT_SHA = 'unknown'
+    print('[!] Warning, you need git installed for some archiving features to save correct version numbers!')
+
+if sys.stdout.encoding.upper() != 'UTF-8':
+    print('[X] Your system is running python3 scripts with a bad locale setting: {} (it should be UTF-8).'.format(sys.stdout.encoding))
+    print('    To fix it, add the line "export PYTHONIOENCODING=UTF-8" to your ~/.bashrc file (without quotes)')
+    print('')
+    print('    Confirm that it\'s fixed by opening a new shell and running:')
+    print('        python3 -c "import sys; print(sys.stdout.encoding)"   # should output UTF-8')
+    print('')
+    print('    Alternatively, run this script with:')
+    print('        env PYTHONIOENCODING=UTF-8 ./archive.py export.html')
@@ -0,0 +1,154 @@
+import os
+import json
+
+from datetime import datetime
+from string import Template
+from distutils.dir_util import copy_tree
+
+from config import (
+    TEMPLATES_DIR,
+    OUTPUT_PERMISSIONS,
+    ANSI,
+    GIT_SHA,
+    FOOTER_INFO,
+)
+from util import (
+    chmod_file,
+    wget_output_path,
+    derived_link_info,
+    pretty_path,
+)
+
+
+### Homepage index for all the links
+
+def write_links_index(out_dir, links):
+    """create index.html file for a given list of links"""
+
+    if not os.path.exists(out_dir):
+        os.makedirs(out_dir)
+
+    write_json_links_index(out_dir, links)
+    write_html_links_index(out_dir, links)
+    
+    print('{green}[√] [{}] Updated main index files:{reset}'.format(
+        datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+        **ANSI))
+    print('    > {}/index.json'.format(pretty_path(out_dir)))
+    print('    > {}/index.html'.format(pretty_path(out_dir)))
+
+def write_json_links_index(out_dir, links):
+    """write the json link index to a given path"""
+
+    path = os.path.join(out_dir, 'index.json')
+
+    index_json = {
+        'info': 'ArchiveBox Index',
+        'help': 'https://github.com/pirate/ArchiveBox',
+        'version': GIT_SHA,
+        'num_links': len(links),
+        'updated': str(datetime.now().timestamp()),
+        'links': links,
+    }
+
+    with open(path, 'w', encoding='utf-8') as f:
+        json.dump(index_json, f, indent=4, default=str)
+
+    chmod_file(path)
+
+def parse_json_links_index(out_dir):
+    """load the index in a given directory and merge it with the given link"""
+    index_path = os.path.join(out_dir, 'index.json')
+    if os.path.exists(index_path):
+        with open(index_path, 'r', encoding='utf-8') as f:
+            return json.load(f)['links']
+
+    return []
+
+def write_html_links_index(out_dir, links):
+    """write the html link index to a given path"""
+
+    path = os.path.join(out_dir, 'index.html')
+
+    copy_tree(os.path.join(TEMPLATES_DIR, 'static'), os.path.join(out_dir, 'static'))
+
+    with open(os.path.join(out_dir, 'robots.txt'), 'w+') as f:
+        f.write('User-agent: *\nDisallow: /')
+
+    with open(os.path.join(TEMPLATES_DIR, 'index.html'), 'r', encoding='utf-8') as f:
+        index_html = f.read()
+
+    with open(os.path.join(TEMPLATES_DIR, 'index_row.html'), 'r', encoding='utf-8') as f:
+        link_row_html = f.read()
+
+    link_rows = '\n'.join(
+        Template(link_row_html).substitute(**derived_link_info(link))
+        for link in links
+    )
+
+    template_vars = {
+        'num_links': len(links),
+        'date_updated': datetime.now().strftime('%Y-%m-%d'),
+        'time_updated': datetime.now().strftime('%Y-%m-%d %H:%M'),
+        'footer_info': FOOTER_INFO,
+        'git_sha': GIT_SHA,
+        'short_git_sha': GIT_SHA[:8],
+        'rows': link_rows,
+    }
+
+    with open(path, 'w', encoding='utf-8') as f:
+        f.write(Template(index_html).substitute(**template_vars))
+
+    chmod_file(path)
+
+
+### Individual link index
+
+def write_link_index(out_dir, link):
+    link['updated'] = str(datetime.now().timestamp())
+    write_json_link_index(out_dir, link)
+    write_html_link_index(out_dir, link)
+
+def write_json_link_index(out_dir, link):
+    """write a json file with some info about the link"""
+    
+    path = os.path.join(out_dir, 'index.json')
+
+    print('      √ index.json')
+
+    with open(path, 'w', encoding='utf-8') as f:
+        json.dump(link, f, indent=4, default=str)
+
+    chmod_file(path)
+
+def parse_json_link_index(out_dir):
+    """load the json link index from a given directory"""
+    existing_index = os.path.join(out_dir, 'index.json')
+    if os.path.exists(existing_index):
+        with open(existing_index, 'r', encoding='utf-8') as f:
+            return json.load(f)
+    return {}
+
+def write_html_link_index(out_dir, link):
+    with open(os.path.join(TEMPLATES_DIR, 'link_index_fancy.html'), 'r', encoding='utf-8') as f:
+        link_html = f.read()
+
+    path = os.path.join(out_dir, 'index.html')
+
+    print('      √ index.html')
+
+    with open(path, 'w', encoding='utf-8') as f:
+        f.write(Template(link_html).substitute({
+            **link,
+            **link['latest'],
+            'type': link['type'] or 'website',
+            'tags': link['tags'] or 'untagged',
+            'bookmarked': datetime.fromtimestamp(float(link['timestamp'])).strftime('%Y-%m-%d %H:%M'),
+            'updated': datetime.fromtimestamp(float(link['updated'])).strftime('%Y-%m-%d %H:%M'),
+            'bookmarked_ts': link['timestamp'],
+            'updated_ts': link['updated'],
+            'archive_org': link['latest'].get('archive_org') or 'https://web.archive.org/save/{}'.format(link['url']),
+            'wget': link['latest'].get('wget') or wget_output_path(link),
+        }))
+
+    chmod_file(path)
@@ -0,0 +1,154 @@
+"""
+In ArchiveBox, a Link represents a single entry that we track in the 
+json index.  All links pass through all archiver functions and the latest,
+most up-to-date canonical output for each is stored in "latest".
+
+
+Link {
+    timestamp: str,     (how we uniquely id links)        _   _  _ _  ___
+    url: str,                                            | \ / \ |\| ' |
+    base_url: str,                                       |_/ \_/ | |   |
+    domain: str,                                          _   _ _ _ _  _
+    tags: str,                                           |_) /| |\| | / `
+    type: str,                                           |  /"| | | | \_,
+    title: str,                                              ,-'"`-.
+    sources: [str],                                     /// /  @ @  \ \\\\
+    latest: {                                           \ :=| ,._,. |=:  /
+        ...,                                            || ,\ \_../ /. ||
+        pdf: 'output.pdf',                              ||','`-._))'`.`||
+        wget: 'example.com/1234/index.html'             `-'     (/    `-'
+    },
+    history: {
+        ...
+        pdf: [
+            {timestamp: 15444234325, status: 'skipped', result='output.pdf'},
+            ...
+        ],
+        wget: [
+            {timestamp: 11534435345, status: 'succeded', result='donuts.com/eat/them.html'}
+        ]
+    },
+}
+
+"""
+
+import datetime
+from html import unescape
+from collections import OrderedDict
+
+from util import (
+    domain,
+    base_url,
+    str_between,
+    get_link_type,
+    merge_links,
+    wget_output_path,
+)
+from config import ANSI
+
+
+def validate_links(links):
+    links = archivable_links(links)  # remove chrome://, about:, mailto: etc.
+    links = uniquefied_links(links)  # merge/dedupe duplicate timestamps & urls
+    links = sorted_links(links)      # deterministically sort the links based on timstamp, url
+    
+    if not links:
+        print('[X] No links found :(')
+        raise SystemExit(1)
+
+    for link in links:
+        link['title'] = unescape(link['title'])
+        link['latest'] = link.get('latest') or {}
+        
+        latest = link['latest']
+        if not link['latest'].get('wget'):
+            link['latest']['wget'] = wget_output_path(link)
+
+        if not link['latest'].get('pdf'):
+            link['latest']['pdf'] = None
+
+        if not link['latest'].get('screenshot'):
+            link['latest']['screenshot'] = None
+
+        if not link['latest'].get('dom'):
+            link['latest']['dom'] = None
+
+        if not latest.get('favicon'):
+            latest['favicon'] = None
+
+    return list(links)
+
+def new_links(all_links, existing_links):
+    """
+    Return all links which are in the all_links but not in the existing_links.
+    This is used to determine which links are new and not indexed jet. Set the
+    ONLY_NEW environment variable to activate this filter mechanism.
+    """
+    existing_urls = {link['url'] for link in existing_links}
+    return [link for link in all_links if link['url'] not in existing_urls]
+
+def archivable_links(links):
+    """remove chrome://, about:// or other schemed links that cant be archived"""
+    return (
+        link
+        for link in links
+        if any(link['url'].startswith(s) for s in ('http://', 'https://', 'ftp://'))
+    )
+
+def uniquefied_links(sorted_links):
+    """
+    ensures that all non-duplicate links have monotonically increasing timestamps
+    """
+
+    unique_urls = OrderedDict()
+
+    lower = lambda url: url.lower().strip()
+    without_www = lambda url: url.replace('://www.', '://', 1)
+    without_trailing_slash = lambda url: url[:-1] if url[-1] == '/' else url.replace('/?', '?')
+
+    for link in sorted_links:
+        fuzzy_url = without_www(without_trailing_slash(lower(link['url'])))
+        if fuzzy_url in unique_urls:
+            # merge with any other links that share the same url
+            link = merge_links(unique_urls[fuzzy_url], link)
+        unique_urls[fuzzy_url] = link
+
+    unique_timestamps = OrderedDict()
+    for link in unique_urls.values():
+        link['timestamp'] = lowest_uniq_timestamp(unique_timestamps, link['timestamp'])
+        unique_timestamps[link['timestamp']] = link
+
+    return unique_timestamps.values()
+
+def sorted_links(links):
+    sort_func = lambda link: (link['timestamp'].split('.', 1)[0], link['url'])
+    return sorted(links, key=sort_func, reverse=True)
+
+def links_after_timestamp(links, timestamp=None):
+    if not timestamp:
+        yield from links
+        return
+
+    for link in links:
+        try:
+            if float(link['timestamp']) <= float(timestamp):
+                yield link
+        except (ValueError, TypeError):
+            print('Resume value and all timestamp values must be valid numbers.')
+
+def lowest_uniq_timestamp(used_timestamps, timestamp):
+    """resolve duplicate timestamps by appending a decimal 1234, 1234 -> 1234.1, 1234.2"""
+
+    timestamp = timestamp.split('.')[0]
+    nonce = 0
+
+    # first try 152323423 before 152323423.0
+    if timestamp not in used_timestamps:
+        return timestamp
+
+    new_timestamp = '{}.{}'.format(timestamp, nonce)
+    while new_timestamp in used_timestamps:
+        nonce += 1
+        new_timestamp = '{}.{}'.format(timestamp, nonce)
+
+    return new_timestamp
@@ -0,0 +1,247 @@
+# coding: utf-8
+
+"""
+Everything related to parsing links from bookmark services.
+
+For a list of supported services, see the README.md.
+For examples of supported files see examples/.
+
+Parsed link schema: {
+    'url': 'https://example.com/example/?abc=123&xyc=345#lmnop',
+    'domain': 'example.com',
+    'base_url': 'example.com/example/',
+    'timestamp': '15442123124234',
+    'tags': 'abc,def',
+    'title': 'Example.com Page Title',
+    'sources': ['ril_export.html', 'downloads/getpocket.com.txt'],
+}
+"""
+
+import re
+import json
+import xml.etree.ElementTree as etree
+
+from datetime import datetime
+
+from util import (
+    domain,
+    base_url,
+    str_between,
+    get_link_type,
+)
+
+
+def get_parsers(file):
+    """return all parsers that work on a given file, defaults to all of them"""
+
+    return {
+        'pocket': parse_pocket_export,
+        'pinboard': parse_json_export,
+        'bookmarks': parse_bookmarks_export,
+        'rss': parse_rss_export,
+        'pinboard_rss': parse_pinboard_rss_feed,
+        'medium_rss': parse_medium_rss_feed,
+    }
+
+def parse_links(path):
+    """parse a list of links dictionaries from a bookmark export file"""
+    
+    links = []
+    with open(path, 'r', encoding='utf-8') as file:
+        for parser_func in get_parsers(file).values():
+            # otherwise try all parsers until one works
+            try:
+                links += list(parser_func(file))
+                if links:
+                    break
+            except (ValueError, TypeError, IndexError, AttributeError, etree.ParseError):
+                # parser not supported on this file
+                pass
+
+    return links
+
+
+def parse_pocket_export(html_file):
+    """Parse Pocket-format bookmarks export files (produced by getpocket.com/export/)"""
+
+    html_file.seek(0)
+    pattern = re.compile("^\\s*<li><a href=\"(.+)\" time_added=\"(\\d+)\" tags=\"(.*)\">(.+)</a></li>", re.UNICODE)
+    for line in html_file:
+        # example line
+        # <li><a href="http://example.com/ time_added="1478739709" tags="tag1,tag2">example title</a></li>
+        match = pattern.search(line)
+        if match:
+            fixed_url = match.group(1).replace('http://www.readability.com/read?url=', '')           # remove old readability prefixes to get original url
+            time = datetime.fromtimestamp(float(match.group(2)))
+            info = {
+                'url': fixed_url,
+                'domain': domain(fixed_url),
+                'base_url': base_url(fixed_url),
+                'timestamp': str(time.timestamp()),
+                'tags': match.group(3),
+                'title': match.group(4).replace(' — Readability', '').replace('http://www.readability.com/read?url=', '') or base_url(fixed_url),
+                'sources': [html_file.name],
+            }
+            info['type'] = get_link_type(info)
+            yield info
+
+def parse_json_export(json_file):
+    """Parse JSON-format bookmarks export files (produced by pinboard.in/export/, or wallabag)"""
+    json_file.seek(0)
+    json_content = json.load(json_file)
+    for line in json_content:
+        # example line
+        # {"href":"http:\/\/www.reddit.com\/r\/example","description":"title here","extended":"","meta":"18a973f09c9cc0608c116967b64e0419","hash":"910293f019c2f4bb1a749fb937ba58e3","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"reddit android"}]
+        if line:
+            erg = line
+            if erg.get('timestamp'):
+                timestamp = str(erg['timestamp']/10000000)  # chrome/ff histories use a very precise timestamp
+            elif erg.get('time'):
+                timestamp = str(datetime.strptime(erg['time'].split(',', 1)[0], '%Y-%m-%dT%H:%M:%SZ').timestamp())
+            elif erg.get('created_at'):
+                timestamp = str(datetime.strptime(erg['created_at'], '%Y-%m-%dT%H:%M:%S%z').timestamp())
+            else:
+                timestamp = str(datetime.now().timestamp())
+            if erg.get('href'):
+                url = erg['href']
+            else:
+                url = erg['url']
+            if erg.get('description'):
+                title = (erg.get('description') or '').replace(' — Readability', '')
+            else:
+                title = erg['title']
+            info = {
+                'url': url,
+                'domain': domain(url),
+                'base_url': base_url(url),
+                'timestamp': timestamp,
+                'tags': erg.get('tags') or '',
+                'title': title,
+                'sources': [json_file.name],
+            }
+            info['type'] = get_link_type(info)
+            yield info
+
+def parse_rss_export(rss_file):
+    """Parse RSS XML-format files into links"""
+
+    rss_file.seek(0)
+    items = rss_file.read().split('</item>\n<item>')
+    for item in items:
+        # example item:
+        # <item>
+        # <title><![CDATA[How JavaScript works: inside the V8 engine]]></title>
+        # <category>Unread</category>
+        # <link>https://blog.sessionstack.com/how-javascript-works-inside</link>
+        # <guid>https://blog.sessionstack.com/how-javascript-works-inside</guid>
+        # <pubDate>Mon, 21 Aug 2017 14:21:58 -0500</pubDate>
+        # </item>
+
+        trailing_removed = item.split('</item>', 1)[0]
+        leading_removed = trailing_removed.split('<item>', 1)[-1]
+        rows = leading_removed.split('\n')
+
+        def get_row(key):
+            return [r for r in rows if r.startswith('<{}>'.format(key))][0]
+
+        title = str_between(get_row('title'), '<![CDATA[', ']]')
+        url = str_between(get_row('link'), '<link>', '</link>')
+        ts_str = str_between(get_row('pubDate'), '<pubDate>', '</pubDate>')
+        time = datetime.strptime(ts_str, "%a, %d %b %Y %H:%M:%S %z")
+
+        info = {
+            'url': url,
+            'domain': domain(url),
+            'base_url': base_url(url),
+            'timestamp': str(time.timestamp()),
+            'tags': '',
+            'title': title,
+            'sources': [rss_file.name],
+        }
+        info['type'] = get_link_type(info)
+
+        yield info
+
+def parse_bookmarks_export(html_file):
+    """Parse netscape-format bookmarks export files (produced by all browsers)"""
+
+    html_file.seek(0)
+    pattern = re.compile("<a href=\"(.+?)\" add_date=\"(\\d+)\"[^>]*>(.+)</a>", re.UNICODE | re.IGNORECASE)
+    for line in html_file:
+        # example line
+        # <DT><A HREF="https://example.com/?q=1+2" ADD_DATE="1497562974" LAST_MODIFIED="1497562974" ICON_URI="https://example.com/favicon.ico" ICON="data:image/png;base64,...">example bookmark title</A>
+        
+        match = pattern.search(line)
+        if match:
+            url = match.group(1)
+            time = datetime.fromtimestamp(float(match.group(2)))
+
+            info = {
+                'url': url,
+                'domain': domain(url),
+                'base_url': base_url(url),
+                'timestamp': str(time.timestamp()),
+                'tags': "",
+                'title': match.group(3),
+                'sources': [html_file.name],
+            }
+            info['type'] = get_link_type(info)
+
+            yield info
+
+def parse_pinboard_rss_feed(rss_file):
+    """Parse Pinboard RSS feed files into links"""
+
+    rss_file.seek(0)
+    root = etree.parse(rss_file).getroot()
+    items = root.findall("{http://purl.org/rss/1.0/}item")
+    for item in items:
+        url = item.find("{http://purl.org/rss/1.0/}link").text
+        tags = item.find("{http://purl.org/dc/elements/1.1/}subject").text
+        title = item.find("{http://purl.org/rss/1.0/}title").text
+        ts_str = item.find("{http://purl.org/dc/elements/1.1/}date").text
+        #       = 🌈🌈🌈🌈
+        #        = 🌈🌈🌈🌈
+        #         = 🏆🏆🏆🏆
+        
+        # Pinboard includes a colon in its date stamp timezone offsets, which
+        # Python can't parse. Remove it:
+        if ":" == ts_str[-3:-2]:
+            ts_str = ts_str[:-3]+ts_str[-2:]
+        time = datetime.strptime(ts_str, "%Y-%m-%dT%H:%M:%S%z")
+        info = {
+            'url': url,
+            'domain': domain(url),
+            'base_url': base_url(url),
+            'timestamp': str(time.timestamp()),
+            'tags': tags,
+            'title': title,
+            'sources': [rss_file.name],
+        }
+        info['type'] = get_link_type(info)
+        yield info
+
+def parse_medium_rss_feed(rss_file):
+    """Parse Medium RSS feed files into links"""
+
+    rss_file.seek(0)
+    root = etree.parse(rss_file).getroot()
+    items = root.find("channel").findall("item")
+    for item in items:
+        # for child in item:
+        #     print(child.tag, child.text)
+        url = item.find("link").text
+        title = item.find("title").text
+        ts_str = item.find("pubDate").text
+        time = datetime.strptime(ts_str, "%a, %d %b %Y %H:%M:%S %Z")
+        info = {
+            'url': url,
+            'domain': domain(url),
+            'base_url': base_url(url),
+            'timestamp': str(time.timestamp()),
+            'tags': "",
+            'title': title,
+            'sources': [rss_file.name],
+        }
+        info['type'] = get_link_type(info)
+        yield info
@@ -0,0 +1,113 @@
+from collections import deque
+
+_marker = object()
+
+class Peekable(object):
+    """Peekable version of a normal python generator.
+       Useful when you don't want to evaluate the entire iterable to look at
+       a specific item at a given idx.
+    """
+    def __init__(self, iterable):
+        self._it = iter(iterable)
+        self._cache = deque()
+
+    def __iter__(self):
+        return self
+
+    def __bool__(self):
+        try:
+            self.peek()
+        except StopIteration:
+            return False
+        return True
+
+    def __nonzero__(self):
+        # For Python 2 compatibility
+        return self.__bool__()
+
+    def peek(self, default=_marker):
+        """Return the item that will be next returned from ``next()``.
+        Return ``default`` if there are no items left. If ``default`` is not
+        provided, raise ``StopIteration``.
+        """
+        if not self._cache:
+            try:
+                self._cache.append(next(self._it))
+            except StopIteration:
+                if default is _marker:
+                    raise
+                return default
+        return self._cache[0]
+
+    def prepend(self, *items):
+        """Stack up items to be the next ones returned from ``next()`` or
+        ``self.peek()``. The items will be returned in
+        first in, first out order::
+            >>> p = peekable([1, 2, 3])
+            >>> p.prepend(10, 11, 12)
+            >>> next(p)
+            10
+            >>> list(p)
+            [11, 12, 1, 2, 3]
+        It is possible, by prepending items, to "resurrect" a peekable that
+        previously raised ``StopIteration``.
+            >>> p = peekable([])
+            >>> next(p)
+            Traceback (most recent call last):
+              ...
+            StopIteration
+            >>> p.prepend(1)
+            >>> next(p)
+            1
+            >>> next(p)
+            Traceback (most recent call last):
+              ...
+            StopIteration
+        """
+        self._cache.extendleft(reversed(items))
+
+    def __next__(self):
+        if self._cache:
+            return self._cache.popleft()
+
+        return next(self._it)
+
+    next = __next__  # For Python 2 compatibility
+
+    def _get_slice(self, index):
+        # Normalize the slice's arguments
+        step = 1 if (index.step is None) else index.step
+        if step > 0:
+            start = 0 if (index.start is None) else index.start
+            stop = maxsize if (index.stop is None) else index.stop
+        elif step < 0:
+            start = -1 if (index.start is None) else index.start
+            stop = (-maxsize - 1) if (index.stop is None) else index.stop
+        else:
+            raise ValueError('slice step cannot be zero')
+
+        # If either the start or stop index is negative, we'll need to cache
+        # the rest of the iterable in order to slice from the right side.
+        if (start < 0) or (stop < 0):
+            self._cache.extend(self._it)
+        # Otherwise we'll need to find the rightmost index and cache to that
+        # point.
+        else:
+            n = min(max(start, stop) + 1, maxsize)
+            cache_len = len(self._cache)
+            if n >= cache_len:
+                self._cache.extend(islice(self._it, n - cache_len))
+
+        return list(self._cache)[index]
+
+    def __getitem__(self, index):
+        if isinstance(index, slice):
+            return self._get_slice(index)
+
+        cache_len = len(self._cache)
+        if index < 0:
+            self._cache.extend(self._it)
+        elif index >= cache_len:
+            self._cache.extend(islice(self._it, index + 1 - cache_len))
+
+        return self._cache[index]
@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+import argparse
+import re
+from typing import List
+
+from archive import parse_json_link_index
+from config import OUTPUT_DIR
+from index import write_json_links_index
+
+
+def cleanup_index(patterns: List[str], yes=False):
+    regexes = [re.compile(p) for p in patterns]
+
+    index = parse_json_link_index(OUTPUT_DIR)
+    links = index['links']
+
+    filtered = []
+    remaining = []
+    for l in links:
+        url = l['url']
+        for r in regexes:
+            if r.search(url):
+                filtered.append((l, r))
+                break
+        else:
+            remaining.append(l)
+
+
+    print("Filtered out {}/{} urls:".format(len(filtered), len(links)))
+    for link, regex in filtered:
+        url = link['url']
+        print(" {url} via {regex}".format(url=url, regex=regex.pattern))
+
+    proceed = False
+    if yes:
+        proceed = True
+    else:
+        res = input("Remove {} entries from index? [y/n] ".format(len(filtered)))
+        proceed = res.strip().lower() in ('y', 'yes')
+
+    if proceed:
+        write_json_links_index(OUTPUT_DIR, remaining)
+    else:
+        exit('aborting')
+
+
+if __name__ == '__main__':
+    p = argparse.ArgumentParser('Index purging tool')
+    p.add_argument('--regex', '-r', action='append', help='Python regex to filter out')
+    p.add_argument('--yes', action='store_true', default=False, help='Do not propmpt for confirmation')
+
+    args = p.parse_args()
+    regexes = args.regex
+    cleanup_index(regexes, yes=args.yes)
@@ -0,0 +1 @@
+requests
@@ -0,0 +1,169 @@
+<html lang="en">
+    <head>
+        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
+        <title>Archived Sites</title>
+        <style>
+            html, body {
+                width: 100%;
+                height: 100%;
+                font-size: 18px;
+                font-weight: 200;
+                text-align: center;
+                margin: 0px;
+                padding: 0px;
+                font-family: "Gill Sans", Helvetica, sans-serif;
+            }
+            header {
+                background-color: #aa1e55;
+                color: white;
+                padding: 10px;
+                padding-top: 0px;
+                padding-bottom: 15px;
+                height: 100px;
+            }
+            header h1 {
+                font-size: 38px;
+                font-weight: 300;
+                color: black;
+                padding-top: 14px;
+                line-height: 1.4;
+                width: 100%;
+            }
+            header h1 small {
+                color: white;
+                font-size:0.45em;
+                margin-left: 10px;
+                display: block;
+            }
+            header h1 small a {
+                text-decoration: none;
+                color: orange;
+                opacity: 0.6;
+                font-weight: 300;
+            }
+            header h1 small a:hover {
+                opacity: 1;
+            }
+            header + div {
+                margin-top: 10px;
+            }
+            .header-center {
+                width: 100%;
+                text-align: center;
+            }
+            .header-left {
+                float: left;
+                width: 50px;
+                height: 60px;
+                text-align: center;
+                padding: 20px;
+                margin-right: -70px;
+            }
+            #table-bookmarks_length, #table-bookmarks_filter {
+                padding: 0px 15px;
+            }
+            table {
+                padding: 6px;
+                width: 100%;
+            }
+            table thead th {
+                font-weight: 400;
+            }
+            table tr {
+                height: 35px;
+            }
+            tbody tr:nth-child(odd) {
+               background-color: #ffebeb !important;
+            }
+            table tr td {
+                white-space: nowrap;
+                overflow: hidden;
+                /*padding-bottom: 0.4em;*/
+                /*padding-top: 0.4em;*/
+                padding-left: 2px;
+                text-align: center;
+            }
+            table tr td a {
+                text-decoration: none;
+            }
+            table tr td img, table tr td object {
+                display: inline-block;
+                margin: auto;
+                height: 24px;
+                width: 24px;
+                padding: 0px;
+                padding-right: 5px;
+                vertical-align: middle;
+                margin-left: 4px;
+            }
+            #table-bookmarks {
+                width: 100%; 
+                overflow-y: scroll;
+                table-layout: fixed;
+            }
+        </style>
+    </head>
+    <body>
+        <header>
+            <div class="header-left">
+                <a href="?" title="Reload...">
+                    <img src="static/archive.png" style="height: 100%;"/>
+                </a>
+                <br/>
+                <a href="https://github.com/pirate/ArchiveBox">
+                    Github
+                </a>
+            </div>
+            <div class="header-center">
+                <h1>
+                    &nbsp;Archived Sites
+                    <br/>
+                    <small>
+                        <a href="?">Last updated $time_updated</a><br/>
+                    </small>
+                </h1>
+            </div>
+        </header>
+        <table id="table-bookmarks">
+            <thead>
+                <tr>
+                    <th style="width: 80px;">Bookmarked</th>
+                    <th style="width: 26px;">Files</th>
+                    <th style="width: 26vw;">Saved Link ($num_links)</th>
+                    <th style="width: 30px;">PNG</th>
+                    <th style="width: 30px">PDF</th>
+                    <th style="width: 30px">HTML</th>
+                    <th style="width: 30px">A.org</th>
+                    <th style="width: 16vw;whitespace:nowrap;overflow-x:hidden;">Original URL</th>
+                </tr>
+            </thead>
+            <tbody>$rows</tbody>
+        </table>
+        <footer>
+            <br/>
+            <center>
+                <small>
+                    Archive created using <a href="https://github.com/pirate/ArchiveBox" title="Github">ArchiveBox</a>
+                    version <a href="https://github.com/pirate/ArchiveBox/commit/$git_sha" title="Git commit">$short_git_sha</a> &nbsp; | &nbsp; 
+                    Download index as <a href="index.json" title="JSON summary of archived links.">JSON</a>
+                    <br/><br/>
+                    $footer_info
+                </small>
+            </center>
+            <br/>
+        </footer>
+        <link rel="stylesheet" href="static/jquery.dataTables.min.css"/>
+        <script src="static/jquery.min.js"></script>
+        <script src="static/jquery.dataTables.min.js"></script>
+        <script>
+            jQuery(document).ready(function() {
+                jQuery('#table-bookmarks').DataTable({
+                    stateSave: true, // save state (filtered input, number of entries shown, etc) in localStorage
+                    dom: '<lf<t>ip>', // how to show the table and its helpers (filter, etc) in the DOM
+                    order: [[0, 'desc']],
+                    iDisplayLength: 100,
+                });
+            });
+        </script>
+    </body>
+</html>
@@ -0,0 +1,16 @@
+<tr>
+    <td title="Bookmarked timestamp: $timestamp">$date</td>
+    <td>
+        <a href="$files_url" title="Link Index">
+            <img src="$favicon_url" onerror="this.src='static/spinner.gif'" class="link-favicon">
+        </a>
+    </td>
+    <td style="text-align: left"><a href="$archive_url" style="font-size:1.4em;text-decoration:none;color:black;" title="$title">
+        $title <small style="background-color: #eee;border-radius:4px; float:right">$tags</small>
+    </td>
+    <td><a href="$screenshot_link" title="Screenshot">🖼</a></td>
+    <td><a href="$pdf_link" title="PDF">📜</a></td>
+    <td><a href="$dom_link" title="DOM">📄</a></td>
+    <td><a href="$archive_org_url" title="Archive.org">🏛</a></td>
+    <td style="text-align: left"><!--🔗 <img src="$google_favicon_url" height="16px">--> <a href="$url">$url</a></td>
+</tr>
@@ -0,0 +1,63 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>$title</title>
+        
+    </head>
+    <body>
+        <header>
+            <h1>
+                <img src="$favicon" height="20px"> $title<br/>
+                <a href="$url" class="title-url">
+                    <small>$base_url</small>
+                </a>
+            </h1>
+        </header>
+        <hr/>
+        <div>
+            Tags: $tags<br/>
+            Type: $type<br/>
+            <br/>
+            Bookmarked:<br/>
+                $bookmarked<br/>
+            Archived:<br/>
+                $updated<br/>
+        </div>
+        <hr/>
+        <ul>
+            <li>
+                <a href="$url"><b>Original</b></a><br/>
+                $base_url<br/>&nbsp;
+            </li>
+            <li>
+                <a href="$wget"><b>Local Archive</b></a><br/>
+                archive/$timestamp/$domain<br/>&nbsp;
+            </li>
+            <li>
+                <a href="$pdf" id="pdf-btn"><b>PDF</b></a><br/>
+                archive/$timestamp/output.pdf<br/>&nbsp;
+            </li>
+            <li>
+                <a href="$screenshot"><b>Screenshot</b></a><br/>
+                archive/$timestamp/screenshot.png<br/>&nbsp;
+            </li>
+            <li>
+                <a href="$dom"><b>HTML</b></a><br/>
+                archive/$timestamp/output.html<br/>&nbsp;
+            </li>
+            <li>
+                <a href="$archive_org"><b>Archive.Org</b></a><br/>
+                web.archive.org/web/$base_url<br/>&nbsp;
+            </li>
+        </ul>
+        <footer>
+            <hr/>
+            <a href="index.json">JSON</a> | <a href=".">Files</a>
+            <hr/>
+            <a href="./../../index.html" class="nav-icon" title="Archived Sites">
+                <img src="https://nicksweeting.com/images/archive.png" alt="Archive Icon" height="20px">
+                ArchiveBox: Link Index
+            </a>
+        </footer>
+    </body>
+</html>
@@ -0,0 +1,317 @@
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>$title</title>
+        <style>
+            html, body {
+                width: 100%;
+                height: 100%;
+            }
+            body {
+                background-color: #ddd;
+            }
+            header {
+                width: 100%;
+                height: 90px;
+                background-color: #aa1e55;
+                margin: 0px;
+                text-align: center;
+                color: white;
+            }
+            header h1 {
+                padding-top: 5px;
+                padding-bottom: 5px;
+                margin: 0px;
+                font-weight: 200;
+                font-family: "Gill Sans", Helvetica, sans-serif;
+                font-size: calc(16px + 1vw);
+            }
+            .collapse-icon {
+                float: right;
+                color: black;
+                width: 126px;
+                font-size: 0.8em;
+                margin-top: 20px;
+                margin-right: 0px;
+                margin-left: -35px;
+            }
+            .nav-icon img {
+                float: left;
+                display: block;
+                margin-right: 13px;
+                color: black;
+                height: 53px;
+                margin-top: 7px;
+                margin-left: 10px;
+            }
+            .nav-icon img:hover {
+                opacity: 0.5;
+            }
+            .title-url {
+                color: black;
+                display: block;
+                width: 75%;
+                white-space: nowrap;
+                overflow: hidden;
+                margin: auto;
+            }
+            .archive-page-header {
+                margin-top: 5px;
+                margin-bottom: 5px;
+            }
+            .archive-page-header .alert {
+                margin-bottom: 0px;
+            }
+            h1 small {
+                opacity: 0.4;
+                font-size: 0.6em;
+            }
+            h1 small:hover {
+                opacity: 0.8;
+            }
+            .card {
+                box-shadow: 2px 3px 14px 0px rgba(0,0,0,0.02);
+            }
+            .card h4 {
+                font-size: 1.4vw;
+            }
+            .card-body {
+                font-size: 1vw;
+                padding-top: 1.2vw;
+                padding-left: 1vw;
+                padding-right: 1vw;
+                padding-bottom: 1vw;
+                line-height: 1.1;
+                word-wrap: break-word;
+                max-height: 102px;
+                overflow: hidden;
+            }
+            .card-img-top {
+                border: 0px;
+                padding: 0px;
+                margin: 0px;
+                overflow: hidden;
+                opacity: 0.8;
+                border-top: 1px solid gray;
+                border-radius: 3px;
+                border-bottom: 1px solid #ddd;
+                height: 430px;
+                width: 400%;
+                margin-bottom: -330px;
+
+                transform: scale(0.25); 
+                transform-origin: 0 0;
+            }
+            .full-page-iframe {
+                border-top: 1px solid #ddd;
+                width: 100%;
+                height: 69vh;
+                margin: 0px;
+                border: 0px;
+                border-top: 3px solid #aa1e55;
+            }
+            .card.selected-card {
+                border: 2px solid orange;
+                box-shadow: 0px -6px 13px 1px rgba(0,0,0,0.05);
+            }
+            .iframe-large {
+                height: 93%;
+                margin-top: -10px;
+            }
+            img.external {
+                height: 30px;
+                margin-right: -10px;
+                padding: 3px;
+                border-radius: 4px;
+                vertical-align: middle;
+                border: 4px solid rgba(0,0,0,0);
+            }
+            img.external:hover {
+                border: 4px solid green;
+            }
+
+            @media(max-width: 1092px) {
+                iframe {
+                    display: none;
+                }
+            }
+                
+
+            @media(max-width: 728px) {
+                .card h4 {
+                    font-size: 5vw;
+                }
+                .card-body {
+                    font-size: 4vw;
+                }
+                .card {
+                    margin-bottom: 5px;
+                }
+                header > h1 > a.collapse-icon, header > h1 > a.nav-icon {
+                    display: none;
+                }
+            }
+        </style>
+        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/css/bootstrap.min.css" integrity="sha384-rwoIResjU2yc3z8GV/NPeZWAv56rSmLldC3R/AZzGRnGxQQKnKkoFVhFQhNUwEyJ" crossorigin="anonymous">
+    </head>
+    <body>
+        <header>
+            <h1 class="page-title">
+                <a href="../../index.html" class="nav-icon" title="Go to Main Index...">
+                    <img src="../../static/archive.png" alt="Archive Icon">
+                </a>
+                <a href="#" class="collapse-icon" style="text-decoration: none" title="Toggle info panel...">
+                    ▾
+                </a>
+                <img src="$favicon" height="20px"> $title<br/>
+                <a href="$url" class="title-url">
+                    <small>$base_url</small>
+                </a>
+            </h1>
+        </header>
+        <div class="site-header container-fluid">
+            <div class="row archive-page-header">
+                <div class="col-lg-4 alert well">
+                    Bookmarked: <small title="Timestamp: $bookmarked_ts">$bookmarked</small>
+                    &nbsp; | &nbsp;
+                    Last updated: <small title="Timestamp: $updated_ts">$updated</small>
+                </div>
+                <div class="col-lg-4 alert well">
+                    Type: 
+                    <span class="badge badge-default">$type</span>
+                    &nbsp; | &nbsp;
+                    Tags:
+                    <span class="badge badge-success">$tags</span> 
+                </div>
+                <div class="col-lg-4 alert well">
+                    Download:
+                    <a href="index.json" title="JSON summary of archived link.">JSON</a> | 
+                    <a href="." title="Webserver-provided index of files directory.">Files</a>
+                </div>
+                <hr/>
+                <div class="col-lg-2">
+                    <div class="card selected-card">
+                      <iframe class="card-img-top" src="$wget" sandbox="allow-same-origin allow-scripts allow-forms"></iframe>
+                      <div class="card-body">
+                        <a href="$wget" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$wget" target="preview"><h4 class="card-title">Local Archive</h4></a>
+                        <p class="card-text">archive/$domain</p>
+                      </div>
+                    </div>
+                </div>
+                <div class="col-lg-2">
+                    <div class="card">
+                      <iframe class="card-img-top" src="$dom" sandbox="allow-same-origin allow-scripts allow-forms"></iframe>
+                      <div class="card-body">
+                        <a href="$dom" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$dom" target="preview"><h4 class="card-title">HTML</h4></a>
+                        <p class="card-text">archive/output.html</p>
+                      </div>
+                    </div>
+                </div>
+                <div class="col-lg-2">
+                    <div class="card">
+                      <iframe class="card-img-top" src="$pdf"></iframe>
+                      <div class="card-body">
+                        <a href="$pdf" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$pdf" target="preview" id="pdf-btn"><h4 class="card-title">PDF</h4></a>
+                        <p class="card-text">archive/output.pdf</p>
+                      </div>
+                    </div>
+                </div>
+                <div class="col-lg-2">
+                    <div class="card">
+                      <iframe class="card-img-top" src="$screenshot" sandbox="allow-same-origin allow-scripts allow-forms"></iframe>
+                      <div class="card-body">
+                        <a href="$screenshot" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$screenshot" target="preview"><h4 class="card-title">Screenshot</h4></a>
+                        <p class="card-text">archive/screenshot.png</p>
+                      </div>
+                    </div>
+                </div>
+                <div class="col-lg-2">
+                    <div class="card">
+                      <iframe class="card-img-top" src="$url" sandbox="allow-same-origin allow-scripts allow-forms"></iframe>
+                      <div class="card-body">
+                        <a href="$url" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$url" target="preview"><h4 class="card-title">Original</h4></a>
+                        <p class="card-text">$domain</p>
+                      </div>
+                    </div>
+                </div>
+                <div class="col-lg-2">
+                    <div class="card">
+                      <iframe class="card-img-top" src="$archive_org" sandbox="allow-same-origin allow-scripts allow-forms"></iframe>
+                      <div class="card-body">
+                        <a href="$archive_org" style="float:right" title="Open in new tab..." target="_blank" rel="noopener">
+                            <img src="../../static/external.png" class="external"/>
+                        </a>
+                        <a href="$archive_org" target="preview"><h4 class="card-title">Archive.Org</h4></a>
+                        <p class="card-text">web.archive.org/web/...</p>
+                      </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+        <iframe sandbox="allow-same-origin allow-scripts allow-forms" class="full-page-iframe" src="$wget" name="preview"></iframe>
+    
+        <script
+              src="https://code.jquery.com/jquery-3.2.1.slim.min.js"
+              integrity="sha256-k2WSCIexGzOj3Euiig+TlR8gA0EmPjuc79OEeY5L45g="
+              crossorigin="anonymous"></script>
+        <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0-alpha.6/js/bootstrap.min.js" integrity="sha384-vBWWzlZJ8ea9aCX4pEW3rVHjgjt7zpkNpZk+02D9phzyeVkE+jo0ieGizqPLForn" crossorigin="anonymous"></script>
+
+        <script>
+            // show selected file in iframe when preview card is clicked
+            jQuery('.card').on('click', function(e) {
+                jQuery('.selected-card').removeClass('selected-card')
+                jQuery(e.target).closest('.card').addClass('selected-card')
+            })
+            jQuery('.card a[target=preview]').on('click', function(e) {
+                if (e.currentTarget.href.endsWith('.pdf')) {
+                    jQuery('.full-page-iframe')[0].removeAttribute('sandbox')
+                } else {
+                    jQuery('.full-page-iframe')[0].sandbox = "allow-same-origin allow-scripts allow-forms"
+                }
+                return true
+            })
+
+            // un-sandbox iframes showing pdfs (required to display pdf viewer)
+            jQuery('iframe').map(function() {
+                if (this.src.endsWith('.pdf')) {
+                    this.removeAttribute('sandbox')
+                    this.src = this.src
+                }
+            })
+
+            // hide header when collapse icon is clicked
+            jQuery('.collapse-icon').on('click', function() {
+                if (jQuery('.collapse-icon').text().includes('▾')) {
+                    jQuery('.collapse-icon').text('▸')
+                    jQuery('.site-header').hide()
+                    jQuery('.full-page-iframe').addClass('iframe-large')
+                } else {
+                    jQuery('.collapse-icon').text('▾')
+                    jQuery('.site-header').show()
+                    jQuery('.full-page-iframe').removeClass('iframe-large')
+                }
+                return true
+            })
+
+            // hide all preview iframes on small screens
+            if (window.innerWidth < 1091) {
+                jQuery('.card a[target=preview]').attr('target', '_self')
+            }
+        </script>
+    </body>
+</html>
@@ -0,0 +1 @@
+table.dataTable{width:100%;margin:0 auto;clear:both;border-collapse:separate;border-spacing:0}table.dataTable thead th,table.dataTable tfoot th{font-weight:bold}table.dataTable thead th,table.dataTable thead td{padding:10px 18px;border-bottom:1px solid #111}table.dataTable thead th:active,table.dataTable thead td:active{outline:none}table.dataTable tfoot th,table.dataTable tfoot td{padding:10px 18px 6px 18px;border-top:1px solid #111}table.dataTable thead .sorting,table.dataTable thead .sorting_asc,table.dataTable thead .sorting_desc,table.dataTable thead .sorting_asc_disabled,table.dataTable thead .sorting_desc_disabled{cursor:pointer;*cursor:hand;background-repeat:no-repeat;background-position:center right}table.dataTable thead .sorting{background-image:url("sort_both.png")}table.dataTable thead .sorting_asc{background-image:url("sort_asc.png")}table.dataTable thead .sorting_desc{background-image:url("sort_desc.png")}table.dataTable thead .sorting_asc_disabled{background-image:url("sort_asc_disabled.png")}table.dataTable thead .sorting_desc_disabled{background-image:url("sort_desc_disabled.png")}table.dataTable tbody tr{background-color:#ffffff}table.dataTable tbody tr.selected{background-color:#B0BED9}table.dataTable tbody th,table.dataTable tbody td{padding:8px 10px}table.dataTable.row-border tbody th,table.dataTable.row-border tbody td,table.dataTable.display tbody th,table.dataTable.display tbody td{border-top:1px solid #ddd}table.dataTable.row-border tbody tr:first-child th,table.dataTable.row-border tbody tr:first-child td,table.dataTable.display tbody tr:first-child th,table.dataTable.display tbody tr:first-child td{border-top:none}table.dataTable.cell-border tbody th,table.dataTable.cell-border tbody td{border-top:1px solid #ddd;border-right:1px solid #ddd}table.dataTable.cell-border tbody tr th:first-child,table.dataTable.cell-border tbody tr td:first-child{border-left:1px solid #ddd}table.dataTable.cell-border tbody tr:first-child th,table.dataTable.cell-border tbody tr:first-child td{border-top:none}table.dataTable.stripe tbody tr.odd,table.dataTable.display tbody tr.odd{background-color:#f9f9f9}table.dataTable.stripe tbody tr.odd.selected,table.dataTable.display tbody tr.odd.selected{background-color:#acbad4}table.dataTable.hover tbody tr:hover,table.dataTable.display tbody tr:hover{background-color:#f6f6f6}table.dataTable.hover tbody tr:hover.selected,table.dataTable.display tbody tr:hover.selected{background-color:#aab7d1}table.dataTable.order-column tbody tr>.sorting_1,table.dataTable.order-column tbody tr>.sorting_2,table.dataTable.order-column tbody tr>.sorting_3,table.dataTable.display tbody tr>.sorting_1,table.dataTable.display tbody tr>.sorting_2,table.dataTable.display tbody tr>.sorting_3{background-color:#fafafa}table.dataTable.order-column tbody tr.selected>.sorting_1,table.dataTable.order-column tbody tr.selected>.sorting_2,table.dataTable.order-column tbody tr.selected>.sorting_3,table.dataTable.display tbody tr.selected>.sorting_1,table.dataTable.display tbody tr.selected>.sorting_2,table.dataTable.display tbody tr.selected>.sorting_3{background-color:#acbad5}table.dataTable.display tbody tr.odd>.sorting_1,table.dataTable.order-column.stripe tbody tr.odd>.sorting_1{background-color:#f1f1f1}table.dataTable.display tbody tr.odd>.sorting_2,table.dataTable.order-column.stripe tbody tr.odd>.sorting_2{background-color:#f3f3f3}table.dataTable.display tbody tr.odd>.sorting_3,table.dataTable.order-column.stripe tbody tr.odd>.sorting_3{background-color:whitesmoke}table.dataTable.display tbody tr.odd.selected>.sorting_1,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_1{background-color:#a6b4cd}table.dataTable.display tbody tr.odd.selected>.sorting_2,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_2{background-color:#a8b5cf}table.dataTable.display tbody tr.odd.selected>.sorting_3,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_3{background-color:#a9b7d1}table.dataTable.display tbody tr.even>.sorting_1,table.dataTable.order-column.stripe tbody tr.even>.sorting_1{background-color:#fafafa}table.dataTable.display tbody tr.even>.sorting_2,table.dataTable.order-column.stripe tbody tr.even>.sorting_2{background-color:#fcfcfc}table.dataTable.display tbody tr.even>.sorting_3,table.dataTable.order-column.stripe tbody tr.even>.sorting_3{background-color:#fefefe}table.dataTable.display tbody tr.even.selected>.sorting_1,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_1{background-color:#acbad5}table.dataTable.display tbody tr.even.selected>.sorting_2,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_2{background-color:#aebcd6}table.dataTable.display tbody tr.even.selected>.sorting_3,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_3{background-color:#afbdd8}table.dataTable.display tbody tr:hover>.sorting_1,table.dataTable.order-column.hover tbody tr:hover>.sorting_1{background-color:#eaeaea}Line truncated
@@ -0,0 +1,166 @@
+/*!
+ DataTables 1.10.19
+ ©2008-2018 SpryMedia Ltd - datatables.net/license
+*/
+(function(h){"function"===typeof define&&define.amd?define(["jquery"],function(E){return h(E,window,document)}):"object"===typeof exports?module.exports=function(E,H){E||(E=window);H||(H="undefined"!==typeof window?require("jquery"):require("jquery")(E));return h(H,E,E.document)}:h(jQuery,window,document)})(function(h,E,H,k){function Z(a){var b,c,d={};h.each(a,function(e){if((b=e.match(/^([^A-Z]+?)([A-Z])/))&&-1!=="a aa ai ao as b fn i m o s ".indexOf(b[1]+" "))c=e.replace(b[0],b[2].toLowerCase()),
+d[c]=e,"o"===b[1]&&Z(a[e])});a._hungarianMap=d}function J(a,b,c){a._hungarianMap||Z(a);var d;h.each(b,function(e){d=a._hungarianMap[e];if(d!==k&&(c||b[d]===k))"o"===d.charAt(0)?(b[d]||(b[d]={}),h.extend(!0,b[d],b[e]),J(a[d],b[d],c)):b[d]=b[e]})}function Ca(a){var b=n.defaults.oLanguage,c=b.sDecimal;c&&Da(c);if(a){var d=a.sZeroRecords;!a.sEmptyTable&&(d&&"No data available in table"===b.sEmptyTable)&&F(a,a,"sZeroRecords","sEmptyTable");!a.sLoadingRecords&&(d&&"Loading..."===b.sLoadingRecords)&&F(a,
+a,"sZeroRecords","sLoadingRecords");a.sInfoThousands&&(a.sThousands=a.sInfoThousands);(a=a.sDecimal)&&c!==a&&Da(a)}}function fb(a){A(a,"ordering","bSort");A(a,"orderMulti","bSortMulti");A(a,"orderClasses","bSortClasses");A(a,"orderCellsTop","bSortCellsTop");A(a,"order","aaSorting");A(a,"orderFixed","aaSortingFixed");A(a,"paging","bPaginate");A(a,"pagingType","sPaginationType");A(a,"pageLength","iDisplayLength");A(a,"searching","bFilter");"boolean"===typeof a.sScrollX&&(a.sScrollX=a.sScrollX?"100%":
+"");"boolean"===typeof a.scrollX&&(a.scrollX=a.scrollX?"100%":"");if(a=a.aoSearchCols)for(var b=0,c=a.length;b<c;b++)a[b]&&J(n.models.oSearch,a[b])}function gb(a){A(a,"orderable","bSortable");A(a,"orderData","aDataSort");A(a,"orderSequence","asSorting");A(a,"orderDataType","sortDataType");var b=a.aDataSort;"number"===typeof b&&!h.isArray(b)&&(a.aDataSort=[b])}function hb(a){if(!n.__browser){var b={};n.__browser=b;var c=h("<div/>").css({position:"fixed",top:0,left:-1*h(E).scrollLeft(),height:1,width:1,
+overflow:"hidden"}).append(h("<div/>").css({position:"absolute",top:1,left:1,width:100,overflow:"scroll"}).append(h("<div/>").css({width:"100%",height:10}))).appendTo("body"),d=c.children(),e=d.children();b.barWidth=d[0].offsetWidth-d[0].clientWidth;b.bScrollOversize=100===e[0].offsetWidth&&100!==d[0].clientWidth;b.bScrollbarLeft=1!==Math.round(e.offset().left);b.bBounding=c[0].getBoundingClientRect().width?!0:!1;c.remove()}h.extend(a.oBrowser,n.__browser);a.oScroll.iBarWidth=n.__browser.barWidth}
+function ib(a,b,c,d,e,f){var g,j=!1;c!==k&&(g=c,j=!0);for(;d!==e;)a.hasOwnProperty(d)&&(g=j?b(g,a[d],d,a):a[d],j=!0,d+=f);return g}function Ea(a,b){var c=n.defaults.column,d=a.aoColumns.length,c=h.extend({},n.models.oColumn,c,{nTh:b?b:H.createElement("th"),sTitle:c.sTitle?c.sTitle:b?b.innerHTML:"",aDataSort:c.aDataSort?c.aDataSort:[d],mData:c.mData?c.mData:d,idx:d});a.aoColumns.push(c);c=a.aoPreSearchCols;c[d]=h.extend({},n.models.oSearch,c[d]);ka(a,d,h(b).data())}function ka(a,b,c){var b=a.aoColumns[b],
+d=a.oClasses,e=h(b.nTh);if(!b.sWidthOrig){b.sWidthOrig=e.attr("width")||null;var f=(e.attr("style")||"").match(/width:\s*(\d+[pxem%]+)/);f&&(b.sWidthOrig=f[1])}c!==k&&null!==c&&(gb(c),J(n.defaults.column,c),c.mDataProp!==k&&!c.mData&&(c.mData=c.mDataProp),c.sType&&(b._sManualType=c.sType),c.className&&!c.sClass&&(c.sClass=c.className),c.sClass&&e.addClass(c.sClass),h.extend(b,c),F(b,c,"sWidth","sWidthOrig"),c.iDataSort!==k&&(b.aDataSort=[c.iDataSort]),F(b,c,"aDataSort"));var g=b.mData,j=S(g),i=b.mRender?
+S(b.mRender):null,c=function(a){return"string"===typeof a&&-1!==a.indexOf("@")};b._bAttrSrc=h.isPlainObject(g)&&(c(g.sort)||c(g.type)||c(g.filter));b._setter=null;b.fnGetData=function(a,b,c){var d=j(a,b,k,c);return i&&b?i(d,b,a,c):d};b.fnSetData=function(a,b,c){return N(g)(a,b,c)};"number"!==typeof g&&(a._rowReadObject=!0);a.oFeatures.bSort||(b.bSortable=!1,e.addClass(d.sSortableNone));a=-1!==h.inArray("asc",b.asSorting);c=-1!==h.inArray("desc",b.asSorting);!b.bSortable||!a&&!c?(b.sSortingClass=d.sSortableNone,
+b.sSortingClassJUI=""):a&&!c?(b.sSortingClass=d.sSortableAsc,b.sSortingClassJUI=d.sSortJUIAscAllowed):!a&&c?(b.sSortingClass=d.sSortableDesc,b.sSortingClassJUI=d.sSortJUIDescAllowed):(b.sSortingClass=d.sSortable,b.sSortingClassJUI=d.sSortJUI)}function $(a){if(!1!==a.oFeatures.bAutoWidth){var b=a.aoColumns;Fa(a);for(var c=0,d=b.length;c<d;c++)b[c].nTh.style.width=b[c].sWidth}b=a.oScroll;(""!==b.sY||""!==b.sX)&&la(a);r(a,null,"column-sizing",[a])}function aa(a,b){var c=ma(a,"bVisible");return"number"===
+typeof c[b]?c[b]:null}function ba(a,b){var c=ma(a,"bVisible"),c=h.inArray(b,c);return-1!==c?c:null}function V(a){var b=0;h.each(a.aoColumns,function(a,d){d.bVisible&&"none"!==h(d.nTh).css("display")&&b++});return b}function ma(a,b){var c=[];h.map(a.aoColumns,function(a,e){a[b]&&c.push(e)});return c}function Ga(a){var b=a.aoColumns,c=a.aoData,d=n.ext.type.detect,e,f,g,j,i,h,l,q,t;e=0;for(f=b.length;e<f;e++)if(l=b[e],t=[],!l.sType&&l._sManualType)l.sType=l._sManualType;else if(!l.sType){g=0;for(j=d.length;g<
+j;g++){i=0;for(h=c.length;i<h;i++){t[i]===k&&(t[i]=B(a,i,e,"type"));q=d[g](t[i],a);if(!q&&g!==d.length-1)break;if("html"===q)break}if(q){l.sType=q;break}}l.sType||(l.sType="string")}}function jb(a,b,c,d){var e,f,g,j,i,m,l=a.aoColumns;if(b)for(e=b.length-1;0<=e;e--){m=b[e];var q=m.targets!==k?m.targets:m.aTargets;h.isArray(q)||(q=[q]);f=0;for(g=q.length;f<g;f++)if("number"===typeof q[f]&&0<=q[f]){for(;l.length<=q[f];)Ea(a);d(q[f],m)}else if("number"===typeof q[f]&&0>q[f])d(l.length+q[f],m);else if("string"===
+typeof q[f]){j=0;for(i=l.length;j<i;j++)("_all"==q[f]||h(l[j].nTh).hasClass(q[f]))&&d(j,m)}}if(c){e=0;for(a=c.length;e<a;e++)d(e,c[e])}}function O(a,b,c,d){var e=a.aoData.length,f=h.extend(!0,{},n.models.oRow,{src:c?"dom":"data",idx:e});f._aData=b;a.aoData.push(f);for(var g=a.aoColumns,j=0,i=g.length;j<i;j++)g[j].sType=null;a.aiDisplayMaster.push(e);b=a.rowIdFn(b);b!==k&&(a.aIds[b]=f);(c||!a.oFeatures.bDeferRender)&&Ha(a,e,c,d);return e}function na(a,b){var c;b instanceof h||(b=h(b));return b.map(function(b,
+e){c=Ia(a,e);return O(a,c.data,e,c.cells)})}function B(a,b,c,d){var e=a.iDraw,f=a.aoColumns[c],g=a.aoData[b]._aData,j=f.sDefaultContent,i=f.fnGetData(g,d,{settings:a,row:b,col:c});if(i===k)return a.iDrawError!=e&&null===j&&(K(a,0,"Requested unknown parameter "+("function"==typeof f.mData?"{function}":"'"+f.mData+"'")+" for row "+b+", column "+c,4),a.iDrawError=e),j;if((i===g||null===i)&&null!==j&&d!==k)i=j;else if("function"===typeof i)return i.call(g);return null===i&&"display"==d?"":i}function kb(a,
+b,c,d){a.aoColumns[c].fnSetData(a.aoData[b]._aData,d,{settings:a,row:b,col:c})}function Ja(a){return h.map(a.match(/(\\.|[^\.])+/g)||[""],function(a){return a.replace(/\\\./g,".")})}function S(a){if(h.isPlainObject(a)){var b={};h.each(a,function(a,c){c&&(b[a]=S(c))});return function(a,c,f,g){var j=b[c]||b._;return j!==k?j(a,c,f,g):a}}if(null===a)return function(a){return a};if("function"===typeof a)return function(b,c,f,g){return a(b,c,f,g)};if("string"===typeof a&&(-1!==a.indexOf(".")||-1!==a.indexOf("[")||
+-1!==a.indexOf("("))){var c=function(a,b,f){var g,j;if(""!==f){j=Ja(f);for(var i=0,m=j.length;i<m;i++){f=j[i].match(ca);g=j[i].match(W);if(f){j[i]=j[i].replace(ca,"");""!==j[i]&&(a=a[j[i]]);g=[];j.splice(0,i+1);j=j.join(".");if(h.isArray(a)){i=0;for(m=a.length;i<m;i++)g.push(c(a[i],b,j))}a=f[0].substring(1,f[0].length-1);a=""===a?g:g.join(a);break}else if(g){j[i]=j[i].replace(W,"");a=a[j[i]]();continue}if(null===a||a[j[i]]===k)return k;a=a[j[i]]}}return a};return function(b,e){return c(b,e,a)}}return function(b){return b[a]}}
+function N(a){if(h.isPlainObject(a))return N(a._);if(null===a)return function(){};if("function"===typeof a)return function(b,d,e){a(b,"set",d,e)};if("string"===typeof a&&(-1!==a.indexOf(".")||-1!==a.indexOf("[")||-1!==a.indexOf("("))){var b=function(a,d,e){var e=Ja(e),f;f=e[e.length-1];for(var g,j,i=0,m=e.length-1;i<m;i++){g=e[i].match(ca);j=e[i].match(W);if(g){e[i]=e[i].replace(ca,"");a[e[i]]=[];f=e.slice();f.splice(0,i+1);g=f.join(".");if(h.isArray(d)){j=0;for(m=d.length;j<m;j++)f={},b(f,d[j],g),
+a[e[i]].push(f)}else a[e[i]]=d;return}j&&(e[i]=e[i].replace(W,""),a=a[e[i]](d));if(null===a[e[i]]||a[e[i]]===k)a[e[i]]={};a=a[e[i]]}if(f.match(W))a[f.replace(W,"")](d);else a[f.replace(ca,"")]=d};return function(c,d){return b(c,d,a)}}return function(b,d){b[a]=d}}function Ka(a){return D(a.aoData,"_aData")}function oa(a){a.aoData.length=0;a.aiDisplayMaster.length=0;a.aiDisplay.length=0;a.aIds={}}function pa(a,b,c){for(var d=-1,e=0,f=a.length;e<f;e++)a[e]==b?d=e:a[e]>b&&a[e]--; -1!=d&&c===k&&a.splice(d,
+1)}function da(a,b,c,d){var e=a.aoData[b],f,g=function(c,d){for(;c.childNodes.length;)c.removeChild(c.firstChild);c.innerHTML=B(a,b,d,"display")};if("dom"===c||(!c||"auto"===c)&&"dom"===e.src)e._aData=Ia(a,e,d,d===k?k:e._aData).data;else{var j=e.anCells;if(j)if(d!==k)g(j[d],d);else{c=0;for(f=j.length;c<f;c++)g(j[c],c)}}e._aSortData=null;e._aFilterData=null;g=a.aoColumns;if(d!==k)g[d].sType=null;else{c=0;for(f=g.length;c<f;c++)g[c].sType=null;La(a,e)}}function Ia(a,b,c,d){var e=[],f=b.firstChild,g,
+j,i=0,m,l=a.aoColumns,q=a._rowReadObject,d=d!==k?d:q?{}:[],t=function(a,b){if("string"===typeof a){var c=a.indexOf("@");-1!==c&&(c=a.substring(c+1),N(a)(d,b.getAttribute(c)))}},G=function(a){if(c===k||c===i)j=l[i],m=h.trim(a.innerHTML),j&&j._bAttrSrc?(N(j.mData._)(d,m),t(j.mData.sort,a),t(j.mData.type,a),t(j.mData.filter,a)):q?(j._setter||(j._setter=N(j.mData)),j._setter(d,m)):d[i]=m;i++};if(f)for(;f;){g=f.nodeName.toUpperCase();if("TD"==g||"TH"==g)G(f),e.push(f);f=f.nextSibling}else{e=b.anCells;
+f=0;for(g=e.length;f<g;f++)G(e[f])}if(b=b.firstChild?b:b.nTr)(b=b.getAttribute("id"))&&N(a.rowId)(d,b);return{data:d,cells:e}}function Ha(a,b,c,d){var e=a.aoData[b],f=e._aData,g=[],j,i,m,l,q;if(null===e.nTr){j=c||H.createElement("tr");e.nTr=j;e.anCells=g;j._DT_RowIndex=b;La(a,e);l=0;for(q=a.aoColumns.length;l<q;l++){m=a.aoColumns[l];i=c?d[l]:H.createElement(m.sCellType);i._DT_CellIndex={row:b,column:l};g.push(i);if((!c||m.mRender||m.mData!==l)&&(!h.isPlainObject(m.mData)||m.mData._!==l+".display"))i.innerHTML=
+B(a,b,l,"display");m.sClass&&(i.className+=" "+m.sClass);m.bVisible&&!c?j.appendChild(i):!m.bVisible&&c&&i.parentNode.removeChild(i);m.fnCreatedCell&&m.fnCreatedCell.call(a.oInstance,i,B(a,b,l),f,b,l)}r(a,"aoRowCreatedCallback",null,[j,f,b,g])}e.nTr.setAttribute("role","row")}function La(a,b){var c=b.nTr,d=b._aData;if(c){var e=a.rowIdFn(d);e&&(c.id=e);d.DT_RowClass&&(e=d.DT_RowClass.split(" "),b.__rowc=b.__rowc?qa(b.__rowc.concat(e)):e,h(c).removeClass(b.__rowc.join(" ")).addClass(d.DT_RowClass));
+d.DT_RowAttr&&h(c).attr(d.DT_RowAttr);d.DT_RowData&&h(c).data(d.DT_RowData)}}function lb(a){var b,c,d,e,f,g=a.nTHead,j=a.nTFoot,i=0===h("th, td",g).length,m=a.oClasses,l=a.aoColumns;i&&(e=h("<tr/>").appendTo(g));b=0;for(c=l.length;b<c;b++)f=l[b],d=h(f.nTh).addClass(f.sClass),i&&d.appendTo(e),a.oFeatures.bSort&&(d.addClass(f.sSortingClass),!1!==f.bSortable&&(d.attr("tabindex",a.iTabIndex).attr("aria-controls",a.sTableId),Ma(a,f.nTh,b))),f.sTitle!=d[0].innerHTML&&d.html(f.sTitle),Na(a,"header")(a,d,
+f,m);i&&ea(a.aoHeader,g);h(g).find(">tr").attr("role","row");h(g).find(">tr>th, >tr>td").addClass(m.sHeaderTH);h(j).find(">tr>th, >tr>td").addClass(m.sFooterTH);if(null!==j){a=a.aoFooter[0];b=0;for(c=a.length;b<c;b++)f=l[b],f.nTf=a[b].cell,f.sClass&&h(f.nTf).addClass(f.sClass)}}function fa(a,b,c){var d,e,f,g=[],j=[],i=a.aoColumns.length,m;if(b){c===k&&(c=!1);d=0;for(e=b.length;d<e;d++){g[d]=b[d].slice();g[d].nTr=b[d].nTr;for(f=i-1;0<=f;f--)!a.aoColumns[f].bVisible&&!c&&g[d].splice(f,1);j.push([])}d=
+0;for(e=g.length;d<e;d++){if(a=g[d].nTr)for(;f=a.firstChild;)a.removeChild(f);f=0;for(b=g[d].length;f<b;f++)if(m=i=1,j[d][f]===k){a.appendChild(g[d][f].cell);for(j[d][f]=1;g[d+i]!==k&&g[d][f].cell==g[d+i][f].cell;)j[d+i][f]=1,i++;for(;g[d][f+m]!==k&&g[d][f].cell==g[d][f+m].cell;){for(c=0;c<i;c++)j[d+c][f+m]=1;m++}h(g[d][f].cell).attr("rowspan",i).attr("colspan",m)}}}}function P(a){var b=r(a,"aoPreDrawCallback","preDraw",[a]);if(-1!==h.inArray(!1,b))C(a,!1);else{var b=[],c=0,d=a.asStripeClasses,e=
+d.length,f=a.oLanguage,g=a.iInitDisplayStart,j="ssp"==y(a),i=a.aiDisplay;a.bDrawing=!0;g!==k&&-1!==g&&(a._iDisplayStart=j?g:g>=a.fnRecordsDisplay()?0:g,a.iInitDisplayStart=-1);var g=a._iDisplayStart,m=a.fnDisplayEnd();if(a.bDeferLoading)a.bDeferLoading=!1,a.iDraw++,C(a,!1);else if(j){if(!a.bDestroying&&!mb(a))return}else a.iDraw++;if(0!==i.length){f=j?a.aoData.length:m;for(j=j?0:g;j<f;j++){var l=i[j],q=a.aoData[l];null===q.nTr&&Ha(a,l);var t=q.nTr;if(0!==e){var G=d[c%e];q._sRowStripe!=G&&(h(t).removeClass(q._sRowStripe).addClass(G),
+q._sRowStripe=G)}r(a,"aoRowCallback",null,[t,q._aData,c,j,l]);b.push(t);c++}}else c=f.sZeroRecords,1==a.iDraw&&"ajax"==y(a)?c=f.sLoadingRecords:f.sEmptyTable&&0===a.fnRecordsTotal()&&(c=f.sEmptyTable),b[0]=h("<tr/>",{"class":e?d[0]:""}).append(h("<td />",{valign:"top",colSpan:V(a),"class":a.oClasses.sRowEmpty}).html(c))[0];r(a,"aoHeaderCallback","header",[h(a.nTHead).children("tr")[0],Ka(a),g,m,i]);r(a,"aoFooterCallback","footer",[h(a.nTFoot).children("tr")[0],Ka(a),g,m,i]);d=h(a.nTBody);d.children().detach();
+d.append(h(b));r(a,"aoDrawCallback","draw",[a]);a.bSorted=!1;a.bFiltered=!1;a.bDrawing=!1}}function T(a,b){var c=a.oFeatures,d=c.bFilter;c.bSort&&nb(a);d?ga(a,a.oPreviousSearch):a.aiDisplay=a.aiDisplayMaster.slice();!0!==b&&(a._iDisplayStart=0);a._drawHold=b;P(a);a._drawHold=!1}function ob(a){var b=a.oClasses,c=h(a.nTable),c=h("<div/>").insertBefore(c),d=a.oFeatures,e=h("<div/>",{id:a.sTableId+"_wrapper","class":b.sWrapper+(a.nTFoot?"":" "+b.sNoFooter)});a.nHolding=c[0];a.nTableWrapper=e[0];a.nTableReinsertBefore=
+a.nTable.nextSibling;for(var f=a.sDom.split(""),g,j,i,m,l,q,k=0;k<f.length;k++){g=null;j=f[k];if("<"==j){i=h("<div/>")[0];m=f[k+1];if("'"==m||'"'==m){l="";for(q=2;f[k+q]!=m;)l+=f[k+q],q++;"H"==l?l=b.sJUIHeader:"F"==l&&(l=b.sJUIFooter);-1!=l.indexOf(".")?(m=l.split("."),i.id=m[0].substr(1,m[0].length-1),i.className=m[1]):"#"==l.charAt(0)?i.id=l.substr(1,l.length-1):i.className=l;k+=q}e.append(i);e=h(i)}else if(">"==j)e=e.parent();else if("l"==j&&d.bPaginate&&d.bLengthChange)g=pb(a);else if("f"==j&&
+d.bFilter)g=qb(a);else if("r"==j&&d.bProcessing)g=rb(a);else if("t"==j)g=sb(a);else if("i"==j&&d.bInfo)g=tb(a);else if("p"==j&&d.bPaginate)g=ub(a);else if(0!==n.ext.feature.length){i=n.ext.feature;q=0;for(m=i.length;q<m;q++)if(j==i[q].cFeature){g=i[q].fnInit(a);break}}g&&(i=a.aanFeatures,i[j]||(i[j]=[]),i[j].push(g),e.append(g))}c.replaceWith(e);a.nHolding=null}function ea(a,b){var c=h(b).children("tr"),d,e,f,g,j,i,m,l,q,k;a.splice(0,a.length);f=0;for(i=c.length;f<i;f++)a.push([]);f=0;for(i=c.length;f<
+i;f++){d=c[f];for(e=d.firstChild;e;){if("TD"==e.nodeName.toUpperCase()||"TH"==e.nodeName.toUpperCase()){l=1*e.getAttribute("colspan");q=1*e.getAttribute("rowspan");l=!l||0===l||1===l?1:l;q=!q||0===q||1===q?1:q;g=0;for(j=a[f];j[g];)g++;m=g;k=1===l?!0:!1;for(j=0;j<l;j++)for(g=0;g<q;g++)a[f+g][m+j]={cell:e,unique:k},a[f+g].nTr=d}e=e.nextSibling}}}function ra(a,b,c){var d=[];c||(c=a.aoHeader,b&&(c=[],ea(c,b)));for(var b=0,e=c.length;b<e;b++)for(var f=0,g=c[b].length;f<g;f++)if(c[b][f].unique&&(!d[f]||
+!a.bSortCellsTop))d[f]=c[b][f].cell;return d}function sa(a,b,c){r(a,"aoServerParams","serverParams",[b]);if(b&&h.isArray(b)){var d={},e=/(.*?)\[\]$/;h.each(b,function(a,b){var c=b.name.match(e);c?(c=c[0],d[c]||(d[c]=[]),d[c].push(b.value)):d[b.name]=b.value});b=d}var f,g=a.ajax,j=a.oInstance,i=function(b){r(a,null,"xhr",[a,b,a.jqXHR]);c(b)};if(h.isPlainObject(g)&&g.data){f=g.data;var m="function"===typeof f?f(b,a):f,b="function"===typeof f&&m?m:h.extend(!0,b,m);delete g.data}m={data:b,success:function(b){var c=
+b.error||b.sError;c&&K(a,0,c);a.json=b;i(b)},dataType:"json",cache:!1,type:a.sServerMethod,error:function(b,c){var d=r(a,null,"xhr",[a,null,a.jqXHR]);-1===h.inArray(!0,d)&&("parsererror"==c?K(a,0,"Invalid JSON response",1):4===b.readyState&&K(a,0,"Ajax error",7));C(a,!1)}};a.oAjaxData=b;r(a,null,"preXhr",[a,b]);a.fnServerData?a.fnServerData.call(j,a.sAjaxSource,h.map(b,function(a,b){return{name:b,value:a}}),i,a):a.sAjaxSource||"string"===typeof g?a.jqXHR=h.ajax(h.extend(m,{url:g||a.sAjaxSource})):
+"function"===typeof g?a.jqXHR=g.call(j,b,i,a):(a.jqXHR=h.ajax(h.extend(m,g)),g.data=f)}function mb(a){return a.bAjaxDataGet?(a.iDraw++,C(a,!0),sa(a,vb(a),function(b){wb(a,b)}),!1):!0}function vb(a){var b=a.aoColumns,c=b.length,d=a.oFeatures,e=a.oPreviousSearch,f=a.aoPreSearchCols,g,j=[],i,m,l,k=X(a);g=a._iDisplayStart;i=!1!==d.bPaginate?a._iDisplayLength:-1;var t=function(a,b){j.push({name:a,value:b})};t("sEcho",a.iDraw);t("iColumns",c);t("sColumns",D(b,"sName").join(","));t("iDisplayStart",g);t("iDisplayLength",
+i);var G={draw:a.iDraw,columns:[],order:[],start:g,length:i,search:{value:e.sSearch,regex:e.bRegex}};for(g=0;g<c;g++)m=b[g],l=f[g],i="function"==typeof m.mData?"function":m.mData,G.columns.push({data:i,name:m.sName,searchable:m.bSearchable,orderable:m.bSortable,search:{value:l.sSearch,regex:l.bRegex}}),t("mDataProp_"+g,i),d.bFilter&&(t("sSearch_"+g,l.sSearch),t("bRegex_"+g,l.bRegex),t("bSearchable_"+g,m.bSearchable)),d.bSort&&t("bSortable_"+g,m.bSortable);d.bFilter&&(t("sSearch",e.sSearch),t("bRegex",
+e.bRegex));d.bSort&&(h.each(k,function(a,b){G.order.push({column:b.col,dir:b.dir});t("iSortCol_"+a,b.col);t("sSortDir_"+a,b.dir)}),t("iSortingCols",k.length));b=n.ext.legacy.ajax;return null===b?a.sAjaxSource?j:G:b?j:G}function wb(a,b){var c=ta(a,b),d=b.sEcho!==k?b.sEcho:b.draw,e=b.iTotalRecords!==k?b.iTotalRecords:b.recordsTotal,f=b.iTotalDisplayRecords!==k?b.iTotalDisplayRecords:b.recordsFiltered;if(d){if(1*d<a.iDraw)return;a.iDraw=1*d}oa(a);a._iRecordsTotal=parseInt(e,10);a._iRecordsDisplay=parseInt(f,
+10);d=0;for(e=c.length;d<e;d++)O(a,c[d]);a.aiDisplay=a.aiDisplayMaster.slice();a.bAjaxDataGet=!1;P(a);a._bInitComplete||ua(a,b);a.bAjaxDataGet=!0;C(a,!1)}function ta(a,b){var c=h.isPlainObject(a.ajax)&&a.ajax.dataSrc!==k?a.ajax.dataSrc:a.sAjaxDataProp;return"data"===c?b.aaData||b[c]:""!==c?S(c)(b):b}function qb(a){var b=a.oClasses,c=a.sTableId,d=a.oLanguage,e=a.oPreviousSearch,f=a.aanFeatures,g='<input type="search" class="'+b.sFilterInput+'"/>',j=d.sSearch,j=j.match(/_INPUT_/)?j.replace("_INPUT_",
+g):j+g,b=h("<div/>",{id:!f.f?c+"_filter":null,"class":b.sFilter}).append(h("<label/>").append(j)),f=function(){var b=!this.value?"":this.value;b!=e.sSearch&&(ga(a,{sSearch:b,bRegex:e.bRegex,bSmart:e.bSmart,bCaseInsensitive:e.bCaseInsensitive}),a._iDisplayStart=0,P(a))},g=null!==a.searchDelay?a.searchDelay:"ssp"===y(a)?400:0,i=h("input",b).val(e.sSearch).attr("placeholder",d.sSearchPlaceholder).on("keyup.DT search.DT input.DT paste.DT cut.DT",g?Oa(f,g):f).on("keypress.DT",function(a){if(13==a.keyCode)return!1}).attr("aria-controls",
+c);h(a.nTable).on("search.dt.DT",function(b,c){if(a===c)try{i[0]!==H.activeElement&&i.val(e.sSearch)}catch(d){}});return b[0]}function ga(a,b,c){var d=a.oPreviousSearch,e=a.aoPreSearchCols,f=function(a){d.sSearch=a.sSearch;d.bRegex=a.bRegex;d.bSmart=a.bSmart;d.bCaseInsensitive=a.bCaseInsensitive};Ga(a);if("ssp"!=y(a)){xb(a,b.sSearch,c,b.bEscapeRegex!==k?!b.bEscapeRegex:b.bRegex,b.bSmart,b.bCaseInsensitive);f(b);for(b=0;b<e.length;b++)yb(a,e[b].sSearch,b,e[b].bEscapeRegex!==k?!e[b].bEscapeRegex:e[b].bRegex,
+e[b].bSmart,e[b].bCaseInsensitive);zb(a)}else f(b);a.bFiltered=!0;r(a,null,"search",[a])}function zb(a){for(var b=n.ext.search,c=a.aiDisplay,d,e,f=0,g=b.length;f<g;f++){for(var j=[],i=0,m=c.length;i<m;i++)e=c[i],d=a.aoData[e],b[f](a,d._aFilterData,e,d._aData,i)&&j.push(e);c.length=0;h.merge(c,j)}}function yb(a,b,c,d,e,f){if(""!==b){for(var g=[],j=a.aiDisplay,d=Pa(b,d,e,f),e=0;e<j.length;e++)b=a.aoData[j[e]]._aFilterData[c],d.test(b)&&g.push(j[e]);a.aiDisplay=g}}function xb(a,b,c,d,e,f){var d=Pa(b,
+d,e,f),f=a.oPreviousSearch.sSearch,g=a.aiDisplayMaster,j,e=[];0!==n.ext.search.length&&(c=!0);j=Ab(a);if(0>=b.length)a.aiDisplay=g.slice();else{if(j||c||f.length>b.length||0!==b.indexOf(f)||a.bSorted)a.aiDisplay=g.slice();b=a.aiDisplay;for(c=0;c<b.length;c++)d.test(a.aoData[b[c]]._sFilterRow)&&e.push(b[c]);a.aiDisplay=e}}function Pa(a,b,c,d){a=b?a:Qa(a);c&&(a="^(?=.*?"+h.map(a.match(/"[^"]+"|[^ ]+/g)||[""],function(a){if('"'===a.charAt(0))var b=a.match(/^"(.*)"$/),a=b?b[1]:a;return a.replace('"',
+"")}).join(")(?=.*?")+").*$");return RegExp(a,d?"i":"")}function Ab(a){var b=a.aoColumns,c,d,e,f,g,j,i,h,l=n.ext.type.search;c=!1;d=0;for(f=a.aoData.length;d<f;d++)if(h=a.aoData[d],!h._aFilterData){j=[];e=0;for(g=b.length;e<g;e++)c=b[e],c.bSearchable?(i=B(a,d,e,"filter"),l[c.sType]&&(i=l[c.sType](i)),null===i&&(i=""),"string"!==typeof i&&i.toString&&(i=i.toString())):i="",i.indexOf&&-1!==i.indexOf("&")&&(va.innerHTML=i,i=Wb?va.textContent:va.innerText),i.replace&&(i=i.replace(/[\r\n]/g,"")),j.push(i);
+h._aFilterData=j;h._sFilterRow=j.join("  ");c=!0}return c}function Bb(a){return{search:a.sSearch,smart:a.bSmart,regex:a.bRegex,caseInsensitive:a.bCaseInsensitive}}function Cb(a){return{sSearch:a.search,bSmart:a.smart,bRegex:a.regex,bCaseInsensitive:a.caseInsensitive}}function tb(a){var b=a.sTableId,c=a.aanFeatures.i,d=h("<div/>",{"class":a.oClasses.sInfo,id:!c?b+"_info":null});c||(a.aoDrawCallback.push({fn:Db,sName:"information"}),d.attr("role","status").attr("aria-live","polite"),h(a.nTable).attr("aria-describedby",
+b+"_info"));return d[0]}function Db(a){var b=a.aanFeatures.i;if(0!==b.length){var c=a.oLanguage,d=a._iDisplayStart+1,e=a.fnDisplayEnd(),f=a.fnRecordsTotal(),g=a.fnRecordsDisplay(),j=g?c.sInfo:c.sInfoEmpty;g!==f&&(j+=" "+c.sInfoFiltered);j+=c.sInfoPostFix;j=Eb(a,j);c=c.fnInfoCallback;null!==c&&(j=c.call(a.oInstance,a,d,e,f,g,j));h(b).html(j)}}function Eb(a,b){var c=a.fnFormatNumber,d=a._iDisplayStart+1,e=a._iDisplayLength,f=a.fnRecordsDisplay(),g=-1===e;return b.replace(/_START_/g,c.call(a,d)).replace(/_END_/g,
+c.call(a,a.fnDisplayEnd())).replace(/_MAX_/g,c.call(a,a.fnRecordsTotal())).replace(/_TOTAL_/g,c.call(a,f)).replace(/_PAGE_/g,c.call(a,g?1:Math.ceil(d/e))).replace(/_PAGES_/g,c.call(a,g?1:Math.ceil(f/e)))}function ha(a){var b,c,d=a.iInitDisplayStart,e=a.aoColumns,f;c=a.oFeatures;var g=a.bDeferLoading;if(a.bInitialised){ob(a);lb(a);fa(a,a.aoHeader);fa(a,a.aoFooter);C(a,!0);c.bAutoWidth&&Fa(a);b=0;for(c=e.length;b<c;b++)f=e[b],f.sWidth&&(f.nTh.style.width=v(f.sWidth));r(a,null,"preInit",[a]);T(a);e=
+y(a);if("ssp"!=e||g)"ajax"==e?sa(a,[],function(c){var f=ta(a,c);for(b=0;b<f.length;b++)O(a,f[b]);a.iInitDisplayStart=d;T(a);C(a,!1);ua(a,c)},a):(C(a,!1),ua(a))}else setTimeout(function(){ha(a)},200)}function ua(a,b){a._bInitComplete=!0;(b||a.oInit.aaData)&&$(a);r(a,null,"plugin-init",[a,b]);r(a,"aoInitComplete","init",[a,b])}function Ra(a,b){var c=parseInt(b,10);a._iDisplayLength=c;Sa(a);r(a,null,"length",[a,c])}function pb(a){for(var b=a.oClasses,c=a.sTableId,d=a.aLengthMenu,e=h.isArray(d[0]),f=
+e?d[0]:d,d=e?d[1]:d,e=h("<select/>",{name:c+"_length","aria-controls":c,"class":b.sLengthSelect}),g=0,j=f.length;g<j;g++)e[0][g]=new Option("number"===typeof d[g]?a.fnFormatNumber(d[g]):d[g],f[g]);var i=h("<div><label/></div>").addClass(b.sLength);a.aanFeatures.l||(i[0].id=c+"_length");i.children().append(a.oLanguage.sLengthMenu.replace("_MENU_",e[0].outerHTML));h("select",i).val(a._iDisplayLength).on("change.DT",function(){Ra(a,h(this).val());P(a)});h(a.nTable).on("length.dt.DT",function(b,c,d){a===
+c&&h("select",i).val(d)});return i[0]}function ub(a){var b=a.sPaginationType,c=n.ext.pager[b],d="function"===typeof c,e=function(a){P(a)},b=h("<div/>").addClass(a.oClasses.sPaging+b)[0],f=a.aanFeatures;d||c.fnInit(a,b,e);f.p||(b.id=a.sTableId+"_paginate",a.aoDrawCallback.push({fn:function(a){if(d){var b=a._iDisplayStart,i=a._iDisplayLength,h=a.fnRecordsDisplay(),l=-1===i,b=l?0:Math.ceil(b/i),i=l?1:Math.ceil(h/i),h=c(b,i),k,l=0;for(k=f.p.length;l<k;l++)Na(a,"pageButton")(a,f.p[l],l,h,b,i)}else c.fnUpdate(a,
+e)},sName:"pagination"}));return b}function Ta(a,b,c){var d=a._iDisplayStart,e=a._iDisplayLength,f=a.fnRecordsDisplay();0===f||-1===e?d=0:"number"===typeof b?(d=b*e,d>f&&(d=0)):"first"==b?d=0:"previous"==b?(d=0<=e?d-e:0,0>d&&(d=0)):"next"==b?d+e<f&&(d+=e):"last"==b?d=Math.floor((f-1)/e)*e:K(a,0,"Unknown paging action: "+b,5);b=a._iDisplayStart!==d;a._iDisplayStart=d;b&&(r(a,null,"page",[a]),c&&P(a));return b}function rb(a){return h("<div/>",{id:!a.aanFeatures.r?a.sTableId+"_processing":null,"class":a.oClasses.sProcessing}).html(a.oLanguage.sProcessing).insertBefore(a.nTable)[0]}
+function C(a,b){a.oFeatures.bProcessing&&h(a.aanFeatures.r).css("display",b?"block":"none");r(a,null,"processing",[a,b])}function sb(a){var b=h(a.nTable);b.attr("role","grid");var c=a.oScroll;if(""===c.sX&&""===c.sY)return a.nTable;var d=c.sX,e=c.sY,f=a.oClasses,g=b.children("caption"),j=g.length?g[0]._captionSide:null,i=h(b[0].cloneNode(!1)),m=h(b[0].cloneNode(!1)),l=b.children("tfoot");l.length||(l=null);i=h("<div/>",{"class":f.sScrollWrapper}).append(h("<div/>",{"class":f.sScrollHead}).css({overflow:"hidden",
+position:"relative",border:0,width:d?!d?null:v(d):"100%"}).append(h("<div/>",{"class":f.sScrollHeadInner}).css({"box-sizing":"content-box",width:c.sXInner||"100%"}).append(i.removeAttr("id").css("margin-left",0).append("top"===j?g:null).append(b.children("thead"))))).append(h("<div/>",{"class":f.sScrollBody}).css({position:"relative",overflow:"auto",width:!d?null:v(d)}).append(b));l&&i.append(h("<div/>",{"class":f.sScrollFoot}).css({overflow:"hidden",border:0,width:d?!d?null:v(d):"100%"}).append(h("<div/>",
+{"class":f.sScrollFootInner}).append(m.removeAttr("id").css("margin-left",0).append("bottom"===j?g:null).append(b.children("tfoot")))));var b=i.children(),k=b[0],f=b[1],t=l?b[2]:null;if(d)h(f).on("scroll.DT",function(){var a=this.scrollLeft;k.scrollLeft=a;l&&(t.scrollLeft=a)});h(f).css(e&&c.bCollapse?"max-height":"height",e);a.nScrollHead=k;a.nScrollBody=f;a.nScrollFoot=t;a.aoDrawCallback.push({fn:la,sName:"scrolling"});return i[0]}function la(a){var b=a.oScroll,c=b.sX,d=b.sXInner,e=b.sY,b=b.iBarWidth,
+f=h(a.nScrollHead),g=f[0].style,j=f.children("div"),i=j[0].style,m=j.children("table"),j=a.nScrollBody,l=h(j),q=j.style,t=h(a.nScrollFoot).children("div"),n=t.children("table"),o=h(a.nTHead),p=h(a.nTable),s=p[0],r=s.style,u=a.nTFoot?h(a.nTFoot):null,x=a.oBrowser,U=x.bScrollOversize,Xb=D(a.aoColumns,"nTh"),Q,L,R,w,Ua=[],y=[],z=[],A=[],B,C=function(a){a=a.style;a.paddingTop="0";a.paddingBottom="0";a.borderTopWidth="0";a.borderBottomWidth="0";a.height=0};L=j.scrollHeight>j.clientHeight;if(a.scrollBarVis!==
+L&&a.scrollBarVis!==k)a.scrollBarVis=L,$(a);else{a.scrollBarVis=L;p.children("thead, tfoot").remove();u&&(R=u.clone().prependTo(p),Q=u.find("tr"),R=R.find("tr"));w=o.clone().prependTo(p);o=o.find("tr");L=w.find("tr");w.find("th, td").removeAttr("tabindex");c||(q.width="100%",f[0].style.width="100%");h.each(ra(a,w),function(b,c){B=aa(a,b);c.style.width=a.aoColumns[B].sWidth});u&&I(function(a){a.style.width=""},R);f=p.outerWidth();if(""===c){r.width="100%";if(U&&(p.find("tbody").height()>j.offsetHeight||
+"scroll"==l.css("overflow-y")))r.width=v(p.outerWidth()-b);f=p.outerWidth()}else""!==d&&(r.width=v(d),f=p.outerWidth());I(C,L);I(function(a){z.push(a.innerHTML);Ua.push(v(h(a).css("width")))},L);I(function(a,b){if(h.inArray(a,Xb)!==-1)a.style.width=Ua[b]},o);h(L).height(0);u&&(I(C,R),I(function(a){A.push(a.innerHTML);y.push(v(h(a).css("width")))},R),I(function(a,b){a.style.width=y[b]},Q),h(R).height(0));I(function(a,b){a.innerHTML='<div class="dataTables_sizing">'+z[b]+"</div>";a.childNodes[0].style.height=
+"0";a.childNodes[0].style.overflow="hidden";a.style.width=Ua[b]},L);u&&I(function(a,b){a.innerHTML='<div class="dataTables_sizing">'+A[b]+"</div>";a.childNodes[0].style.height="0";a.childNodes[0].style.overflow="hidden";a.style.width=y[b]},R);if(p.outerWidth()<f){Q=j.scrollHeight>j.offsetHeight||"scroll"==l.css("overflow-y")?f+b:f;if(U&&(j.scrollHeight>j.offsetHeight||"scroll"==l.css("overflow-y")))r.width=v(Q-b);(""===c||""!==d)&&K(a,1,"Possible column misalignment",6)}else Q="100%";q.width=v(Q);
+g.width=v(Q);u&&(a.nScrollFoot.style.width=v(Q));!e&&U&&(q.height=v(s.offsetHeight+b));c=p.outerWidth();m[0].style.width=v(c);i.width=v(c);d=p.height()>j.clientHeight||"scroll"==l.css("overflow-y");e="padding"+(x.bScrollbarLeft?"Left":"Right");i[e]=d?b+"px":"0px";u&&(n[0].style.width=v(c),t[0].style.width=v(c),t[0].style[e]=d?b+"px":"0px");p.children("colgroup").insertBefore(p.children("thead"));l.scroll();if((a.bSorted||a.bFiltered)&&!a._drawHold)j.scrollTop=0}}function I(a,b,c){for(var d=0,e=0,
+f=b.length,g,j;e<f;){g=b[e].firstChild;for(j=c?c[e].firstChild:null;g;)1===g.nodeType&&(c?a(g,j,d):a(g,d),d++),g=g.nextSibling,j=c?j.nextSibling:null;e++}}function Fa(a){var b=a.nTable,c=a.aoColumns,d=a.oScroll,e=d.sY,f=d.sX,g=d.sXInner,j=c.length,i=ma(a,"bVisible"),m=h("th",a.nTHead),l=b.getAttribute("width"),k=b.parentNode,t=!1,n,o,p=a.oBrowser,d=p.bScrollOversize;(n=b.style.width)&&-1!==n.indexOf("%")&&(l=n);for(n=0;n<i.length;n++)o=c[i[n]],null!==o.sWidth&&(o.sWidth=Fb(o.sWidthOrig,k),t=!0);if(d||
+!t&&!f&&!e&&j==V(a)&&j==m.length)for(n=0;n<j;n++)i=aa(a,n),null!==i&&(c[i].sWidth=v(m.eq(n).width()));else{j=h(b).clone().css("visibility","hidden").removeAttr("id");j.find("tbody tr").remove();var s=h("<tr/>").appendTo(j.find("tbody"));j.find("thead, tfoot").remove();j.append(h(a.nTHead).clone()).append(h(a.nTFoot).clone());j.find("tfoot th, tfoot td").css("width","");m=ra(a,j.find("thead")[0]);for(n=0;n<i.length;n++)o=c[i[n]],m[n].style.width=null!==o.sWidthOrig&&""!==o.sWidthOrig?v(o.sWidthOrig):
+"",o.sWidthOrig&&f&&h(m[n]).append(h("<div/>").css({width:o.sWidthOrig,margin:0,padding:0,border:0,height:1}));if(a.aoData.length)for(n=0;n<i.length;n++)t=i[n],o=c[t],h(Gb(a,t)).clone(!1).append(o.sContentPadding).appendTo(s);h("[name]",j).removeAttr("name");o=h("<div/>").css(f||e?{position:"absolute",top:0,left:0,height:1,right:0,overflow:"hidden"}:{}).append(j).appendTo(k);f&&g?j.width(g):f?(j.css("width","auto"),j.removeAttr("width"),j.width()<k.clientWidth&&l&&j.width(k.clientWidth)):e?j.width(k.clientWidth):
+l&&j.width(l);for(n=e=0;n<i.length;n++)k=h(m[n]),g=k.outerWidth()-k.width(),k=p.bBounding?Math.ceil(m[n].getBoundingClientRect().width):k.outerWidth(),e+=k,c[i[n]].sWidth=v(k-g);b.style.width=v(e);o.remove()}l&&(b.style.width=v(l));if((l||f)&&!a._reszEvt)b=function(){h(E).on("resize.DT-"+a.sInstance,Oa(function(){$(a)}))},d?setTimeout(b,1E3):b(),a._reszEvt=!0}function Fb(a,b){if(!a)return 0;var c=h("<div/>").css("width",v(a)).appendTo(b||H.body),d=c[0].offsetWidth;c.remove();return d}function Gb(a,
+b){var c=Hb(a,b);if(0>c)return null;var d=a.aoData[c];return!d.nTr?h("<td/>").html(B(a,c,b,"display"))[0]:d.anCells[b]}function Hb(a,b){for(var c,d=-1,e=-1,f=0,g=a.aoData.length;f<g;f++)c=B(a,f,b,"display")+"",c=c.replace(Yb,""),c=c.replace(/&nbsp;/g," "),c.length>d&&(d=c.length,e=f);return e}function v(a){return null===a?"0px":"number"==typeof a?0>a?"0px":a+"px":a.match(/\d$/)?a+"px":a}function X(a){var b,c,d=[],e=a.aoColumns,f,g,j,i;b=a.aaSortingFixed;c=h.isPlainObject(b);var m=[];f=function(a){a.length&&
+!h.isArray(a[0])?m.push(a):h.merge(m,a)};h.isArray(b)&&f(b);c&&b.pre&&f(b.pre);f(a.aaSorting);c&&b.post&&f(b.post);for(a=0;a<m.length;a++){i=m[a][0];f=e[i].aDataSort;b=0;for(c=f.length;b<c;b++)g=f[b],j=e[g].sType||"string",m[a]._idx===k&&(m[a]._idx=h.inArray(m[a][1],e[g].asSorting)),d.push({src:i,col:g,dir:m[a][1],index:m[a]._idx,type:j,formatter:n.ext.type.order[j+"-pre"]})}return d}function nb(a){var b,c,d=[],e=n.ext.type.order,f=a.aoData,g=0,j,i=a.aiDisplayMaster,h;Ga(a);h=X(a);b=0;for(c=h.length;b<
+c;b++)j=h[b],j.formatter&&g++,Ib(a,j.col);if("ssp"!=y(a)&&0!==h.length){b=0;for(c=i.length;b<c;b++)d[i[b]]=b;g===h.length?i.sort(function(a,b){var c,e,g,j,i=h.length,k=f[a]._aSortData,n=f[b]._aSortData;for(g=0;g<i;g++)if(j=h[g],c=k[j.col],e=n[j.col],c=c<e?-1:c>e?1:0,0!==c)return"asc"===j.dir?c:-c;c=d[a];e=d[b];return c<e?-1:c>e?1:0}):i.sort(function(a,b){var c,g,j,i,k=h.length,n=f[a]._aSortData,o=f[b]._aSortData;for(j=0;j<k;j++)if(i=h[j],c=n[i.col],g=o[i.col],i=e[i.type+"-"+i.dir]||e["string-"+i.dir],
+c=i(c,g),0!==c)return c;c=d[a];g=d[b];return c<g?-1:c>g?1:0})}a.bSorted=!0}function Jb(a){for(var b,c,d=a.aoColumns,e=X(a),a=a.oLanguage.oAria,f=0,g=d.length;f<g;f++){c=d[f];var j=c.asSorting;b=c.sTitle.replace(/<.*?>/g,"");var i=c.nTh;i.removeAttribute("aria-sort");c.bSortable&&(0<e.length&&e[0].col==f?(i.setAttribute("aria-sort","asc"==e[0].dir?"ascending":"descending"),c=j[e[0].index+1]||j[0]):c=j[0],b+="asc"===c?a.sSortAscending:a.sSortDescending);i.setAttribute("aria-label",b)}}function Va(a,
+b,c,d){var e=a.aaSorting,f=a.aoColumns[b].asSorting,g=function(a,b){var c=a._idx;c===k&&(c=h.inArray(a[1],f));return c+1<f.length?c+1:b?null:0};"number"===typeof e[0]&&(e=a.aaSorting=[e]);c&&a.oFeatures.bSortMulti?(c=h.inArray(b,D(e,"0")),-1!==c?(b=g(e[c],!0),null===b&&1===e.length&&(b=0),null===b?e.splice(c,1):(e[c][1]=f[b],e[c]._idx=b)):(e.push([b,f[0],0]),e[e.length-1]._idx=0)):e.length&&e[0][0]==b?(b=g(e[0]),e.length=1,e[0][1]=f[b],e[0]._idx=b):(e.length=0,e.push([b,f[0]]),e[0]._idx=0);T(a);"function"==
+typeof d&&d(a)}function Ma(a,b,c,d){var e=a.aoColumns[c];Wa(b,{},function(b){!1!==e.bSortable&&(a.oFeatures.bProcessing?(C(a,!0),setTimeout(function(){Va(a,c,b.shiftKey,d);"ssp"!==y(a)&&C(a,!1)},0)):Va(a,c,b.shiftKey,d))})}function wa(a){var b=a.aLastSort,c=a.oClasses.sSortColumn,d=X(a),e=a.oFeatures,f,g;if(e.bSort&&e.bSortClasses){e=0;for(f=b.length;e<f;e++)g=b[e].src,h(D(a.aoData,"anCells",g)).removeClass(c+(2>e?e+1:3));e=0;for(f=d.length;e<f;e++)g=d[e].src,h(D(a.aoData,"anCells",g)).addClass(c+
+(2>e?e+1:3))}a.aLastSort=d}function Ib(a,b){var c=a.aoColumns[b],d=n.ext.order[c.sSortDataType],e;d&&(e=d.call(a.oInstance,a,b,ba(a,b)));for(var f,g=n.ext.type.order[c.sType+"-pre"],j=0,i=a.aoData.length;j<i;j++)if(c=a.aoData[j],c._aSortData||(c._aSortData=[]),!c._aSortData[b]||d)f=d?e[j]:B(a,j,b,"sort"),c._aSortData[b]=g?g(f):f}function xa(a){if(a.oFeatures.bStateSave&&!a.bDestroying){var b={time:+new Date,start:a._iDisplayStart,length:a._iDisplayLength,order:h.extend(!0,[],a.aaSorting),search:Bb(a.oPreviousSearch),
+columns:h.map(a.aoColumns,function(b,d){return{visible:b.bVisible,search:Bb(a.aoPreSearchCols[d])}})};r(a,"aoStateSaveParams","stateSaveParams",[a,b]);a.oSavedState=b;a.fnStateSaveCallback.call(a.oInstance,a,b)}}function Kb(a,b,c){var d,e,f=a.aoColumns,b=function(b){if(b&&b.time){var g=r(a,"aoStateLoadParams","stateLoadParams",[a,b]);if(-1===h.inArray(!1,g)&&(g=a.iStateDuration,!(0<g&&b.time<+new Date-1E3*g)&&!(b.columns&&f.length!==b.columns.length))){a.oLoadedState=h.extend(!0,{},b);b.start!==k&&
+(a._iDisplayStart=b.start,a.iInitDisplayStart=b.start);b.length!==k&&(a._iDisplayLength=b.length);b.order!==k&&(a.aaSorting=[],h.each(b.order,function(b,c){a.aaSorting.push(c[0]>=f.length?[0,c[1]]:c)}));b.search!==k&&h.extend(a.oPreviousSearch,Cb(b.search));if(b.columns){d=0;for(e=b.columns.length;d<e;d++)g=b.columns[d],g.visible!==k&&(f[d].bVisible=g.visible),g.search!==k&&h.extend(a.aoPreSearchCols[d],Cb(g.search))}r(a,"aoStateLoaded","stateLoaded",[a,b])}}c()};if(a.oFeatures.bStateSave){var g=
+a.fnStateLoadCallback.call(a.oInstance,a,b);g!==k&&b(g)}else c()}function ya(a){var b=n.settings,a=h.inArray(a,D(b,"nTable"));return-1!==a?b[a]:null}function K(a,b,c,d){c="DataTables warning: "+(a?"table id="+a.sTableId+" - ":"")+c;d&&(c+=". For more information about this error, please see http://datatables.net/tn/"+d);if(b)E.console&&console.log&&console.log(c);else if(b=n.ext,b=b.sErrMode||b.errMode,a&&r(a,null,"error",[a,d,c]),"alert"==b)alert(c);else{if("throw"==b)throw Error(c);"function"==
+typeof b&&b(a,d,c)}}function F(a,b,c,d){h.isArray(c)?h.each(c,function(c,d){h.isArray(d)?F(a,b,d[0],d[1]):F(a,b,d)}):(d===k&&(d=c),b[c]!==k&&(a[d]=b[c]))}function Xa(a,b,c){var d,e;for(e in b)b.hasOwnProperty(e)&&(d=b[e],h.isPlainObject(d)?(h.isPlainObject(a[e])||(a[e]={}),h.extend(!0,a[e],d)):a[e]=c&&"data"!==e&&"aaData"!==e&&h.isArray(d)?d.slice():d);return a}function Wa(a,b,c){h(a).on("click.DT",b,function(b){h(a).blur();c(b)}).on("keypress.DT",b,function(a){13===a.which&&(a.preventDefault(),c(a))}).on("selectstart.DT",
+function(){return!1})}function z(a,b,c,d){c&&a[b].push({fn:c,sName:d})}function r(a,b,c,d){var e=[];b&&(e=h.map(a[b].slice().reverse(),function(b){return b.fn.apply(a.oInstance,d)}));null!==c&&(b=h.Event(c+".dt"),h(a.nTable).trigger(b,d),e.push(b.result));return e}function Sa(a){var b=a._iDisplayStart,c=a.fnDisplayEnd(),d=a._iDisplayLength;b>=c&&(b=c-d);b-=b%d;if(-1===d||0>b)b=0;a._iDisplayStart=b}function Na(a,b){var c=a.renderer,d=n.ext.renderer[b];return h.isPlainObject(c)&&c[b]?d[c[b]]||d._:"string"===
+typeof c?d[c]||d._:d._}function y(a){return a.oFeatures.bServerSide?"ssp":a.ajax||a.sAjaxSource?"ajax":"dom"}function ia(a,b){var c=[],c=Lb.numbers_length,d=Math.floor(c/2);b<=c?c=Y(0,b):a<=d?(c=Y(0,c-2),c.push("ellipsis"),c.push(b-1)):(a>=b-1-d?c=Y(b-(c-2),b):(c=Y(a-d+2,a+d-1),c.push("ellipsis"),c.push(b-1)),c.splice(0,0,"ellipsis"),c.splice(0,0,0));c.DT_el="span";return c}function Da(a){h.each({num:function(b){return za(b,a)},"num-fmt":function(b){return za(b,a,Ya)},"html-num":function(b){return za(b,
+a,Aa)},"html-num-fmt":function(b){return za(b,a,Aa,Ya)}},function(b,c){x.type.order[b+a+"-pre"]=c;b.match(/^html\-/)&&(x.type.search[b+a]=x.type.search.html)})}function Mb(a){return function(){var b=[ya(this[n.ext.iApiIndex])].concat(Array.prototype.slice.call(arguments));return n.ext.internal[a].apply(this,b)}}var n=function(a){this.$=function(a,b){return this.api(!0).$(a,b)};this._=function(a,b){return this.api(!0).rows(a,b).data()};this.api=function(a){return a?new s(ya(this[x.iApiIndex])):new s(this)};
+this.fnAddData=function(a,b){var c=this.api(!0),d=h.isArray(a)&&(h.isArray(a[0])||h.isPlainObject(a[0]))?c.rows.add(a):c.row.add(a);(b===k||b)&&c.draw();return d.flatten().toArray()};this.fnAdjustColumnSizing=function(a){var b=this.api(!0).columns.adjust(),c=b.settings()[0],d=c.oScroll;a===k||a?b.draw(!1):(""!==d.sX||""!==d.sY)&&la(c)};this.fnClearTable=function(a){var b=this.api(!0).clear();(a===k||a)&&b.draw()};this.fnClose=function(a){this.api(!0).row(a).child.hide()};this.fnDeleteRow=function(a,
+b,c){var d=this.api(!0),a=d.rows(a),e=a.settings()[0],h=e.aoData[a[0][0]];a.remove();b&&b.call(this,e,h);(c===k||c)&&d.draw();return h};this.fnDestroy=function(a){this.api(!0).destroy(a)};this.fnDraw=function(a){this.api(!0).draw(a)};this.fnFilter=function(a,b,c,d,e,h){e=this.api(!0);null===b||b===k?e.search(a,c,d,h):e.column(b).search(a,c,d,h);e.draw()};this.fnGetData=function(a,b){var c=this.api(!0);if(a!==k){var d=a.nodeName?a.nodeName.toLowerCase():"";return b!==k||"td"==d||"th"==d?c.cell(a,b).data():
+c.row(a).data()||null}return c.data().toArray()};this.fnGetNodes=function(a){var b=this.api(!0);return a!==k?b.row(a).node():b.rows().nodes().flatten().toArray()};this.fnGetPosition=function(a){var b=this.api(!0),c=a.nodeName.toUpperCase();return"TR"==c?b.row(a).index():"TD"==c||"TH"==c?(a=b.cell(a).index(),[a.row,a.columnVisible,a.column]):null};this.fnIsOpen=function(a){return this.api(!0).row(a).child.isShown()};this.fnOpen=function(a,b,c){return this.api(!0).row(a).child(b,c).show().child()[0]};
+this.fnPageChange=function(a,b){var c=this.api(!0).page(a);(b===k||b)&&c.draw(!1)};this.fnSetColumnVis=function(a,b,c){a=this.api(!0).column(a).visible(b);(c===k||c)&&a.columns.adjust().draw()};this.fnSettings=function(){return ya(this[x.iApiIndex])};this.fnSort=function(a){this.api(!0).order(a).draw()};this.fnSortListener=function(a,b,c){this.api(!0).order.listener(a,b,c)};this.fnUpdate=function(a,b,c,d,e){var h=this.api(!0);c===k||null===c?h.row(b).data(a):h.cell(b,c).data(a);(e===k||e)&&h.columns.adjust();
+(d===k||d)&&h.draw();return 0};this.fnVersionCheck=x.fnVersionCheck;var b=this,c=a===k,d=this.length;c&&(a={});this.oApi=this.internal=x.internal;for(var e in n.ext.internal)e&&(this[e]=Mb(e));this.each(function(){var e={},g=1<d?Xa(e,a,!0):a,j=0,i,e=this.getAttribute("id"),m=!1,l=n.defaults,q=h(this);if("table"!=this.nodeName.toLowerCase())K(null,0,"Non-table node initialisation ("+this.nodeName+")",2);else{fb(l);gb(l.column);J(l,l,!0);J(l.column,l.column,!0);J(l,h.extend(g,q.data()));var t=n.settings,
+j=0;for(i=t.length;j<i;j++){var o=t[j];if(o.nTable==this||o.nTHead&&o.nTHead.parentNode==this||o.nTFoot&&o.nTFoot.parentNode==this){var s=g.bRetrieve!==k?g.bRetrieve:l.bRetrieve;if(c||s)return o.oInstance;if(g.bDestroy!==k?g.bDestroy:l.bDestroy){o.oInstance.fnDestroy();break}else{K(o,0,"Cannot reinitialise DataTable",3);return}}if(o.sTableId==this.id){t.splice(j,1);break}}if(null===e||""===e)this.id=e="DataTables_Table_"+n.ext._unique++;var p=h.extend(!0,{},n.models.oSettings,{sDestroyWidth:q[0].style.width,
+sInstance:e,sTableId:e});p.nTable=this;p.oApi=b.internal;p.oInit=g;t.push(p);p.oInstance=1===b.length?b:q.dataTable();fb(g);Ca(g.oLanguage);g.aLengthMenu&&!g.iDisplayLength&&(g.iDisplayLength=h.isArray(g.aLengthMenu[0])?g.aLengthMenu[0][0]:g.aLengthMenu[0]);g=Xa(h.extend(!0,{},l),g);F(p.oFeatures,g,"bPaginate bLengthChange bFilter bSort bSortMulti bInfo bProcessing bAutoWidth bSortClasses bServerSide bDeferRender".split(" "));F(p,g,["asStripeClasses","ajax","fnServerData","fnFormatNumber","sServerMethod",
+"aaSorting","aaSortingFixed","aLengthMenu","sPaginationType","sAjaxSource","sAjaxDataProp","iStateDuration","sDom","bSortCellsTop","iTabIndex","fnStateLoadCallback","fnStateSaveCallback","renderer","searchDelay","rowId",["iCookieDuration","iStateDuration"],["oSearch","oPreviousSearch"],["aoSearchCols","aoPreSearchCols"],["iDisplayLength","_iDisplayLength"]]);F(p.oScroll,g,[["sScrollX","sX"],["sScrollXInner","sXInner"],["sScrollY","sY"],["bScrollCollapse","bCollapse"]]);F(p.oLanguage,g,"fnInfoCallback");
+z(p,"aoDrawCallback",g.fnDrawCallback,"user");z(p,"aoServerParams",g.fnServerParams,"user");z(p,"aoStateSaveParams",g.fnStateSaveParams,"user");z(p,"aoStateLoadParams",g.fnStateLoadParams,"user");z(p,"aoStateLoaded",g.fnStateLoaded,"user");z(p,"aoRowCallback",g.fnRowCallback,"user");z(p,"aoRowCreatedCallback",g.fnCreatedRow,"user");z(p,"aoHeaderCallback",g.fnHeaderCallback,"user");z(p,"aoFooterCallback",g.fnFooterCallback,"user");z(p,"aoInitComplete",g.fnInitComplete,"user");z(p,"aoPreDrawCallback",
+g.fnPreDrawCallback,"user");p.rowIdFn=S(g.rowId);hb(p);var u=p.oClasses;h.extend(u,n.ext.classes,g.oClasses);q.addClass(u.sTable);p.iInitDisplayStart===k&&(p.iInitDisplayStart=g.iDisplayStart,p._iDisplayStart=g.iDisplayStart);null!==g.iDeferLoading&&(p.bDeferLoading=!0,e=h.isArray(g.iDeferLoading),p._iRecordsDisplay=e?g.iDeferLoading[0]:g.iDeferLoading,p._iRecordsTotal=e?g.iDeferLoading[1]:g.iDeferLoading);var v=p.oLanguage;h.extend(!0,v,g.oLanguage);v.sUrl&&(h.ajax({dataType:"json",url:v.sUrl,success:function(a){Ca(a);
+J(l.oLanguage,a);h.extend(true,v,a);ha(p)},error:function(){ha(p)}}),m=!0);null===g.asStripeClasses&&(p.asStripeClasses=[u.sStripeOdd,u.sStripeEven]);var e=p.asStripeClasses,x=q.children("tbody").find("tr").eq(0);-1!==h.inArray(!0,h.map(e,function(a){return x.hasClass(a)}))&&(h("tbody tr",this).removeClass(e.join(" ")),p.asDestroyStripes=e.slice());e=[];t=this.getElementsByTagName("thead");0!==t.length&&(ea(p.aoHeader,t[0]),e=ra(p));if(null===g.aoColumns){t=[];j=0;for(i=e.length;j<i;j++)t.push(null)}else t=
+g.aoColumns;j=0;for(i=t.length;j<i;j++)Ea(p,e?e[j]:null);jb(p,g.aoColumnDefs,t,function(a,b){ka(p,a,b)});if(x.length){var w=function(a,b){return a.getAttribute("data-"+b)!==null?b:null};h(x[0]).children("th, td").each(function(a,b){var c=p.aoColumns[a];if(c.mData===a){var d=w(b,"sort")||w(b,"order"),e=w(b,"filter")||w(b,"search");if(d!==null||e!==null){c.mData={_:a+".display",sort:d!==null?a+".@data-"+d:k,type:d!==null?a+".@data-"+d:k,filter:e!==null?a+".@data-"+e:k};ka(p,a)}}})}var U=p.oFeatures,
+e=function(){if(g.aaSorting===k){var a=p.aaSorting;j=0;for(i=a.length;j<i;j++)a[j][1]=p.aoColumns[j].asSorting[0]}wa(p);U.bSort&&z(p,"aoDrawCallback",function(){if(p.bSorted){var a=X(p),b={};h.each(a,function(a,c){b[c.src]=c.dir});r(p,null,"order",[p,a,b]);Jb(p)}});z(p,"aoDrawCallback",function(){(p.bSorted||y(p)==="ssp"||U.bDeferRender)&&wa(p)},"sc");var a=q.children("caption").each(function(){this._captionSide=h(this).css("caption-side")}),b=q.children("thead");b.length===0&&(b=h("<thead/>").appendTo(q));
+p.nTHead=b[0];b=q.children("tbody");b.length===0&&(b=h("<tbody/>").appendTo(q));p.nTBody=b[0];b=q.children("tfoot");if(b.length===0&&a.length>0&&(p.oScroll.sX!==""||p.oScroll.sY!==""))b=h("<tfoot/>").appendTo(q);if(b.length===0||b.children().length===0)q.addClass(u.sNoFooter);else if(b.length>0){p.nTFoot=b[0];ea(p.aoFooter,p.nTFoot)}if(g.aaData)for(j=0;j<g.aaData.length;j++)O(p,g.aaData[j]);else(p.bDeferLoading||y(p)=="dom")&&na(p,h(p.nTBody).children("tr"));p.aiDisplay=p.aiDisplayMaster.slice();
+p.bInitialised=true;m===false&&ha(p)};g.bStateSave?(U.bStateSave=!0,z(p,"aoDrawCallback",xa,"state_save"),Kb(p,g,e)):e()}});b=null;return this},x,s,o,u,Za={},Nb=/[\r\n]/g,Aa=/<.*?>/g,Zb=/^\d{2,4}[\.\/\-]\d{1,2}[\.\/\-]\d{1,2}([T ]{1}\d{1,2}[:\.]\d{2}([\.:]\d{2})?)?$/,$b=RegExp("(\\/|\\.|\\*|\\+|\\?|\\||\\(|\\)|\\[|\\]|\\{|\\}|\\\\|\\$|\\^|\\-)","g"),Ya=/[',$£€¥%\u2009\u202F\u20BD\u20a9\u20BArfkɃΞ]/gi,M=function(a){return!a||!0===a||"-"===a?!0:!1},Ob=function(a){var b=parseInt(a,10);return!isNaN(b)&&
+isFinite(a)?b:null},Pb=function(a,b){Za[b]||(Za[b]=RegExp(Qa(b),"g"));return"string"===typeof a&&"."!==b?a.replace(/\./g,"").replace(Za[b],"."):a},$a=function(a,b,c){var d="string"===typeof a;if(M(a))return!0;b&&d&&(a=Pb(a,b));c&&d&&(a=a.replace(Ya,""));return!isNaN(parseFloat(a))&&isFinite(a)},Qb=function(a,b,c){return M(a)?!0:!(M(a)||"string"===typeof a)?null:$a(a.replace(Aa,""),b,c)?!0:null},D=function(a,b,c){var d=[],e=0,f=a.length;if(c!==k)for(;e<f;e++)a[e]&&a[e][b]&&d.push(a[e][b][c]);else for(;e<
+f;e++)a[e]&&d.push(a[e][b]);return d},ja=function(a,b,c,d){var e=[],f=0,g=b.length;if(d!==k)for(;f<g;f++)a[b[f]][c]&&e.push(a[b[f]][c][d]);else for(;f<g;f++)e.push(a[b[f]][c]);return e},Y=function(a,b){var c=[],d;b===k?(b=0,d=a):(d=b,b=a);for(var e=b;e<d;e++)c.push(e);return c},Rb=function(a){for(var b=[],c=0,d=a.length;c<d;c++)a[c]&&b.push(a[c]);return b},qa=function(a){var b;a:{if(!(2>a.length)){b=a.slice().sort();for(var c=b[0],d=1,e=b.length;d<e;d++){if(b[d]===c){b=!1;break a}c=b[d]}}b=!0}if(b)return a.slice();
+b=[];var e=a.length,f,g=0,d=0;a:for(;d<e;d++){c=a[d];for(f=0;f<g;f++)if(b[f]===c)continue a;b.push(c);g++}return b};n.util={throttle:function(a,b){var c=b!==k?b:200,d,e;return function(){var b=this,g=+new Date,j=arguments;d&&g<d+c?(clearTimeout(e),e=setTimeout(function(){d=k;a.apply(b,j)},c)):(d=g,a.apply(b,j))}},escapeRegex:function(a){return a.replace($b,"\\$1")}};var A=function(a,b,c){a[b]!==k&&(a[c]=a[b])},ca=/\[.*?\]$/,W=/\(\)$/,Qa=n.util.escapeRegex,va=h("<div>")[0],Wb=va.textContent!==k,Yb=
+/<.*?>/g,Oa=n.util.throttle,Sb=[],w=Array.prototype,ac=function(a){var b,c,d=n.settings,e=h.map(d,function(a){return a.nTable});if(a){if(a.nTable&&a.oApi)return[a];if(a.nodeName&&"table"===a.nodeName.toLowerCase())return b=h.inArray(a,e),-1!==b?[d[b]]:null;if(a&&"function"===typeof a.settings)return a.settings().toArray();"string"===typeof a?c=h(a):a instanceof h&&(c=a)}else return[];if(c)return c.map(function(){b=h.inArray(this,e);return-1!==b?d[b]:null}).toArray()};s=function(a,b){if(!(this instanceof
+s))return new s(a,b);var c=[],d=function(a){(a=ac(a))&&(c=c.concat(a))};if(h.isArray(a))for(var e=0,f=a.length;e<f;e++)d(a[e]);else d(a);this.context=qa(c);b&&h.merge(this,b);this.selector={rows:null,cols:null,opts:null};s.extend(this,this,Sb)};n.Api=s;h.extend(s.prototype,{any:function(){return 0!==this.count()},concat:w.concat,context:[],count:function(){return this.flatten().length},each:function(a){for(var b=0,c=this.length;b<c;b++)a.call(this,this[b],b,this);return this},eq:function(a){var b=
+this.context;return b.length>a?new s(b[a],this[a]):null},filter:function(a){var b=[];if(w.filter)b=w.filter.call(this,a,this);else for(var c=0,d=this.length;c<d;c++)a.call(this,this[c],c,this)&&b.push(this[c]);return new s(this.context,b)},flatten:function(){var a=[];return new s(this.context,a.concat.apply(a,this.toArray()))},join:w.join,indexOf:w.indexOf||function(a,b){for(var c=b||0,d=this.length;c<d;c++)if(this[c]===a)return c;return-1},iterator:function(a,b,c,d){var e=[],f,g,j,h,m,l=this.context,
+n,o,u=this.selector;"string"===typeof a&&(d=c,c=b,b=a,a=!1);g=0;for(j=l.length;g<j;g++){var r=new s(l[g]);if("table"===b)f=c.call(r,l[g],g),f!==k&&e.push(f);else if("columns"===b||"rows"===b)f=c.call(r,l[g],this[g],g),f!==k&&e.push(f);else if("column"===b||"column-rows"===b||"row"===b||"cell"===b){o=this[g];"column-rows"===b&&(n=Ba(l[g],u.opts));h=0;for(m=o.length;h<m;h++)f=o[h],f="cell"===b?c.call(r,l[g],f.row,f.column,g,h):c.call(r,l[g],f,g,h,n),f!==k&&e.push(f)}}return e.length||d?(a=new s(l,a?
+e.concat.apply([],e):e),b=a.selector,b.rows=u.rows,b.cols=u.cols,b.opts=u.opts,a):this},lastIndexOf:w.lastIndexOf||function(a,b){return this.indexOf.apply(this.toArray.reverse(),arguments)},length:0,map:function(a){var b=[];if(w.map)b=w.map.call(this,a,this);else for(var c=0,d=this.length;c<d;c++)b.push(a.call(this,this[c],c));return new s(this.context,b)},pluck:function(a){return this.map(function(b){return b[a]})},pop:w.pop,push:w.push,reduce:w.reduce||function(a,b){return ib(this,a,b,0,this.length,
+1)},reduceRight:w.reduceRight||function(a,b){return ib(this,a,b,this.length-1,-1,-1)},reverse:w.reverse,selector:null,shift:w.shift,slice:function(){return new s(this.context,this)},sort:w.sort,splice:w.splice,toArray:function(){return w.slice.call(this)},to$:function(){return h(this)},toJQuery:function(){return h(this)},unique:function(){return new s(this.context,qa(this))},unshift:w.unshift});s.extend=function(a,b,c){if(c.length&&b&&(b instanceof s||b.__dt_wrapper)){var d,e,f,g=function(a,b,c){return function(){var d=
+b.apply(a,arguments);s.extend(d,d,c.methodExt);return d}};d=0;for(e=c.length;d<e;d++)f=c[d],b[f.name]="function"===typeof f.val?g(a,f.val,f):h.isPlainObject(f.val)?{}:f.val,b[f.name].__dt_wrapper=!0,s.extend(a,b[f.name],f.propExt)}};s.register=o=function(a,b){if(h.isArray(a))for(var c=0,d=a.length;c<d;c++)s.register(a[c],b);else for(var e=a.split("."),f=Sb,g,j,c=0,d=e.length;c<d;c++){g=(j=-1!==e[c].indexOf("()"))?e[c].replace("()",""):e[c];var i;a:{i=0;for(var m=f.length;i<m;i++)if(f[i].name===g){i=
+f[i];break a}i=null}i||(i={name:g,val:{},methodExt:[],propExt:[]},f.push(i));c===d-1?i.val=b:f=j?i.methodExt:i.propExt}};s.registerPlural=u=function(a,b,c){s.register(a,c);s.register(b,function(){var a=c.apply(this,arguments);return a===this?this:a instanceof s?a.length?h.isArray(a[0])?new s(a.context,a[0]):a[0]:k:a})};o("tables()",function(a){var b;if(a){b=s;var c=this.context;if("number"===typeof a)a=[c[a]];else var d=h.map(c,function(a){return a.nTable}),a=h(d).filter(a).map(function(){var a=h.inArray(this,
+d);return c[a]}).toArray();b=new b(a)}else b=this;return b});o("table()",function(a){var a=this.tables(a),b=a.context;return b.length?new s(b[0]):a});u("tables().nodes()","table().node()",function(){return this.iterator("table",function(a){return a.nTable},1)});u("tables().body()","table().body()",function(){return this.iterator("table",function(a){return a.nTBody},1)});u("tables().header()","table().header()",function(){return this.iterator("table",function(a){return a.nTHead},1)});u("tables().footer()",
+"table().footer()",function(){return this.iterator("table",function(a){return a.nTFoot},1)});u("tables().containers()","table().container()",function(){return this.iterator("table",function(a){return a.nTableWrapper},1)});o("draw()",function(a){return this.iterator("table",function(b){"page"===a?P(b):("string"===typeof a&&(a="full-hold"===a?!1:!0),T(b,!1===a))})});o("page()",function(a){return a===k?this.page.info().page:this.iterator("table",function(b){Ta(b,a)})});o("page.info()",function(){if(0===
+this.context.length)return k;var a=this.context[0],b=a._iDisplayStart,c=a.oFeatures.bPaginate?a._iDisplayLength:-1,d=a.fnRecordsDisplay(),e=-1===c;return{page:e?0:Math.floor(b/c),pages:e?1:Math.ceil(d/c),start:b,end:a.fnDisplayEnd(),length:c,recordsTotal:a.fnRecordsTotal(),recordsDisplay:d,serverSide:"ssp"===y(a)}});o("page.len()",function(a){return a===k?0!==this.context.length?this.context[0]._iDisplayLength:k:this.iterator("table",function(b){Ra(b,a)})});var Tb=function(a,b,c){if(c){var d=new s(a);
+d.one("draw",function(){c(d.ajax.json())})}if("ssp"==y(a))T(a,b);else{C(a,!0);var e=a.jqXHR;e&&4!==e.readyState&&e.abort();sa(a,[],function(c){oa(a);for(var c=ta(a,c),d=0,e=c.length;d<e;d++)O(a,c[d]);T(a,b);C(a,!1)})}};o("ajax.json()",function(){var a=this.context;if(0<a.length)return a[0].json});o("ajax.params()",function(){var a=this.context;if(0<a.length)return a[0].oAjaxData});o("ajax.reload()",function(a,b){return this.iterator("table",function(c){Tb(c,!1===b,a)})});o("ajax.url()",function(a){var b=
+this.context;if(a===k){if(0===b.length)return k;b=b[0];return b.ajax?h.isPlainObject(b.ajax)?b.ajax.url:b.ajax:b.sAjaxSource}return this.iterator("table",function(b){h.isPlainObject(b.ajax)?b.ajax.url=a:b.ajax=a})});o("ajax.url().load()",function(a,b){return this.iterator("table",function(c){Tb(c,!1===b,a)})});var ab=function(a,b,c,d,e){var f=[],g,j,i,m,l,n;i=typeof b;if(!b||"string"===i||"function"===i||b.length===k)b=[b];i=0;for(m=b.length;i<m;i++){j=b[i]&&b[i].split&&!b[i].match(/[\[\(:]/)?b[i].split(","):
+[b[i]];l=0;for(n=j.length;l<n;l++)(g=c("string"===typeof j[l]?h.trim(j[l]):j[l]))&&g.length&&(f=f.concat(g))}a=x.selector[a];if(a.length){i=0;for(m=a.length;i<m;i++)f=a[i](d,e,f)}return qa(f)},bb=function(a){a||(a={});a.filter&&a.search===k&&(a.search=a.filter);return h.extend({search:"none",order:"current",page:"all"},a)},cb=function(a){for(var b=0,c=a.length;b<c;b++)if(0<a[b].length)return a[0]=a[b],a[0].length=1,a.length=1,a.context=[a.context[b]],a;a.length=0;return a},Ba=function(a,b){var c,
+d,e,f=[],g=a.aiDisplay;e=a.aiDisplayMaster;var j=b.search;c=b.order;d=b.page;if("ssp"==y(a))return"removed"===j?[]:Y(0,e.length);if("current"==d){c=a._iDisplayStart;for(d=a.fnDisplayEnd();c<d;c++)f.push(g[c])}else if("current"==c||"applied"==c)if("none"==j)f=e.slice();else if("applied"==j)f=g.slice();else{if("removed"==j){var i={};c=0;for(d=g.length;c<d;c++)i[g[c]]=null;f=h.map(e,function(a){return!i.hasOwnProperty(a)?a:null})}}else if("index"==c||"original"==c){c=0;for(d=a.aoData.length;c<d;c++)"none"==
+j?f.push(c):(e=h.inArray(c,g),(-1===e&&"removed"==j||0<=e&&"applied"==j)&&f.push(c))}return f};o("rows()",function(a,b){a===k?a="":h.isPlainObject(a)&&(b=a,a="");var b=bb(b),c=this.iterator("table",function(c){var e=b,f;return ab("row",a,function(a){var b=Ob(a),i=c.aoData;if(b!==null&&!e)return[b];f||(f=Ba(c,e));if(b!==null&&h.inArray(b,f)!==-1)return[b];if(a===null||a===k||a==="")return f;if(typeof a==="function")return h.map(f,function(b){var c=i[b];return a(b,c._aData,c.nTr)?b:null});if(a.nodeName){var b=
+a._DT_RowIndex,m=a._DT_CellIndex;if(b!==k)return i[b]&&i[b].nTr===a?[b]:[];if(m)return i[m.row]&&i[m.row].nTr===a?[m.row]:[];b=h(a).closest("*[data-dt-row]");return b.length?[b.data("dt-row")]:[]}if(typeof a==="string"&&a.charAt(0)==="#"){b=c.aIds[a.replace(/^#/,"")];if(b!==k)return[b.idx]}b=Rb(ja(c.aoData,f,"nTr"));return h(b).filter(a).map(function(){return this._DT_RowIndex}).toArray()},c,e)},1);c.selector.rows=a;c.selector.opts=b;return c});o("rows().nodes()",function(){return this.iterator("row",
+function(a,b){return a.aoData[b].nTr||k},1)});o("rows().data()",function(){return this.iterator(!0,"rows",function(a,b){return ja(a.aoData,b,"_aData")},1)});u("rows().cache()","row().cache()",function(a){return this.iterator("row",function(b,c){var d=b.aoData[c];return"search"===a?d._aFilterData:d._aSortData},1)});u("rows().invalidate()","row().invalidate()",function(a){return this.iterator("row",function(b,c){da(b,c,a)})});u("rows().indexes()","row().index()",function(){return this.iterator("row",
+function(a,b){return b},1)});u("rows().ids()","row().id()",function(a){for(var b=[],c=this.context,d=0,e=c.length;d<e;d++)for(var f=0,g=this[d].length;f<g;f++){var h=c[d].rowIdFn(c[d].aoData[this[d][f]]._aData);b.push((!0===a?"#":"")+h)}return new s(c,b)});u("rows().remove()","row().remove()",function(){var a=this;this.iterator("row",function(b,c,d){var e=b.aoData,f=e[c],g,h,i,m,l;e.splice(c,1);g=0;for(h=e.length;g<h;g++)if(i=e[g],l=i.anCells,null!==i.nTr&&(i.nTr._DT_RowIndex=g),null!==l){i=0;for(m=
+l.length;i<m;i++)l[i]._DT_CellIndex.row=g}pa(b.aiDisplayMaster,c);pa(b.aiDisplay,c);pa(a[d],c,!1);0<b._iRecordsDisplay&&b._iRecordsDisplay--;Sa(b);c=b.rowIdFn(f._aData);c!==k&&delete b.aIds[c]});this.iterator("table",function(a){for(var c=0,d=a.aoData.length;c<d;c++)a.aoData[c].idx=c});return this});o("rows.add()",function(a){var b=this.iterator("table",function(b){var c,f,g,h=[];f=0;for(g=a.length;f<g;f++)c=a[f],c.nodeName&&"TR"===c.nodeName.toUpperCase()?h.push(na(b,c)[0]):h.push(O(b,c));return h},
+1),c=this.rows(-1);c.pop();h.merge(c,b);return c});o("row()",function(a,b){return cb(this.rows(a,b))});o("row().data()",function(a){var b=this.context;if(a===k)return b.length&&this.length?b[0].aoData[this[0]]._aData:k;var c=b[0].aoData[this[0]];c._aData=a;h.isArray(a)&&c.nTr.id&&N(b[0].rowId)(a,c.nTr.id);da(b[0],this[0],"data");return this});o("row().node()",function(){var a=this.context;return a.length&&this.length?a[0].aoData[this[0]].nTr||null:null});o("row.add()",function(a){a instanceof h&&
+a.length&&(a=a[0]);var b=this.iterator("table",function(b){return a.nodeName&&"TR"===a.nodeName.toUpperCase()?na(b,a)[0]:O(b,a)});return this.row(b[0])});var db=function(a,b){var c=a.context;if(c.length&&(c=c[0].aoData[b!==k?b:a[0]])&&c._details)c._details.remove(),c._detailsShow=k,c._details=k},Ub=function(a,b){var c=a.context;if(c.length&&a.length){var d=c[0].aoData[a[0]];if(d._details){(d._detailsShow=b)?d._details.insertAfter(d.nTr):d._details.detach();var e=c[0],f=new s(e),g=e.aoData;f.off("draw.dt.DT_details column-visibility.dt.DT_details destroy.dt.DT_details");
+0<D(g,"_details").length&&(f.on("draw.dt.DT_details",function(a,b){e===b&&f.rows({page:"current"}).eq(0).each(function(a){a=g[a];a._detailsShow&&a._details.insertAfter(a.nTr)})}),f.on("column-visibility.dt.DT_details",function(a,b){if(e===b)for(var c,d=V(b),f=0,h=g.length;f<h;f++)c=g[f],c._details&&c._details.children("td[colspan]").attr("colspan",d)}),f.on("destroy.dt.DT_details",function(a,b){if(e===b)for(var c=0,d=g.length;c<d;c++)g[c]._details&&db(f,c)}))}}};o("row().child()",function(a,b){var c=
+this.context;if(a===k)return c.length&&this.length?c[0].aoData[this[0]]._details:k;if(!0===a)this.child.show();else if(!1===a)db(this);else if(c.length&&this.length){var d=c[0],c=c[0].aoData[this[0]],e=[],f=function(a,b){if(h.isArray(a)||a instanceof h)for(var c=0,k=a.length;c<k;c++)f(a[c],b);else a.nodeName&&"tr"===a.nodeName.toLowerCase()?e.push(a):(c=h("<tr><td/></tr>").addClass(b),h("td",c).addClass(b).html(a)[0].colSpan=V(d),e.push(c[0]))};f(a,b);c._details&&c._details.detach();c._details=h(e);
+c._detailsShow&&c._details.insertAfter(c.nTr)}return this});o(["row().child.show()","row().child().show()"],function(){Ub(this,!0);return this});o(["row().child.hide()","row().child().hide()"],function(){Ub(this,!1);return this});o(["row().child.remove()","row().child().remove()"],function(){db(this);return this});o("row().child.isShown()",function(){var a=this.context;return a.length&&this.length?a[0].aoData[this[0]]._detailsShow||!1:!1});var bc=/^([^:]+):(name|visIdx|visible)$/,Vb=function(a,b,
+c,d,e){for(var c=[],d=0,f=e.length;d<f;d++)c.push(B(a,e[d],b));return c};o("columns()",function(a,b){a===k?a="":h.isPlainObject(a)&&(b=a,a="");var b=bb(b),c=this.iterator("table",function(c){var e=a,f=b,g=c.aoColumns,j=D(g,"sName"),i=D(g,"nTh");return ab("column",e,function(a){var b=Ob(a);if(a==="")return Y(g.length);if(b!==null)return[b>=0?b:g.length+b];if(typeof a==="function"){var e=Ba(c,f);return h.map(g,function(b,f){return a(f,Vb(c,f,0,0,e),i[f])?f:null})}var k=typeof a==="string"?a.match(bc):
+"";if(k)switch(k[2]){case "visIdx":case "visible":b=parseInt(k[1],10);if(b<0){var n=h.map(g,function(a,b){return a.bVisible?b:null});return[n[n.length+b]]}return[aa(c,b)];case "name":return h.map(j,function(a,b){return a===k[1]?b:null});default:return[]}if(a.nodeName&&a._DT_CellIndex)return[a._DT_CellIndex.column];b=h(i).filter(a).map(function(){return h.inArray(this,i)}).toArray();if(b.length||!a.nodeName)return b;b=h(a).closest("*[data-dt-column]");return b.length?[b.data("dt-column")]:[]},c,f)},
+1);c.selector.cols=a;c.selector.opts=b;return c});u("columns().header()","column().header()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].nTh},1)});u("columns().footer()","column().footer()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].nTf},1)});u("columns().data()","column().data()",function(){return this.iterator("column-rows",Vb,1)});u("columns().dataSrc()","column().dataSrc()",function(){return this.iterator("column",function(a,b){return a.aoColumns[b].mData},
+1)});u("columns().cache()","column().cache()",function(a){return this.iterator("column-rows",function(b,c,d,e,f){return ja(b.aoData,f,"search"===a?"_aFilterData":"_aSortData",c)},1)});u("columns().nodes()","column().nodes()",function(){return this.iterator("column-rows",function(a,b,c,d,e){return ja(a.aoData,e,"anCells",b)},1)});u("columns().visible()","column().visible()",function(a,b){var c=this.iterator("column",function(b,c){if(a===k)return b.aoColumns[c].bVisible;var f=b.aoColumns,g=f[c],j=b.aoData,
+i,m,l;if(a!==k&&g.bVisible!==a){if(a){var n=h.inArray(!0,D(f,"bVisible"),c+1);i=0;for(m=j.length;i<m;i++)l=j[i].nTr,f=j[i].anCells,l&&l.insertBefore(f[c],f[n]||null)}else h(D(b.aoData,"anCells",c)).detach();g.bVisible=a;fa(b,b.aoHeader);fa(b,b.aoFooter);b.aiDisplay.length||h(b.nTBody).find("td[colspan]").attr("colspan",V(b));xa(b)}});a!==k&&(this.iterator("column",function(c,e){r(c,null,"column-visibility",[c,e,a,b])}),(b===k||b)&&this.columns.adjust());return c});u("columns().indexes()","column().index()",
+function(a){return this.iterator("column",function(b,c){return"visible"===a?ba(b,c):c},1)});o("columns.adjust()",function(){return this.iterator("table",function(a){$(a)},1)});o("column.index()",function(a,b){if(0!==this.context.length){var c=this.context[0];if("fromVisible"===a||"toData"===a)return aa(c,b);if("fromData"===a||"toVisible"===a)return ba(c,b)}});o("column()",function(a,b){return cb(this.columns(a,b))});o("cells()",function(a,b,c){h.isPlainObject(a)&&(a.row===k?(c=a,a=null):(c=b,b=null));
+h.isPlainObject(b)&&(c=b,b=null);if(null===b||b===k)return this.iterator("table",function(b){var d=a,e=bb(c),f=b.aoData,g=Ba(b,e),j=Rb(ja(f,g,"anCells")),i=h([].concat.apply([],j)),l,m=b.aoColumns.length,n,o,u,s,r,v;return ab("cell",d,function(a){var c=typeof a==="function";if(a===null||a===k||c){n=[];o=0;for(u=g.length;o<u;o++){l=g[o];for(s=0;s<m;s++){r={row:l,column:s};if(c){v=f[l];a(r,B(b,l,s),v.anCells?v.anCells[s]:null)&&n.push(r)}else n.push(r)}}return n}if(h.isPlainObject(a))return a.column!==
+k&&a.row!==k&&h.inArray(a.row,g)!==-1?[a]:[];c=i.filter(a).map(function(a,b){return{row:b._DT_CellIndex.row,column:b._DT_CellIndex.column}}).toArray();if(c.length||!a.nodeName)return c;v=h(a).closest("*[data-dt-row]");return v.length?[{row:v.data("dt-row"),column:v.data("dt-column")}]:[]},b,e)});var d=this.columns(b),e=this.rows(a),f,g,j,i,m;this.iterator("table",function(a,b){f=[];g=0;for(j=e[b].length;g<j;g++){i=0;for(m=d[b].length;i<m;i++)f.push({row:e[b][g],column:d[b][i]})}},1);var l=this.cells(f,
+c);h.extend(l.selector,{cols:b,rows:a,opts:c});return l});u("cells().nodes()","cell().node()",function(){return this.iterator("cell",function(a,b,c){return(a=a.aoData[b])&&a.anCells?a.anCells[c]:k},1)});o("cells().data()",function(){return this.iterator("cell",function(a,b,c){return B(a,b,c)},1)});u("cells().cache()","cell().cache()",function(a){a="search"===a?"_aFilterData":"_aSortData";return this.iterator("cell",function(b,c,d){return b.aoData[c][a][d]},1)});u("cells().render()","cell().render()",
+function(a){return this.iterator("cell",function(b,c,d){return B(b,c,d,a)},1)});u("cells().indexes()","cell().index()",function(){return this.iterator("cell",function(a,b,c){return{row:b,column:c,columnVisible:ba(a,c)}},1)});u("cells().invalidate()","cell().invalidate()",function(a){return this.iterator("cell",function(b,c,d){da(b,c,a,d)})});o("cell()",function(a,b,c){return cb(this.cells(a,b,c))});o("cell().data()",function(a){var b=this.context,c=this[0];if(a===k)return b.length&&c.length?B(b[0],
+c[0].row,c[0].column):k;kb(b[0],c[0].row,c[0].column,a);da(b[0],c[0].row,"data",c[0].column);return this});o("order()",function(a,b){var c=this.context;if(a===k)return 0!==c.length?c[0].aaSorting:k;"number"===typeof a?a=[[a,b]]:a.length&&!h.isArray(a[0])&&(a=Array.prototype.slice.call(arguments));return this.iterator("table",function(b){b.aaSorting=a.slice()})});o("order.listener()",function(a,b,c){return this.iterator("table",function(d){Ma(d,a,b,c)})});o("order.fixed()",function(a){if(!a){var b=
+this.context,b=b.length?b[0].aaSortingFixed:k;return h.isArray(b)?{pre:b}:b}return this.iterator("table",function(b){b.aaSortingFixed=h.extend(!0,{},a)})});o(["columns().order()","column().order()"],function(a){var b=this;return this.iterator("table",function(c,d){var e=[];h.each(b[d],function(b,c){e.push([c,a])});c.aaSorting=e})});o("search()",function(a,b,c,d){var e=this.context;return a===k?0!==e.length?e[0].oPreviousSearch.sSearch:k:this.iterator("table",function(e){e.oFeatures.bFilter&&ga(e,
+h.extend({},e.oPreviousSearch,{sSearch:a+"",bRegex:null===b?!1:b,bSmart:null===c?!0:c,bCaseInsensitive:null===d?!0:d}),1)})});u("columns().search()","column().search()",function(a,b,c,d){return this.iterator("column",function(e,f){var g=e.aoPreSearchCols;if(a===k)return g[f].sSearch;e.oFeatures.bFilter&&(h.extend(g[f],{sSearch:a+"",bRegex:null===b?!1:b,bSmart:null===c?!0:c,bCaseInsensitive:null===d?!0:d}),ga(e,e.oPreviousSearch,1))})});o("state()",function(){return this.context.length?this.context[0].oSavedState:
+null});o("state.clear()",function(){return this.iterator("table",function(a){a.fnStateSaveCallback.call(a.oInstance,a,{})})});o("state.loaded()",function(){return this.context.length?this.context[0].oLoadedState:null});o("state.save()",function(){return this.iterator("table",function(a){xa(a)})});n.versionCheck=n.fnVersionCheck=function(a){for(var b=n.version.split("."),a=a.split("."),c,d,e=0,f=a.length;e<f;e++)if(c=parseInt(b[e],10)||0,d=parseInt(a[e],10)||0,c!==d)return c>d;return!0};n.isDataTable=
+n.fnIsDataTable=function(a){var b=h(a).get(0),c=!1;if(a instanceof n.Api)return!0;h.each(n.settings,function(a,e){var f=e.nScrollHead?h("table",e.nScrollHead)[0]:null,g=e.nScrollFoot?h("table",e.nScrollFoot)[0]:null;if(e.nTable===b||f===b||g===b)c=!0});return c};n.tables=n.fnTables=function(a){var b=!1;h.isPlainObject(a)&&(b=a.api,a=a.visible);var c=h.map(n.settings,function(b){if(!a||a&&h(b.nTable).is(":visible"))return b.nTable});return b?new s(c):c};n.camelToHungarian=J;o("$()",function(a,b){var c=
+this.rows(b).nodes(),c=h(c);return h([].concat(c.filter(a).toArray(),c.find(a).toArray()))});h.each(["on","one","off"],function(a,b){o(b+"()",function(){var a=Array.prototype.slice.call(arguments);a[0]=h.map(a[0].split(/\s/),function(a){return!a.match(/\.dt\b/)?a+".dt":a}).join(" ");var d=h(this.tables().nodes());d[b].apply(d,a);return this})});o("clear()",function(){return this.iterator("table",function(a){oa(a)})});o("settings()",function(){return new s(this.context,this.context)});o("init()",function(){var a=
+this.context;return a.length?a[0].oInit:null});o("data()",function(){return this.iterator("table",function(a){return D(a.aoData,"_aData")}).flatten()});o("destroy()",function(a){a=a||!1;return this.iterator("table",function(b){var c=b.nTableWrapper.parentNode,d=b.oClasses,e=b.nTable,f=b.nTBody,g=b.nTHead,j=b.nTFoot,i=h(e),f=h(f),k=h(b.nTableWrapper),l=h.map(b.aoData,function(a){return a.nTr}),o;b.bDestroying=!0;r(b,"aoDestroyCallback","destroy",[b]);a||(new s(b)).columns().visible(!0);k.off(".DT").find(":not(tbody *)").off(".DT");
+h(E).off(".DT-"+b.sInstance);e!=g.parentNode&&(i.children("thead").detach(),i.append(g));j&&e!=j.parentNode&&(i.children("tfoot").detach(),i.append(j));b.aaSorting=[];b.aaSortingFixed=[];wa(b);h(l).removeClass(b.asStripeClasses.join(" "));h("th, td",g).removeClass(d.sSortable+" "+d.sSortableAsc+" "+d.sSortableDesc+" "+d.sSortableNone);f.children().detach();f.append(l);g=a?"remove":"detach";i[g]();k[g]();!a&&c&&(c.insertBefore(e,b.nTableReinsertBefore),i.css("width",b.sDestroyWidth).removeClass(d.sTable),
+(o=b.asDestroyStripes.length)&&f.children().each(function(a){h(this).addClass(b.asDestroyStripes[a%o])}));c=h.inArray(b,n.settings);-1!==c&&n.settings.splice(c,1)})});h.each(["column","row","cell"],function(a,b){o(b+"s().every()",function(a){var d=this.selector.opts,e=this;return this.iterator(b,function(f,g,h,i,m){a.call(e[b](g,"cell"===b?h:d,"cell"===b?d:k),g,h,i,m)})})});o("i18n()",function(a,b,c){var d=this.context[0],a=S(a)(d.oLanguage);a===k&&(a=b);c!==k&&h.isPlainObject(a)&&(a=a[c]!==k?a[c]:
+a._);return a.replace("%d",c)});n.version="1.10.19";n.settings=[];n.models={};n.models.oSearch={bCaseInsensitive:!0,sSearch:"",bRegex:!1,bSmart:!0};n.models.oRow={nTr:null,anCells:null,_aData:[],_aSortData:null,_aFilterData:null,_sFilterRow:null,_sRowStripe:"",src:null,idx:-1};n.models.oColumn={idx:null,aDataSort:null,asSorting:null,bSearchable:null,bSortable:null,bVisible:null,_sManualType:null,_bAttrSrc:!1,fnCreatedCell:null,fnGetData:null,fnSetData:null,mData:null,mRender:null,nTh:null,nTf:null,
+sClass:null,sContentPadding:null,sDefaultContent:null,sName:null,sSortDataType:"std",sSortingClass:null,sSortingClassJUI:null,sTitle:null,sType:null,sWidth:null,sWidthOrig:null};n.defaults={aaData:null,aaSorting:[[0,"asc"]],aaSortingFixed:[],ajax:null,aLengthMenu:[10,25,50,100],aoColumns:null,aoColumnDefs:null,aoSearchCols:[],asStripeClasses:null,bAutoWidth:!0,bDeferRender:!1,bDestroy:!1,bFilter:!0,bInfo:!0,bLengthChange:!0,bPaginate:!0,bProcessing:!1,bRetrieve:!1,bScrollCollapse:!1,bServerSide:!1,
+bSort:!0,bSortMulti:!0,bSortCellsTop:!1,bSortClasses:!0,bStateSave:!1,fnCreatedRow:null,fnDrawCallback:null,fnFooterCallback:null,fnFormatNumber:function(a){return a.toString().replace(/\B(?=(\d{3})+(?!\d))/g,this.oLanguage.sThousands)},fnHeaderCallback:null,fnInfoCallback:null,fnInitComplete:null,fnPreDrawCallback:null,fnRowCallback:null,fnServerData:null,fnServerParams:null,fnStateLoadCallback:function(a){try{return JSON.parse((-1===a.iStateDuration?sessionStorage:localStorage).getItem("DataTables_"+
+a.sInstance+"_"+location.pathname))}catch(b){}},fnStateLoadParams:null,fnStateLoaded:null,fnStateSaveCallback:function(a,b){try{(-1===a.iStateDuration?sessionStorage:localStorage).setItem("DataTables_"+a.sInstance+"_"+location.pathname,JSON.stringify(b))}catch(c){}},fnStateSaveParams:null,iStateDuration:7200,iDeferLoading:null,iDisplayLength:10,iDisplayStart:0,iTabIndex:0,oClasses:{},oLanguage:{oAria:{sSortAscending:": activate to sort column ascending",sSortDescending:": activate to sort column descending"},
+oPaginate:{sFirst:"First",sLast:"Last",sNext:"Next",sPrevious:"Previous"},sEmptyTable:"No data available in table",sInfo:"Showing _START_ to _END_ of _TOTAL_ entries",sInfoEmpty:"Showing 0 to 0 of 0 entries",sInfoFiltered:"(filtered from _MAX_ total entries)",sInfoPostFix:"",sDecimal:"",sThousands:",",sLengthMenu:"Show _MENU_ entries",sLoadingRecords:"Loading...",sProcessing:"Processing...",sSearch:"Search:",sSearchPlaceholder:"",sUrl:"",sZeroRecords:"No matching records found"},oSearch:h.extend({},
+n.models.oSearch),sAjaxDataProp:"data",sAjaxSource:null,sDom:"lfrtip",searchDelay:null,sPaginationType:"simple_numbers",sScrollX:"",sScrollXInner:"",sScrollY:"",sServerMethod:"GET",renderer:null,rowId:"DT_RowId"};Z(n.defaults);n.defaults.column={aDataSort:null,iDataSort:-1,asSorting:["asc","desc"],bSearchable:!0,bSortable:!0,bVisible:!0,fnCreatedCell:null,mData:null,mRender:null,sCellType:"td",sClass:"",sContentPadding:"",sDefaultContent:null,sName:"",sSortDataType:"std",sTitle:null,sType:null,sWidth:null};
+Z(n.defaults.column);n.models.oSettings={oFeatures:{bAutoWidth:null,bDeferRender:null,bFilter:null,bInfo:null,bLengthChange:null,bPaginate:null,bProcessing:null,bServerSide:null,bSort:null,bSortMulti:null,bSortClasses:null,bStateSave:null},oScroll:{bCollapse:null,iBarWidth:0,sX:null,sXInner:null,sY:null},oLanguage:{fnInfoCallback:null},oBrowser:{bScrollOversize:!1,bScrollbarLeft:!1,bBounding:!1,barWidth:0},ajax:null,aanFeatures:[],aoData:[],aiDisplay:[],aiDisplayMaster:[],aIds:{},aoColumns:[],aoHeader:[],
+aoFooter:[],oPreviousSearch:{},aoPreSearchCols:[],aaSorting:null,aaSortingFixed:[],asStripeClasses:null,asDestroyStripes:[],sDestroyWidth:0,aoRowCallback:[],aoHeaderCallback:[],aoFooterCallback:[],aoDrawCallback:[],aoRowCreatedCallback:[],aoPreDrawCallback:[],aoInitComplete:[],aoStateSaveParams:[],aoStateLoadParams:[],aoStateLoaded:[],sTableId:"",nTable:null,nTHead:null,nTFoot:null,nTBody:null,nTableWrapper:null,bDeferLoading:!1,bInitialised:!1,aoOpenRows:[],sDom:null,searchDelay:null,sPaginationType:"two_button",
+iStateDuration:0,aoStateSave:[],aoStateLoad:[],oSavedState:null,oLoadedState:null,sAjaxSource:null,sAjaxDataProp:null,bAjaxDataGet:!0,jqXHR:null,json:k,oAjaxData:k,fnServerData:null,aoServerParams:[],sServerMethod:null,fnFormatNumber:null,aLengthMenu:null,iDraw:0,bDrawing:!1,iDrawError:-1,_iDisplayLength:10,_iDisplayStart:0,_iRecordsTotal:0,_iRecordsDisplay:0,oClasses:{},bFiltered:!1,bSorted:!1,bSortCellsTop:null,oInit:null,aoDestroyCallback:[],fnRecordsTotal:function(){return"ssp"==y(this)?1*this._iRecordsTotal:
+this.aiDisplayMaster.length},fnRecordsDisplay:function(){return"ssp"==y(this)?1*this._iRecordsDisplay:this.aiDisplay.length},fnDisplayEnd:function(){var a=this._iDisplayLength,b=this._iDisplayStart,c=b+a,d=this.aiDisplay.length,e=this.oFeatures,f=e.bPaginate;return e.bServerSide?!1===f||-1===a?b+d:Math.min(b+a,this._iRecordsDisplay):!f||c>d||-1===a?d:c},oInstance:null,sInstance:null,iTabIndex:0,nScrollHead:null,nScrollFoot:null,aLastSort:[],oPlugins:{},rowIdFn:null,rowId:null};n.ext=x={buttons:{},
+classes:{},builder:"-source-",errMode:"alert",feature:[],search:[],selector:{cell:[],column:[],row:[]},internal:{},legacy:{ajax:null},pager:{},renderer:{pageButton:{},header:{}},order:{},type:{detect:[],search:{},order:{}},_unique:0,fnVersionCheck:n.fnVersionCheck,iApiIndex:0,oJUIClasses:{},sVersion:n.version};h.extend(x,{afnFiltering:x.search,aTypes:x.type.detect,ofnSearch:x.type.search,oSort:x.type.order,afnSortData:x.order,aoFeatures:x.feature,oApi:x.internal,oStdClasses:x.classes,oPagination:x.pager});
+h.extend(n.ext.classes,{sTable:"dataTable",sNoFooter:"no-footer",sPageButton:"paginate_button",sPageButtonActive:"current",sPageButtonDisabled:"disabled",sStripeOdd:"odd",sStripeEven:"even",sRowEmpty:"dataTables_empty",sWrapper:"dataTables_wrapper",sFilter:"dataTables_filter",sInfo:"dataTables_info",sPaging:"dataTables_paginate paging_",sLength:"dataTables_length",sProcessing:"dataTables_processing",sSortAsc:"sorting_asc",sSortDesc:"sorting_desc",sSortable:"sorting",sSortableAsc:"sorting_asc_disabled",
+sSortableDesc:"sorting_desc_disabled",sSortableNone:"sorting_disabled",sSortColumn:"sorting_",sFilterInput:"",sLengthSelect:"",sScrollWrapper:"dataTables_scroll",sScrollHead:"dataTables_scrollHead",sScrollHeadInner:"dataTables_scrollHeadInner",sScrollBody:"dataTables_scrollBody",sScrollFoot:"dataTables_scrollFoot",sScrollFootInner:"dataTables_scrollFootInner",sHeaderTH:"",sFooterTH:"",sSortJUIAsc:"",sSortJUIDesc:"",sSortJUI:"",sSortJUIAscAllowed:"",sSortJUIDescAllowed:"",sSortJUIWrapper:"",sSortIcon:"",
+sJUIHeader:"",sJUIFooter:""});var Lb=n.ext.pager;h.extend(Lb,{simple:function(){return["previous","next"]},full:function(){return["first","previous","next","last"]},numbers:function(a,b){return[ia(a,b)]},simple_numbers:function(a,b){return["previous",ia(a,b),"next"]},full_numbers:function(a,b){return["first","previous",ia(a,b),"next","last"]},first_last_numbers:function(a,b){return["first",ia(a,b),"last"]},_numbers:ia,numbers_length:7});h.extend(!0,n.ext.renderer,{pageButton:{_:function(a,b,c,d,e,
+f){var g=a.oClasses,j=a.oLanguage.oPaginate,i=a.oLanguage.oAria.paginate||{},m,l,n=0,o=function(b,d){var k,s,u,r,v=function(b){Ta(a,b.data.action,true)};k=0;for(s=d.length;k<s;k++){r=d[k];if(h.isArray(r)){u=h("<"+(r.DT_el||"div")+"/>").appendTo(b);o(u,r)}else{m=null;l="";switch(r){case "ellipsis":b.append('<span class="ellipsis">&#x2026;</span>');break;case "first":m=j.sFirst;l=r+(e>0?"":" "+g.sPageButtonDisabled);break;case "previous":m=j.sPrevious;l=r+(e>0?"":" "+g.sPageButtonDisabled);break;case "next":m=
+j.sNext;l=r+(e<f-1?"":" "+g.sPageButtonDisabled);break;case "last":m=j.sLast;l=r+(e<f-1?"":" "+g.sPageButtonDisabled);break;default:m=r+1;l=e===r?g.sPageButtonActive:""}if(m!==null){u=h("<a>",{"class":g.sPageButton+" "+l,"aria-controls":a.sTableId,"aria-label":i[r],"data-dt-idx":n,tabindex:a.iTabIndex,id:c===0&&typeof r==="string"?a.sTableId+"_"+r:null}).html(m).appendTo(b);Wa(u,{action:r},v);n++}}}},s;try{s=h(b).find(H.activeElement).data("dt-idx")}catch(u){}o(h(b).empty(),d);s!==k&&h(b).find("[data-dt-idx="+
+s+"]").focus()}}});h.extend(n.ext.type.detect,[function(a,b){var c=b.oLanguage.sDecimal;return $a(a,c)?"num"+c:null},function(a){if(a&&!(a instanceof Date)&&!Zb.test(a))return null;var b=Date.parse(a);return null!==b&&!isNaN(b)||M(a)?"date":null},function(a,b){var c=b.oLanguage.sDecimal;return $a(a,c,!0)?"num-fmt"+c:null},function(a,b){var c=b.oLanguage.sDecimal;return Qb(a,c)?"html-num"+c:null},function(a,b){var c=b.oLanguage.sDecimal;return Qb(a,c,!0)?"html-num-fmt"+c:null},function(a){return M(a)||
+"string"===typeof a&&-1!==a.indexOf("<")?"html":null}]);h.extend(n.ext.type.search,{html:function(a){return M(a)?a:"string"===typeof a?a.replace(Nb," ").replace(Aa,""):""},string:function(a){return M(a)?a:"string"===typeof a?a.replace(Nb," "):a}});var za=function(a,b,c,d){if(0!==a&&(!a||"-"===a))return-Infinity;b&&(a=Pb(a,b));a.replace&&(c&&(a=a.replace(c,"")),d&&(a=a.replace(d,"")));return 1*a};h.extend(x.type.order,{"date-pre":function(a){a=Date.parse(a);return isNaN(a)?-Infinity:a},"html-pre":function(a){return M(a)?
+"":a.replace?a.replace(/<.*?>/g,"").toLowerCase():a+""},"string-pre":function(a){return M(a)?"":"string"===typeof a?a.toLowerCase():!a.toString?"":a.toString()},"string-asc":function(a,b){return a<b?-1:a>b?1:0},"string-desc":function(a,b){return a<b?1:a>b?-1:0}});Da("");h.extend(!0,n.ext.renderer,{header:{_:function(a,b,c,d){h(a.nTable).on("order.dt.DT",function(e,f,g,h){if(a===f){e=c.idx;b.removeClass(c.sSortingClass+" "+d.sSortAsc+" "+d.sSortDesc).addClass(h[e]=="asc"?d.sSortAsc:h[e]=="desc"?d.sSortDesc:
+c.sSortingClass)}})},jqueryui:function(a,b,c,d){h("<div/>").addClass(d.sSortJUIWrapper).append(b.contents()).append(h("<span/>").addClass(d.sSortIcon+" "+c.sSortingClassJUI)).appendTo(b);h(a.nTable).on("order.dt.DT",function(e,f,g,h){if(a===f){e=c.idx;b.removeClass(d.sSortAsc+" "+d.sSortDesc).addClass(h[e]=="asc"?d.sSortAsc:h[e]=="desc"?d.sSortDesc:c.sSortingClass);b.find("span."+d.sSortIcon).removeClass(d.sSortJUIAsc+" "+d.sSortJUIDesc+" "+d.sSortJUI+" "+d.sSortJUIAscAllowed+" "+d.sSortJUIDescAllowed).addClass(h[e]==
+"asc"?d.sSortJUIAsc:h[e]=="desc"?d.sSortJUIDesc:c.sSortingClassJUI)}})}}});var eb=function(a){return"string"===typeof a?a.replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;"):a};n.render={number:function(a,b,c,d,e){return{display:function(f){if("number"!==typeof f&&"string"!==typeof f)return f;var g=0>f?"-":"",h=parseFloat(f);if(isNaN(h))return eb(f);h=h.toFixed(c);f=Math.abs(h);h=parseInt(f,10);f=c?b+(f-h).toFixed(c).substring(2):"";return g+(d||"")+h.toString().replace(/\B(?=(\d{3})+(?!\d))/g,
+a)+f+(e||"")}}},text:function(){return{display:eb,filter:eb}}};h.extend(n.ext.internal,{_fnExternApiFunc:Mb,_fnBuildAjax:sa,_fnAjaxUpdate:mb,_fnAjaxParameters:vb,_fnAjaxUpdateDraw:wb,_fnAjaxDataSrc:ta,_fnAddColumn:Ea,_fnColumnOptions:ka,_fnAdjustColumnSizing:$,_fnVisibleToColumnIndex:aa,_fnColumnIndexToVisible:ba,_fnVisbleColumns:V,_fnGetColumns:ma,_fnColumnTypes:Ga,_fnApplyColumnDefs:jb,_fnHungarianMap:Z,_fnCamelToHungarian:J,_fnLanguageCompat:Ca,_fnBrowserDetect:hb,_fnAddData:O,_fnAddTr:na,_fnNodeToDataIndex:function(a,
+b){return b._DT_RowIndex!==k?b._DT_RowIndex:null},_fnNodeToColumnIndex:function(a,b,c){return h.inArray(c,a.aoData[b].anCells)},_fnGetCellData:B,_fnSetCellData:kb,_fnSplitObjNotation:Ja,_fnGetObjectDataFn:S,_fnSetObjectDataFn:N,_fnGetDataMaster:Ka,_fnClearTable:oa,_fnDeleteIndex:pa,_fnInvalidate:da,_fnGetRowElements:Ia,_fnCreateTr:Ha,_fnBuildHead:lb,_fnDrawHead:fa,_fnDraw:P,_fnReDraw:T,_fnAddOptionsHtml:ob,_fnDetectHeader:ea,_fnGetUniqueThs:ra,_fnFeatureHtmlFilter:qb,_fnFilterComplete:ga,_fnFilterCustom:zb,
+_fnFilterColumn:yb,_fnFilter:xb,_fnFilterCreateSearch:Pa,_fnEscapeRegex:Qa,_fnFilterData:Ab,_fnFeatureHtmlInfo:tb,_fnUpdateInfo:Db,_fnInfoMacros:Eb,_fnInitialise:ha,_fnInitComplete:ua,_fnLengthChange:Ra,_fnFeatureHtmlLength:pb,_fnFeatureHtmlPaginate:ub,_fnPageChange:Ta,_fnFeatureHtmlProcessing:rb,_fnProcessingDisplay:C,_fnFeatureHtmlTable:sb,_fnScrollDraw:la,_fnApplyToChildren:I,_fnCalculateColumnWidths:Fa,_fnThrottle:Oa,_fnConvertToWidth:Fb,_fnGetWidestNode:Gb,_fnGetMaxLenString:Hb,_fnStringToCss:v,
+_fnSortFlatten:X,_fnSort:nb,_fnSortAria:Jb,_fnSortListener:Va,_fnSortAttachListener:Ma,_fnSortingClasses:wa,_fnSortData:Ib,_fnSaveState:xa,_fnLoadState:Kb,_fnSettingsFromNode:ya,_fnLog:K,_fnMap:F,_fnBindAction:Wa,_fnCallbackReg:z,_fnCallbackFire:r,_fnLengthOverflow:Sa,_fnRenderer:Na,_fnDataSource:y,_fnRowAttributes:La,_fnExtend:Xa,_fnCalculateEnd:function(){}});h.fn.dataTable=n;n.$=h;h.fn.dataTableSettings=n.settings;h.fn.dataTableExt=n.ext;h.fn.DataTable=function(a){return h(this).dataTable(a).api()};
+h.each(n,function(a,b){h.fn.DataTable[a]=b});return h.fn.dataTable});
@@ -0,0 +1,2 @@
+/*! jQuery v3.3.1 | (c) JS Foundation and other contributors | jquery.org/license */
+!function(e,t){"use strict";"object"==typeof module&&"object"==typeof module.exports?module.exports=e.document?t(e,!0):function(e){if(!e.document)throw new Error("jQuery requires a window with a document");return t(e)}:t(e)}("undefined"!=typeof window?window:this,function(e,t){"use strict";var n=[],r=e.document,i=Object.getPrototypeOf,o=n.slice,a=n.concat,s=n.push,u=n.indexOf,l={},c=l.toString,f=l.hasOwnProperty,p=f.toString,d=p.call(Object),h={},g=function e(t){return"function"==typeof t&&"number"!=typeof t.nodeType},y=function e(t){return null!=t&&t===t.window},v={type:!0,src:!0,noModule:!0};function m(e,t,n){var i,o=(t=t||r).createElement("script");if(o.text=e,n)for(i in v)n[i]&&(o[i]=n[i]);t.head.appendChild(o).parentNode.removeChild(o)}function x(e){return null==e?e+"":"object"==typeof e||"function"==typeof e?l[c.call(e)]||"object":typeof e}var b="3.3.1",w=function(e,t){return new w.fn.init(e,t)},T=/^[\s\uFEFF\xA0]+|[\s\uFEFF\xA0]+$/g;w.fn=w.prototype={jquery:"3.3.1",constructor:w,length:0,toArray:function(){return o.call(this)},get:function(e){return null==e?o.call(this):e<0?this[e+this.length]:this[e]},pushStack:function(e){var t=w.merge(this.constructor(),e);return t.prevObject=this,t},each:function(e){return w.each(this,e)},map:function(e){return this.pushStack(w.map(this,function(t,n){return e.call(t,n,t)}))},slice:function(){return this.pushStack(o.apply(this,arguments))},first:function(){return this.eq(0)},last:function(){return this.eq(-1)},eq:function(e){var t=this.length,n=+e+(e<0?t:0);return this.pushStack(n>=0&&n<t?[this[n]]:[])},end:function(){return this.prevObject||this.constructor()},push:s,sort:n.sort,splice:n.splice},w.extend=w.fn.extend=function(){var e,t,n,r,i,o,a=arguments[0]||{},s=1,u=arguments.length,l=!1;for("boolean"==typeof a&&(l=a,a=arguments[s]||{},s++),"object"==typeof a||g(a)||(a={}),s===u&&(a=this,s--);s<u;s++)if(null!=(e=arguments[s]))for(t in e)n=a[t],a!==(r=e[t])&&(l&&r&&(w.isPlainObject(r)||(i=Array.isArray(r)))?(i?(i=!1,o=n&&Array.isArray(n)?n:[]):o=n&&w.isPlainObject(n)?n:{},a[t]=w.extend(l,o,r)):void 0!==r&&(a[t]=r));return a},w.extend({expando:"jQuery"+("3.3.1"+Math.random()).replace(/\D/g,""),isReady:!0,error:function(e){throw new Error(e)},noop:function(){},isPlainObject:function(e){var t,n;return!(!e||"[object Object]"!==c.call(e))&&(!(t=i(e))||"function"==typeof(n=f.call(t,"constructor")&&t.constructor)&&p.call(n)===d)},isEmptyObject:function(e){var t;for(t in e)return!1;return!0},globalEval:function(e){m(e)},each:function(e,t){var n,r=0;if(C(e)){for(n=e.length;r<n;r++)if(!1===t.call(e[r],r,e[r]))break}else for(r in e)if(!1===t.call(e[r],r,e[r]))break;return e},trim:function(e){return null==e?"":(e+"").replace(T,"")},makeArray:function(e,t){var n=t||[];return null!=e&&(C(Object(e))?w.merge(n,"string"==typeof e?[e]:e):s.call(n,e)),n},inArray:function(e,t,n){return null==t?-1:u.call(t,e,n)},merge:function(e,t){for(var n=+t.length,r=0,i=e.length;r<n;r++)e[i++]=t[r];return e.length=i,e},grep:function(e,t,n){for(var r,i=[],o=0,a=e.length,s=!n;o<a;o++)(r=!t(e[o],o))!==s&&i.push(e[o]);return i},map:function(e,t,n){var r,i,o=0,s=[];if(C(e))for(r=e.length;o<r;o++)null!=(i=t(e[o],o,n))&&s.push(i);else for(o in e)null!=(i=t(e[o],o,n))&&s.push(i);return a.apply([],s)},guid:1,support:h}),"function"==typeof Symbol&&(w.fn[Symbol.iterator]=n[Symbol.iterator]),w.each("Boolean Number String Function Array Date RegExp Object Error Symbol".split(" "),function(e,t){l["[object "+t+"]"]=t.toLowerCase()});function C(e){var t=!!e&&"length"in e&&e.length,n=x(e);return!g(e)&&!y(e)&&("array"===n||0===t||"number"==typeof t&&t>0&&t-1 in e)}var E=function(e){var t,n,r,i,o,a,s,u,l,c,f,p,d,h,g,y,v,m,x,b="sizzle"+1*new Date,w=e.document,T=0,C=0,E=ae(),k=ae(),S=ae(),D=function(e,t){return e===t&&(f=!0),0},N={}.hasOwnProperty,A=[],j=A.pop,q=A.push,L=A.push,H=A.slice,O=function(e,t){for(var n=0,r=e.length;n<r;n++)if(e[n]===t)return n;return-1},P="checked|selected|async|autofocus|autoplay|controls|defer|disabled|hidden|ismap|loop|multiple|open|readonly|required|scoped",M="[\\x20\\t\\r\\n\\f]",R="(?:\\\\.|[\\w-]|[^\0-\\xa0])+",I="\\["+M+"*("+R+")(?:"+M+"*([*^$|!~]?=)"+M+"*(?:'((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\"|("+R+"))|)"+M+"*\\]",W=":("+R+")(?:\\((('((?:\\\\.|[^\\\\'])*)'|\"((?:\\\\.|[^\\\\\"])*)\")|((?:\\\\.|[^\\\\()[\\]]|"+I+")*)|.*)\\)|)",$=new RegExp(M+"+","g"),B=new RegExp("^"+M+"+|((?:^|[^\\\\])(?:\\\\.)*)"+M+"+$","g"),F=new RegExp("^"+M+"*,"+M+"*"),_=new RegExp("^"+M+"*([>+~]|"+M+")"+M+"*"),z=new RegExp("="+M+"*([^\\]'\"]*?)"+M+"*\\]","g"),X=new RegExp(W),U=new RegExp("^"+R+"$"),V={ID:new RegExp("^#("+R+")"),CLASS:new RegExp("^\\.("+R+")"),TAG:new RegExp("^("+R+"|[*])"),ATTR:new RegExp("^"+I),PSEUDO:new RegExp("^"+W),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+M+"*(even|odd|(([+-]|)(\\d*)n|)"+M+"*(?:([+-]|)"+M+"*(\\d+)|))"+M+"*\\)|)","i"),bool:new RegExp("^(?:"+P+")$","i"),needsContext:new RegExp("^"+M+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)Line truncated
@@ -21,7 +21,8 @@
        <DT><A HREF="https://duckduckgo.com/?q=firefox+export+bookmarks&t=ffhp&ia=web" ADD_DATE="1497562974" LAST_MODIFIED="1497562974" ICON_URI="https://duckduckgo.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAG7UlEQVRYha2XbYwbVxWG798oorJQhYKqSJPxzJ35gWDVComiKrK86/H4Y+2J21/5kIyUgiKk1GqArVpEXQlKSlFNm35IFMVREgQNJE4KAsqCpiGtsrsIJjvO2N6xvbPj9SbZJBvjdsmGzcYvP+xxxt6PbBFHeiTP9T3nfe+ZO+NrQjYZmsJ4aoo3WU+w2fqTO7S5BA839Sd3aPUEm60p3qSmMJ7N1n1gWArnqyfYbD3Bw2F2F9eYVTi1h11cwz2nlvDmLIXz/c/CmsJ46gk2W9/Fo76LR11hrXqCTVmKyKxvVmTqCTZVV1irm5dgs5+5I5bCDNQV1prtCDsrsRRmwE7QdE3hVOd7Z05N4VQ7QdOWwgx0O9dToz2+KXE7zjVqCg87xmWcYrU4p9YUHpsizqmOaTvGZWoKDzvONR5ownSJO5vJjrPZ8ruv4E5+HHfy41j862ksvDGC+p7HHmjEjrfbX1O8SceEuZ4JTWE8M1GuYcfvi8/Edmh2nMeN359Ef9xbbOL6ywdgx/kNmYnt0BwTdpzHTJRrrLkn7DibteM87GEuoymMx+qI23EehQMhFA6EYP/mXTSbzR4jn7x/DFdSsQ1NWB0T9jCXaY+x2R7xsiL6ZmIUVozX2mb47EyMwuHKMzHcna+v6oI7br4+AndOP3aczxJCiBXjtZkYRVkRfV0Dlbg3Z3UGy4ros2IUDuaer2HxxrUNxZ24+vxeuHP7cdevxL05QgghRUVkrBhFtbP66SinupOMt38IAJiYvIpj7+n4p351lfDpP5p4Nv0XfPzLcxsamI5yKiGEVGO8ZsUoiorIkGpESE0PU1QjQsoMMAPTwxRu9DPtDfjzX+mb6kL9YBz9NdyYAWbArUkqUW+uOtx2Ux7mMtVhCjf66ftPwNx8E8Xp62sKH3z5HCbys/jXuWPor+GmPMxliorIVIcpKlFvjlTCrGWG+AYhhExHOLUapXDzj/eyXZHHd7+Fw79Q1zTxpfhreOH1P+G2Po7+Gm6mI+3bUAmzViXMWqQSpSh3BqdCfKMSpXBz/s3DXZFvvHAKj+9+C3PzzVUG5uabmJtv4rY+jkqUYvopDtd+8Ag++fXnsXjOgyvf245KlGKqs9hyhFMrUQpSiVCU5Y6rCEU/Y+mDq579/mgtTaK1+De0bp3AXfNbuP3BQ1jRt/RwM7OtW5MQQsoyp1YiFKQcoTA7BsoRin70b4YxOzu7tvJ/ZnCvGlwl5rA8sRW33vkCrL1sT01CCDFlTi1HKIgZvm+gJPENM0zhprTrK5icnFxTf6UkdsUWz3q6NLIPY+6729FfywxTlKT2LTBlTjXDFGQqxFqlEGu5B/u5+IdzGxpYnti6ppgZppjew/aOOYsNsdZUiLVIKeTNTYUpNFlkzCCXmQpT9PPhkcNrGmgtTWLF2IYVfQv+/cFDuPHati6fnvVgeWIrVvQtuPajL3ZrmUEuo8kiMxWmKIW8OVIICalSiMKQvUkzwAyUQhT9jD/3NBYWFtY0URvZi9nvbEcj+zDunP9czx749KwHV77/SE8tM8AMGLI3WQpRFEJCimiyyJRCFCWp/SouyqzVb0DbH4ZlWesaWMv0WhTl9q0uSbxWCrW7TgghpBDk1KJMkZdFX0HyKkWZop/1NuKNk0dWzV2PguRV8rLoK8oUhWB7LxBCCMnLoq8gUxidLhiSN1eQKdx8dOoEAODm0jWUbt03s1QtdOf8+ekv4/A7T+C3hx5Ff74htX/9DInXCp3F9pwJDInmCkEBeVlMqwrjMSSqFYICHM7/NI3l5WW8Xz2B/aMSRj7ah1z5OEbtM3jp6E68dHQn9o9K2D8qIfOzr8Oda0hUUxXGk5fFdOc6t+pEpPoYz2WJNoygAEMWk51rzQgKMIICxlN7sbCw0DWwf1TCU2cC3c9u0kd3wsm7LFFN9TEeQxaTneuG6lvnmG74mAEn0fC3TegSzRlBAdoeP0zT7DGw+3cbG9AlmlN9jMfwt8WNoADD94CTseEXk5clAZclAZOSmCaEkMKgV7nsZ62xsTFo1z/uCv1k7EU8rx7CG39/FcWTr+DUs4/hx28/gZMjj1qFQa9CCCGTkph26hl+MbmhuLsTeoA28pKASwGqaUPtDXPhwgWf3ZzKniy+ad1c6j2mrTQb1qTEZ5wVakOi71KAanlJgB6gjQeuvD9UH+PRB2kuHxCQDwjQhzhV94vJu7du5FqtltpqtdR79+5p3bdiq6U2tYtp3S8m9SFO7eYNtm/DZxJ3hzYk+vQhTtUDAhyqh/ahemgfZl99DvPHj2D++BHMvPhtGMpXu3P0IU51Ovd/iYuyyGh+IaUN0twlP2tNDglwc8nPWtogzWl+IXVRXv8PbH/8Fyy0G0H5udT6AAAAAElFTkSuQmCC">firefox export bookmarks at DuckDuckGo</A>
        <DT><A HREF="https://duckduckgo.com/?q=archive+firefox+bookmarks&t=ffab&ia=web" ADD_DATE="1497562974" LAST_MODIFIED="1497562974" ICON_URI="https://duckduckgo.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAG7UlEQVRYha2XbYwbVxWG798oorJQhYKqSJPxzJ35gWDVComiKrK86/H4Y+2J21/5kIyUgiKk1GqArVpEXQlKSlFNm35IFMVREgQNJE4KAsqCpiGtsrsIJjvO2N6xvbPj9SbZJBvjdsmGzcYvP+xxxt6PbBFHeiTP9T3nfe+ZO+NrQjYZmsJ4aoo3WU+w2fqTO7S5BA839Sd3aPUEm60p3qSmMJ7N1n1gWArnqyfYbD3Bw2F2F9eYVTi1h11cwz2nlvDmLIXz/c/CmsJ46gk2W9/Fo76LR11hrXqCTVmKyKxvVmTqCTZVV1irm5dgs5+5I5bCDNQV1prtCDsrsRRmwE7QdE3hVOd7Z05N4VQ7QdOWwgx0O9dToz2+KXE7zjVqCg87xmWcYrU4p9YUHpsizqmOaTvGZWoKDzvONR5ownSJO5vJjrPZ8ruv4E5+HHfy41j862ksvDGC+p7HHmjEjrfbX1O8SceEuZ4JTWE8M1GuYcfvi8/Edmh2nMeN359Ef9xbbOL6ywdgx/kNmYnt0BwTdpzHTJRrrLkn7DibteM87GEuoymMx+qI23EehQMhFA6EYP/mXTSbzR4jn7x/DFdSsQ1NWB0T9jCXaY+x2R7xsiL6ZmIUVozX2mb47EyMwuHKMzHcna+v6oI7br4+AndOP3aczxJCiBXjtZkYRVkRfV0Dlbg3Z3UGy4ros2IUDuaer2HxxrUNxZ24+vxeuHP7cdevxL05QgghRUVkrBhFtbP66SinupOMt38IAJiYvIpj7+n4p351lfDpP5p4Nv0XfPzLcxsamI5yKiGEVGO8ZsUoiorIkGpESE0PU1QjQsoMMAPTwxRu9DPtDfjzX+mb6kL9YBz9NdyYAWbArUkqUW+uOtx2Ux7mMtVhCjf66ftPwNx8E8Xp62sKH3z5HCbys/jXuWPor+GmPMxliorIVIcpKlFvjlTCrGWG+AYhhExHOLUapXDzj/eyXZHHd7+Fw79Q1zTxpfhreOH1P+G2Po7+Gm6mI+3bUAmzViXMWqQSpSh3BqdCfKMSpXBz/s3DXZFvvHAKj+9+C3PzzVUG5uabmJtv4rY+jkqUYvopDtd+8Ag++fXnsXjOgyvf245KlGKqs9hyhFMrUQpSiVCU5Y6rCEU/Y+mDq579/mgtTaK1+De0bp3AXfNbuP3BQ1jRt/RwM7OtW5MQQsoyp1YiFKQcoTA7BsoRin70b4YxOzu7tvJ/ZnCvGlwl5rA8sRW33vkCrL1sT01CCDFlTi1HKIgZvm+gJPENM0zhprTrK5icnFxTf6UkdsUWz3q6NLIPY+6729FfywxTlKT2LTBlTjXDFGQqxFqlEGu5B/u5+IdzGxpYnti6ppgZppjew/aOOYsNsdZUiLVIKeTNTYUpNFlkzCCXmQpT9PPhkcNrGmgtTWLF2IYVfQv+/cFDuPHati6fnvVgeWIrVvQtuPajL3ZrmUEuo8kiMxWmKIW8OVIICalSiMKQvUkzwAyUQhT9jD/3NBYWFtY0URvZi9nvbEcj+zDunP9czx749KwHV77/SE8tM8AMGLI3WQpRFEJCimiyyJRCFCWp/SouyqzVb0DbH4ZlWesaWMv0WhTl9q0uSbxWCrW7TgghpBDk1KJMkZdFX0HyKkWZop/1NuKNk0dWzV2PguRV8rLoK8oUhWB7LxBCCMnLoq8gUxidLhiSN1eQKdx8dOoEAODm0jWUbt03s1QtdOf8+ekv4/A7T+C3hx5Ff74htX/9DInXCp3F9pwJDInmCkEBeVlMqwrjMSSqFYICHM7/NI3l5WW8Xz2B/aMSRj7ah1z5OEbtM3jp6E68dHQn9o9K2D8qIfOzr8Oda0hUUxXGk5fFdOc6t+pEpPoYz2WJNoygAEMWk51rzQgKMIICxlN7sbCw0DWwf1TCU2cC3c9u0kd3wsm7LFFN9TEeQxaTneuG6lvnmG74mAEn0fC3TegSzRlBAdoeP0zT7DGw+3cbG9AlmlN9jMfwt8WNoADD94CTseEXk5clAZclAZOSmCaEkMKgV7nsZ62xsTFo1z/uCv1k7EU8rx7CG39/FcWTr+DUs4/hx28/gZMjj1qFQa9CCCGTkph26hl+MbmhuLsTeoA28pKASwGqaUPtDXPhwgWf3ZzKniy+ad1c6j2mrTQb1qTEZ5wVakOi71KAanlJgB6gjQeuvD9UH+PRB2kuHxCQDwjQhzhV94vJu7du5FqtltpqtdR79+5p3bdiq6U2tYtp3S8m9SFO7eYNtm/DZxJ3hzYk+vQhTtUDAhyqh/ahemgfZl99DvPHj2D++BHMvPhtGMpXu3P0IU51Ovd/iYuyyGh+IaUN0twlP2tNDglwc8nPWtogzWl+IXVRXv8PbH/8Fyy0G0H5udT6AAAAAElFTkSuQmCC">archive firefox bookmarks at DuckDuckGo</A>
        <DT><A HREF="https://github.com/nodiscc" ADD_DATE="1497562974" LAST_MODIFIED="1497562974" ICON_URI="https://assets-cdn.github.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADC0lEQVRYhb2Wv2tUQRDHv9Fwefdul535zp7GRrGz9RcoiEIK/wdBjBhFOwsVxMI/wMbSQrSyiWKwE0t/NHbRIoURJdqI2iRBBZVocxeez93L5XI68Jo3M/v5zuzOvgf0b9F7P2Uid8zsZTRb3tpu/9rabv+KZstm9tJE7njvpwDEdazb28qy3E1y2sgfXeBaj5E/SE6XZbl7YLCqBpI3jVzpF5wQskLypqqGdVcdyTeDgutPJN/03Q3n3ISRi8OCV7qx6Jyb6AkPIez9F/CqiBDC3hxcjXxbS5gz8v4gooxc7OTO1d6/DSHoXwKMvF1fRFWPd9xj3vupaPahOnYUeUyRx9WxjGYfOqM4BgCqejwh7nb90O1LVVGW5Z5qnKqGRqOxC8CmRBM3NRqNXfUTX5blnsza+6rVz2SC9vc8NH1YWZb7M1s0040ZN/JnKkhEJjcqQEQmMwJ+AhiHOnc2FdAm3wPgRgUAIMl3KYY6dxYkp1NO7/2pIcABAN77UykGyWnUx6TbnnVfnz1MVUNqm42cQzT7+pfDbGFY8K6Z2UKdE82+ItWaaDY/bAHR7HWKlRRg5NKwBRi5lBSQdQBbhsjfki00isxm7oATw6KLyInkVovMguStjPMFgNEh8EejyIvMGN6CtlrHUs5OwPWN0klez62vrdYxAPDR7EsuyMh7AMYHYG8z8l5u3Wj2BYAHAKjqjVVVqtecc0co8qQi4nubvMsQznS+YpsTwFHfbB5U584ZOWPk9xy8w7mxmqmq26PZtw5shSIXAIwZ+TzRkRkAIwkBIybyoBe0Uv03Vd3+R7aIXK0GNZvNA0VR7Kx/SFT1UK7nzrnD/QgQkaup/FGSzyqVPupq895PqeqVVqt1NNP+rrXXgpN8hh7TNV69MkXkfA9YymSN1r/GWge6KIod0Wx+NSnGpxrCJe/9SVW9DKAYREA0my+KYke/lZDkw8wVLesVQPIhBvi5GWEIp6PZx0EFRPITQziD9NT0bY4iF8zslZGf0fndztiYkZ/N7BVDuAjAbQT8X+w36KQvZccCoxkAAAAASUVORK5CYII=">nodiscc (nodiscc) · GitHub</A>
-        <DT><A HREF="https://github.com/pirate/bookmark-archiver#troubleshooting" ADD_DATE="1497562975" LAST_MODIFIED="1497562975" ICON_URI="https://assets-cdn.github.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADC0lEQVRYhb2Wv2tUQRDHv9Fwefdul535zp7GRrGz9RcoiEIK/wdBjBhFOwsVxMI/wMbSQrSyiWKwE0t/NHbRIoURJdqI2iRBBZVocxeez93L5XI68Jo3M/v5zuzOvgf0b9F7P2Uid8zsZTRb3tpu/9rabv+KZstm9tJE7njvpwDEdazb28qy3E1y2sgfXeBaj5E/SE6XZbl7YLCqBpI3jVzpF5wQskLypqqGdVcdyTeDgutPJN/03Q3n3ISRi8OCV7qx6Jyb6AkPIez9F/CqiBDC3hxcjXxbS5gz8v4gooxc7OTO1d6/DSHoXwKMvF1fRFWPd9xj3vupaPahOnYUeUyRx9WxjGYfOqM4BgCqejwh7nb90O1LVVGW5Z5qnKqGRqOxC8CmRBM3NRqNXfUTX5blnsza+6rVz2SC9vc8NH1YWZb7M1s0040ZN/JnKkhEJjcqQEQmMwJ+AhiHOnc2FdAm3wPgRgUAIMl3KYY6dxYkp1NO7/2pIcABAN77UykGyWnUx6TbnnVfnz1MVUNqm42cQzT7+pfDbGFY8K6Z2UKdE82+ItWaaDY/bAHR7HWKlRRg5NKwBRi5lBSQdQBbhsjfki00isxm7oATw6KLyInkVovMguStjPMFgNEh8EejyIvMGN6CtlrHUs5OwPWN0klez62vrdYxAPDR7EsuyMh7AMYHYG8z8l5u3Wj2BYAHAKjqjVVVqtecc0co8qQi4nubvMsQznS+YpsTwFHfbB5U584ZOWPk9xy8w7mxmqmq26PZtw5shSIXAIwZ+TzRkRkAIwkBIybyoBe0Uv03Vd3+R7aIXK0GNZvNA0VR7Kx/SFT1UK7nzrnD/QgQkaup/FGSzyqVPupq895PqeqVVqt1NNP+rrXXgpN8hh7TNV69MkXkfA9YymSN1r/GWge6KIod0Wx+NSnGpxrCJe/9SVW9DKAYREA0my+KYke/lZDkw8wVLesVQPIhBvi5GWEIp6PZx0EFRPITQziD9NT0bY4iF8zslZGf0fndztiYkZ/N7BVDuAjAbQT8X+w36KQvZccCoxkAAAAASUVORK5CYII=">pirate/bookmark-archiver · Github</A>
+        <DT><A HREF="https://github.com/pirate/ArchiveBox#troubleshooting" ADD_DATE="1497562975" LAST_MODIFIED="1497562975" ICON_URI="https://assets-cdn.github.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADC0lEQVRYhb2Wv2tUQRDHv9Fwefdul535zp7GRrGz9RcoiEIK/wdBjBhFOwsVxMI/wMbSQrSyiWKwE0t/NHbRIoURJdqI2iRBBZVocxeez93L5XI68Jo3M/v5zuzOvgf0b9F7P2Uid8zsZTRb3tpu/9rabv+KZstm9tJE7njvpwDEdazb28qy3E1y2sgfXeBaj5E/SE6XZbl7YLCqBpI3jVzpF5wQskLypqqGdVcdyTeDgutPJN/03Q3n3ISRi8OCV7qx6Jyb6AkPIez9F/CqiBDC3hxcjXxbS5gz8v4gooxc7OTO1d6/DSHoXwKMvF1fRFWPd9xj3vupaPahOnYUeUyRx9WxjGYfOqM4BgCqejwh7nb90O1LVVGW5Z5qnKqGRqOxC8CmRBM3NRqNXfUTX5blnsza+6rVz2SC9vc8NH1YWZb7M1s0040ZN/JnKkhEJjcqQEQmMwJ+AhiHOnc2FdAm3wPgRgUAIMl3KYY6dxYkp1NO7/2pIcABAN77UykGyWnUx6TbnnVfnz1MVUNqm42cQzT7+pfDbGFY8K6Z2UKdE82+ItWaaDY/bAHR7HWKlRRg5NKwBRi5lBSQdQBbhsjfki00isxm7oATw6KLyInkVovMguStjPMFgNEh8EejyIvMGN6CtlrHUs5OwPWN0klez62vrdYxAPDR7EsuyMh7AMYHYG8z8l5u3Wj2BYAHAKjqjVVVqtecc0co8qQi4nubvMsQznS+YpsTwFHfbB5U584ZOWPk9xy8w7mxmqmq26PZtw5shSIXAIwZ+TzRkRkAIwkBIybyoBe0Uv03Vd3+R7aIXK0GNZvNA0VR7Kx/SFT1UK7nzrnD/QgQkaup/FGSzyqVPupq895PqeqVVqt1NNP+rrXXgpN8hh7TNV69MkXkfA9YymSN1r/GWge6KIod0Wx+NSnGpxrCJe/9SVW9DKAYREA0my+KYke/lZDkw8wVLesVQPIhBvi5GWEIp6PZx0EFRPITQziD9NT0bY4iF8zslZGf0fndztiYkZ/N7BVDuAjAbQT8X+w36KQvZccCoxkAAAAASUVORK5CYII=">pirate/ArchiveBox · Github</A>
+        <DT><A HREF="http://www.cs.unc.edu/~fabian/papers/foniks-oak11.pdf" ADD_DATE="1497562976" LAST_MODIFIED="1497562976" ICON_URI="https://assets-cdn.github.com/favicon.ico" ICON="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAADC0lEQVRYhb2Wv2tUQRDHv9Fwefdul535zp7GRrGz9RcoiEIK/wdBjBhFOwsVxMI/wMbSQrSyiWKwE0t/NHbRIoURJdqI2iRBBZVocxeez93L5XI68Jo3M/v5zuzOvgf0b9F7P2Uid8zsZTRb3tpu/9rabv+KZstm9tJE7njvpwDEdazb28qy3E1y2sgfXeBaj5E/SE6XZbl7YLCqBpI3jVzpF5wQskLypqqGdVcdyTeDgutPJN/03Q3n3ISRi8OCV7qx6Jyb6AkPIez9F/CqiBDC3hxcjXxbS5gz8v4gooxc7OTO1d6/DSHoXwKMvF1fRFWPd9xj3vupaPahOnYUeUyRx9WxjGYfOqM4BgCqejwh7nb90O1LVVGW5Z5qnKqGRqOxC8CmRBM3NRqNXfUTX5blnsza+6rVz2SC9vc8NH1YWZb7M1s0040ZN/JnKkhEJjcqQEQmMwJ+AhiHOnc2FdAm3wPgRgUAIMl3KYY6dxYkp1NO7/2pIcABAN77UykGyWnUx6TbnnVfnz1MVUNqm42cQzT7+pfDbGFY8K6Z2UKdE82+ItWaaDY/bAHR7HWKlRRg5NKwBRi5lBSQdQBbhsjfki00isxm7oATw6KLyInkVovMguStjPMFgNEh8EejyIvMGN6CtlrHUs5OwPWN0klez62vrdYxAPDR7EsuyMh7AMYHYG8z8l5u3Wj2BYAHAKjqjVVVqtecc0co8qQi4nubvMsQznS+YpsTwFHfbB5U584ZOWPk9xy8w7mxmqmq26PZtw5shSIXAIwZ+TzRkRkAIwkBIybyoBe0Uv03Vd3+R7aIXK0GNZvNA0VR7Kx/SFT1UK7nzrnD/QgQkaup/FGSzyqVPupq895PqeqVVqt1NNP+rrXXgpN8hh7TNV69MkXkfA9YymSN1r/GWge6KIod0Wx+NSnGpxrCJe/9SVW9DKAYREA0my+KYke/lZDkw8wVLesVQPIhBvi5GWEIp6PZx0EFRPITQziD9NT0bY4iF8zslZGf0fndztiYkZ/N7BVDuAjAbQT8X+w36KQvZccCoxkAAAAASUVORK5CYII=">Phonotactic Reconstruction of Encrypted VoIP Conversations</A>
        <DT><A HREF="https://www.ghacks.net/2009/07/23/firefox-bookmarks-archiver/" ADD_DATE="1497562974" LAST_MODIFIED="1497562974" ICON_URI="https://www.ghacks.net/wp-content/uploads/2005/10/favicon.ico" ICON="data:image/png;base64,AAABAAEAEBAAAAEAGABoAwAAFgAAACgAAAAQAAAAIAAAAAEAGAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAD+/P76/Pzs7Ozg2uXNyeW/ueOXmuKJleOpsOrAvubNzOff3ebv7+77/f39/f7+/v79/vv+/f77+v3s7vjIy++5vO+truh9keNggNuRpemrtua/0Orz+f79/P38/v3+/v78/f78/vna7PpyltlTfNxsjOSdruq4u+6grPFlidxxl+SdtummvN7w+Pz9/v3+//79/fv49/2cqttZgNtFeOJGeeNbhdaqtubGxeuyw/VuleFvm9ybuui2yOb5/fv+/v79/Pzj4ve8vumzuvCZqO1ojeCUsvSJq+pyneGAqOqYuupsm+F7ouCkveXk6/b8/f74+fqhsuSRouios/G5v+vDx+mjtu9Jht43h+k+iuJ5qeG61vdsnNeNsevU3vD7/v7h7fxUgNRDe+JSg96JpObByu+dt+s6ieM3jOkzj+dBkt281PK31vFmm9XC1/T7/f7d6fV8ntxfkuRFhN1Bg+NtnOalyvRElts4kuc4k+c9leWn0ezm7/ebweaNuuD3/P7z9fnY1+vV3PGsyu1bld9Cjd9nouCFw+1QnN48mONWoNnC5/fY5/XP6vqHstr1+v7x+P3AyeTAz+7f6fTS5vdhpeA5kOlssOG54/iv2e/C5/iZ0ux2s9ve8/u91un3/P72+/yCpNpIh96JtOXg8vvZ7/xjp90+meGQxezy+/ze8vpcpto/ouG+4/Tw9vb8/v78/fu91O9eldxMjdp+seLp+P7I6vdNoeJIouDC5vbw/f6Jxek9oOWi0Or4/P3+/v73/v3v+f3E1+qex/FOktubyur2+/yez+05pN93ueXy/fy94/dQo9e+3PH7/f7//v78/v38/f32+PfA2exanuBVmuHg8vza8vxVqeBPpN3a8vva6/Wm0+P0+v38/v7+/v79/f77/P77/v3t+fyrz+lwp9XT7PXl+fxost1VptbQ6fjz+Pz0/vv+/vz9/v7+/v/8/vz8/v38/vz8/fz5/P3x+v7y/Pv5+vnJ5PPQ8Pn2/Pz8/fz+/v38/f7+/v3///4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA">Firefox Bookmarks Archiver - gHacks Tech News</A>
    </DL><p>
    <DT><H3 ADD_DATE="1409779227" LAST_MODIFIED="1470506008" PERSONAL_TOOLBAR_FOLDER="true">Bookmarks Toolbar</H3>
@@ -3,4 +3,6 @@
 {"href":"https:\/\/github.com\/google\/styleguide","description":"GitHub - google\/styleguide: Style guides for Google-originated open-source projects","extended":"","meta":"15a8d50f7295f18ccb6dd19cb689c68a","hash":"1028bf9872d8e4ea1b1858f4044abb58","time":"2016-02-24T08:49:25Z","shared":"no","toread":"no","tags":"code.style.guide programming reference web.dev"},
 {"href":"http:\/\/en.wikipedia.org\/wiki\/List_of_XML_and_HTML_character_entity_references","description":"List of XML and HTML character entity references - Wikipedia, the free encyclopedia","extended":"","meta":"6683a70f0f59c92c0bfd0bce653eab69","hash":"344d975c6251a8d460971fa2c43d9bbb","time":"2014-06-16T04:17:15Z","shared":"no","toread":"no","tags":"html reference web.dev typography"},
 {"href":"https:\/\/pushover.net\/","description":"Pushover: Simple Notifications for Android, iOS, and Desktop","extended":"","meta":"1e68511234d9390d10b7772c8ccc4b9e","hash":"bb93374ead8a937b18c7c46e13168a7d","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"app android"},
-{"href":"http:\/\/www.reddit.com\/r\/Android","description":"r\/android","extended":"","meta":"18a973f09c9cc0608c116967b64e0419","hash":"910293f019c2f4bb1a749fb937ba58e3","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"reddit android"}]
+{"href":"http:\/\/www.reddit.com\/r\/Android","description":"r\/android","extended":"","meta":"18a973f09c9cc0608c116967b64e0419","hash":"910293f019c2f4bb1a749fb937ba58e3","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"reddit android 1"},
+{"href":"http:\/\/www.reddit.com\/r\/Android2","description":"r\/android","extended":"","meta":"18a973f09c9cc0608c116967b64e0419","hash":"910293f019c2f4bb1a749fb937ba58e2","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"reddit android 2"},
+{"href":"http:\/\/www.reddit.com\/r\/Android3","description":"r\/android","extended":"","meta":"18a973f09c9cc0608c116967b64e0419","hash":"910293f019c2f4bb1a749fb937ba58e4","time":"2014-06-14T15:51:42Z","shared":"no","toread":"no","tags":"reddit android 3"}]
@@ -17,6 +17,7 @@
 <li><a href="https://chromium.googlesource.com/chromium/src/+/master/docs/linux_sandboxing.md" time_added="1493869649" tags="">Linux Sandboxing</a></li>
 <li><a href="https://hackernoon.com/rems-and-ems-and-why-you-probably-dont-need-them-664b9ce1e09f" time_added="1493694979" tags="">rems and ems, and why you probably don’t need them – Hacker Noon</a></li>
 <li><a href="https://wiki.archlinux.org/index.php/full_system_backup_with_rsync" time_added="1493581911" tags="">Full system backup with rsync - ArchWiki</a></li>
+<li><a href="https://www.youtube.com/watch?v=iNnAQpAHfmA" time_added="1493581911" tags="">SingUnltd. - Nature Boy (Flying Lotus Massage Situation Sample?! )</a></li>
 		</ul>

 		<h1>Read Archive</h1>
@@ -0,0 +1,228 @@
+<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
+	xmlns:content="http://purl.org/rss/1.0/modules/content/"
+	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
+	xmlns:dc="http://purl.org/dc/elements/1.1/"
+	xmlns:atom="http://www.w3.org/2005/Atom"
+	>
+
+<channel>
+
+<title>My Reading List: Read and Unread</title>
+<description>Items I've saved to read</description>
+<link>http://readitlaterlist.com/users/nikisweeting/feed/all</link>
+<atom:link href="http://readitlaterlist.com/users/nikisweeting/feed/all" rel="self" type="application/rss+xml" />
+
+
+<item>
+<title><![CDATA[Cell signaling]]></title>
+<category>Unread</category>
+<link>https://en.wikipedia.org/wiki/Cell_signaling</link>
+<guid>https://en.wikipedia.org/wiki/Cell_signaling</guid>
+<pubDate>Mon, 30 Oct 2017 01:12:10 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Hayflick limit]]></title>
+<category>Unread</category>
+<link>https://en.wikipedia.org/wiki/Hayflick_limit</link>
+<guid>https://en.wikipedia.org/wiki/Hayflick_limit</guid>
+<pubDate>Mon, 30 Oct 2017 01:11:38 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Even moderate drinking by parents can upset children – study]]></title>
+<category>Unread</category>
+<link>https://theguardian.com/society/2017/oct/18/even-moderate-drinking-by-parents-can-upset-children-study?CMP=Share_AndroidApp_Signal</link>
+<guid>https://theguardian.com/society/2017/oct/18/even-moderate-drinking-by-parents-can-upset-children-study?CMP=Share_AndroidApp_Signal</guid>
+<pubDate>Mon, 30 Oct 2017 01:11:30 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[How Merkle trees enable the decentralized Web]]></title>
+<category>Unread</category>
+<link>https://taravancil.com/blog/how-merkle-trees-enable-decentralized-web</link>
+<guid>https://taravancil.com/blog/how-merkle-trees-enable-decentralized-web</guid>
+<pubDate>Mon, 30 Oct 2017 01:11:30 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Inertial navigation system]]></title>
+<category>Unread</category>
+<link>https://en.wikipedia.org/wiki/Inertial_navigation_system</link>
+<guid>https://en.wikipedia.org/wiki/Inertial_navigation_system</guid>
+<pubDate>Mon, 30 Oct 2017 01:10:10 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Dead reckoning]]></title>
+<category>Unread</category>
+<link>https://en.wikipedia.org/wiki/Dead_reckoning</link>
+<guid>https://en.wikipedia.org/wiki/Dead_reckoning</guid>
+<pubDate>Mon, 30 Oct 2017 01:10:08 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Calling Rust From Python]]></title>
+<category>Unread</category>
+<link>https://bheisler.github.io/post/calling-rust-in-python</link>
+<guid>https://bheisler.github.io/post/calling-rust-in-python</guid>
+<pubDate>Mon, 30 Oct 2017 01:04:33 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Why would anyone choose Docker over fat binaries?]]></title>
+<category>Unread</category>
+<link>http://smashcompany.com/technology/why-would-anyone-choose-docker-over-fat-binaries</link>
+<guid>http://smashcompany.com/technology/why-would-anyone-choose-docker-over-fat-binaries</guid>
+<pubDate>Sun, 29 Oct 2017 14:57:25 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[]]></title>
+<category>Unread</category>
+<link>https://heml.io</link>
+<guid>https://heml.io</guid>
+<pubDate>Sun, 29 Oct 2017 14:55:26 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[A surprising amount of people want to be in North Korea]]></title>
+<category>Unread</category>
+<link>https://blog.benjojo.co.uk/post/north-korea-dprk-bgp-geoip-fruad</link>
+<guid>https://blog.benjojo.co.uk/post/north-korea-dprk-bgp-geoip-fruad</guid>
+<pubDate>Sat, 28 Oct 2017 05:41:41 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Learning a Hierarchy]]></title>
+<category>Unread</category>
+<link>https://blog.openai.com/learning-a-hierarchy</link>
+<guid>https://blog.openai.com/learning-a-hierarchy</guid>
+<pubDate>Thu, 26 Oct 2017 16:43:48 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[High Performance Browser Networking]]></title>
+<category>Unread</category>
+<link>https://hpbn.co</link>
+<guid>https://hpbn.co</guid>
+<pubDate>Wed, 25 Oct 2017 19:05:24 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[What tender and juicy drama is going on at your school/workplace?]]></title>
+<category>Unread</category>
+<link>https://reddit.com/r/AskReddit/comments/78nc2a/what_tender_and_juicy_drama_is_going_on_at_your/dovab2v</link>
+<guid>https://reddit.com/r/AskReddit/comments/78nc2a/what_tender_and_juicy_drama_is_going_on_at_your/dovab2v</guid>
+<pubDate>Wed, 25 Oct 2017 18:05:58 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Using an SSH Bastion Host]]></title>
+<category>Unread</category>
+<link>https://blog.scottlowe.org/2015/11/21/using-ssh-bastion-host</link>
+<guid>https://blog.scottlowe.org/2015/11/21/using-ssh-bastion-host</guid>
+<pubDate>Wed, 25 Oct 2017 11:38:47 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Let's Define &quot;undefined&quot; | NathanShane.me]]></title>
+<category>Unread</category>
+<link>https://nathanshane.me/blog/let's-define-undefined</link>
+<guid>https://nathanshane.me/blog/let's-define-undefined</guid>
+<pubDate>Wed, 25 Oct 2017 11:32:59 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Control theory]]></title>
+<category>Unread</category>
+<link>https://en.wikipedia.org/wiki/Control_theory#Closed-loop_transfer_function</link>
+<guid>https://en.wikipedia.org/wiki/Control_theory#Closed-loop_transfer_function</guid>
+<pubDate>Tue, 24 Oct 2017 22:57:43 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[J012-86-intractable.pdf]]></title>
+<category>Unread</category>
+<link>http://mit.edu/~jnt/Papers/J012-86-intractable.pdf</link>
+<guid>http://mit.edu/~jnt/Papers/J012-86-intractable.pdf</guid>
+<pubDate>Tue, 24 Oct 2017 22:56:32 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Dynamic Programming: First Principles]]></title>
+<category>Unread</category>
+<link>http://flawlessrhetoric.com/Dynamic-Programming-First-Principles</link>
+<guid>http://flawlessrhetoric.com/Dynamic-Programming-First-Principles</guid>
+<pubDate>Tue, 24 Oct 2017 22:56:30 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[What Would Happen If There Were No Number 6?]]></title>
+<category>Unread</category>
+<link>https://fivethirtyeight.com/features/what-would-happen-if-there-were-no-number-6</link>
+<guid>https://fivethirtyeight.com/features/what-would-happen-if-there-were-no-number-6</guid>
+<pubDate>Tue, 24 Oct 2017 22:21:59 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Ten Basic Rules for Adventure]]></title>
+<category>Unread</category>
+<link>https://outsideonline.com/2252916/10-basic-rules-adventure</link>
+<guid>https://outsideonline.com/2252916/10-basic-rules-adventure</guid>
+<pubDate>Tue, 24 Oct 2017 20:56:25 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Insects Are In Serious Trouble]]></title>
+<category>Unread</category>
+<link>https://theatlantic.com/science/archive/2017/10/oh-no/543390?single_page=true</link>
+<guid>https://theatlantic.com/science/archive/2017/10/oh-no/543390?single_page=true</guid>
+<pubDate>Mon, 23 Oct 2017 23:10:10 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Netflix/bless]]></title>
+<category>Unread</category>
+<link>https://github.com/Netflix/bless</link>
+<guid>https://github.com/Netflix/bless</guid>
+<pubDate>Mon, 23 Oct 2017 23:04:46 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Getting Your First 10 Customers]]></title>
+<category>Unread</category>
+<link>https://stripe.com/atlas/guides/starting-sales</link>
+<guid>https://stripe.com/atlas/guides/starting-sales</guid>
+<pubDate>Mon, 23 Oct 2017 22:27:36 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[GPS Hardware]]></title>
+<category>Unread</category>
+<link>https://novasummits.com/gps-hardware</link>
+<guid>https://novasummits.com/gps-hardware</guid>
+<pubDate>Mon, 23 Oct 2017 04:44:40 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Bicycle Tires and Tubes]]></title>
+<category>Unread</category>
+<link>http://sheldonbrown.com/tires.html#pressure</link>
+<guid>http://sheldonbrown.com/tires.html#pressure</guid>
+<pubDate>Mon, 23 Oct 2017 01:28:32 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Tire light is on]]></title>
+<category>Unread</category>
+<link>https://reddit.com/r/Justrolledintotheshop/comments/77zm9e/tire_light_is_on/doqbshe</link>
+<guid>https://reddit.com/r/Justrolledintotheshop/comments/77zm9e/tire_light_is_on/doqbshe</guid>
+<pubDate>Mon, 23 Oct 2017 01:21:42 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Bad_Salish_Boo ?? on Twitter]]></title>
+<category>Unread</category>
+<link>https://t.co/PDLlNjACv9</link>
+<guid>https://t.co/PDLlNjACv9</guid>
+<pubDate>Sat, 21 Oct 2017 06:48:07 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Is an Open Marriage a Happier Marriage?]]></title>
+<category>Unread</category>
+<link>https://nytimes.com/2017/05/11/magazine/is-an-open-marriage-a-happier-marriage.html</link>
+<guid>https://nytimes.com/2017/05/11/magazine/is-an-open-marriage-a-happier-marriage.html</guid>
+<pubDate>Fri, 20 Oct 2017 13:08:52 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[The Invention of Monogamy]]></title>
+<category>Unread</category>
+<link>https://thenib.com/the-invention-of-monogamy</link>
+<guid>https://thenib.com/the-invention-of-monogamy</guid>
+<pubDate>Fri, 20 Oct 2017 12:19:00 -0500</pubDate>
+</item>
+<item>
+<title><![CDATA[Google Chrome May Add a Permission to Stop In-Browser Cryptocurrency Miners]]></title>
+<category>Unread</category>
+<link>https://bleepingcomputer.com/news/google/google-chrome-may-add-a-permission-to-stop-in-browser-cryptocurrency-miners</link>
+<guid>https://bleepingcomputer.com/news/google/google-chrome-may-add-a-permission-to-stop-in-browser-cryptocurrency-miners</guid>
+<pubDate>Fri, 20 Oct 2017 03:57:41 -0500</pubDate>
+</item>
+</channel>
+
+</rss>
@@ -0,0 +1,500 @@
+import os
+import re
+import sys
+import time
+import json
+import requests
+
+from datetime import datetime
+from subprocess import run, PIPE, DEVNULL
+from multiprocessing import Process
+from urllib.parse import quote
+
+from config import (
+    IS_TTY,
+    OUTPUT_PERMISSIONS,
+    REPO_DIR,
+    SOURCES_DIR,
+    OUTPUT_DIR,
+    ARCHIVE_DIR,
+    TIMEOUT,
+    TERM_WIDTH,
+    SHOW_PROGRESS,
+    ANSI,
+    CHROME_BINARY,
+    FETCH_WGET,
+    FETCH_PDF,
+    FETCH_SCREENSHOT,
+    FETCH_DOM,
+    FETCH_FAVICON,
+    FETCH_AUDIO,
+    FETCH_VIDEO,
+    SUBMIT_ARCHIVE_DOT_ORG,
+)
+
+# URL helpers
+without_scheme = lambda url: url.replace('http://', '').replace('https://', '').replace('ftp://', '')
+without_query = lambda url: url.split('?', 1)[0]
+without_hash = lambda url: url.split('#', 1)[0]
+without_path = lambda url: url.split('/', 1)[0]
+domain = lambda url: without_hash(without_query(without_path(without_scheme(url))))
+base_url = lambda url: without_scheme(url)  # uniq base url used to dedupe links
+
+short_ts = lambda ts: ts.split('.')[0]
+
+
+def check_dependencies():
+    """Check that all necessary dependencies are installed, and have valid versions"""
+
+    python_vers = float('{}.{}'.format(sys.version_info.major, sys.version_info.minor))
+    if python_vers < 3.5:
+        print('{}[X] Python version is not new enough: {} (>3.5 is required){}'.format(ANSI['red'], python_vers, ANSI['reset']))
+        print('    See https://github.com/pirate/ArchiveBox#troubleshooting for help upgrading your Python installation.')
+        raise SystemExit(1)
+
+    if FETCH_PDF or FETCH_SCREENSHOT or FETCH_DOM:
+        if run(['which', CHROME_BINARY], stdout=DEVNULL).returncode:
+            print('{}[X] Missing dependency: {}{}'.format(ANSI['red'], CHROME_BINARY, ANSI['reset']))
+            print('    Run ./setup.sh, then confirm it was installed with: {} --version'.format(CHROME_BINARY))
+            print('    See https://github.com/pirate/ArchiveBox for help.')
+            raise SystemExit(1)
+
+        # parse chrome --version e.g. Google Chrome 61.0.3114.0 canary / Chromium 59.0.3029.110 built on Ubuntu, running on Ubuntu 16.04
+        try:
+            result = run([CHROME_BINARY, '--version'], stdout=PIPE)
+            version_str = result.stdout.decode('utf-8')
+            version_lines = re.sub("(Google Chrome|Chromium) (\\d+?)\\.(\\d+?)\\.(\\d+?).*?$", "\\2", version_str).split('\n')
+            version = [l for l in version_lines if l.isdigit()][-1]
+            if int(version) < 59:
+                print(version_lines)
+                print('{red}[X] Chrome version must be 59 or greater for headless PDF, screenshot, and DOM saving{reset}'.format(**ANSI))
+                print('    See https://github.com/pirate/ArchiveBox for help.')
+                raise SystemExit(1)
+        except (IndexError, TypeError, OSError):
+            print('{red}[X] Failed to parse Chrome version, is it installed properly?{reset}'.format(**ANSI))
+            print('    Run ./setup.sh, then confirm it was installed with: {} --version'.format(CHROME_BINARY))
+            print('    See https://github.com/pirate/ArchiveBox for help.')
+            raise SystemExit(1)
+
+    if FETCH_WGET:
+        if run(['which', 'wget'], stdout=DEVNULL).returncode or run(['wget', '--version'], stdout=DEVNULL).returncode:
+            print('{red}[X] Missing dependency: wget{reset}'.format(**ANSI))
+            print('    Run ./setup.sh, then confirm it was installed with: {} --version'.format('wget'))
+            print('    See https://github.com/pirate/ArchiveBox for help.')
+            raise SystemExit(1)
+
+    if FETCH_FAVICON or SUBMIT_ARCHIVE_DOT_ORG:
+        if run(['which', 'curl'], stdout=DEVNULL).returncode or run(['curl', '--version'], stdout=DEVNULL).returncode:
+            print('{red}[X] Missing dependency: curl{reset}'.format(**ANSI))
+            print('    Run ./setup.sh, then confirm it was installed with: {} --version'.format('curl'))
+            print('    See https://github.com/pirate/ArchiveBox for help.')
+            raise SystemExit(1)
+
+    if FETCH_AUDIO or FETCH_VIDEO:
+        if run(['which', 'youtube-dl'], stdout=DEVNULL).returncode or run(['youtube-dl', '--version'], stdout=DEVNULL).returncode:
+            print('{red}[X] Missing dependency: youtube-dl{reset}'.format(**ANSI))
+            print('    Run ./setup.sh, then confirm it was installed with: {} --version'.format('youtube-dl'))
+            print('    See https://github.com/pirate/ArchiveBox for help.')
+            raise SystemExit(1)
+
+
+def chmod_file(path, cwd='.', permissions=OUTPUT_PERMISSIONS, timeout=30):
+    """chmod -R <permissions> <cwd>/<path>"""
+
+    if not os.path.exists(os.path.join(cwd, path)):
+        raise Exception('Failed to chmod: {} does not exist (did the previous step fail?)'.format(path))
+
+    chmod_result = run(['chmod', '-R', permissions, path], cwd=cwd, stdout=DEVNULL, stderr=PIPE, timeout=timeout)
+    if chmod_result.returncode == 1:
+        print('     ', chmod_result.stderr.decode())
+        raise Exception('Failed to chmod {}/{}'.format(cwd, path))
+
+
+def progress(seconds=TIMEOUT, prefix=''):
+    """Show a (subprocess-controlled) progress bar with a <seconds> timeout,
+       returns end() function to instantly finish the progress
+    """
+
+    if not SHOW_PROGRESS:
+        return lambda: None
+
+    chunk = '█' if sys.stdout.encoding == 'UTF-8' else '#'
+    chunks = TERM_WIDTH - len(prefix) - 20  # number of progress chunks to show (aka max bar width)
+
+    def progress_bar(seconds=seconds, prefix=prefix):
+        """show timer in the form of progress bar, with percentage and seconds remaining"""
+        try:
+            for s in range(seconds * chunks):
+                progress = s / chunks / seconds * 100
+                bar_width = round(progress/(100/chunks))
+
+                # ████████████████████           0.9% (1/60sec)
+                sys.stdout.write('\r{0}{1}{2}{3} {4}% ({5}/{6}sec)'.format(
+                    prefix,
+                    ANSI['green'],
+                    (chunk * bar_width).ljust(chunks),
+                    ANSI['reset'],
+                    round(progress, 1),
+                    round(s/chunks),
+                    seconds,
+                ))
+                sys.stdout.flush()
+                time.sleep(1 / chunks)
+
+            # ██████████████████████████████████ 100.0% (60/60sec)
+            sys.stdout.write('\r{0}{1}{2}{3} {4}% ({5}/{6}sec)\n'.format(
+                prefix,
+                ANSI['red'],
+                chunk * chunks,
+                ANSI['reset'],
+                100.0,
+                seconds,
+                seconds,
+            ))
+            sys.stdout.flush()
+        except KeyboardInterrupt:
+            print()
+            pass
+
+    p = Process(target=progress_bar)
+    p.start()
+
+    def end():
+        """immediately finish progress and clear the progressbar line"""
+        nonlocal p
+        if p is None: # protect from double termination
+            return
+
+        p.terminate()
+        p = None
+
+        sys.stdout.write('\r{}{}\r'.format((' ' * TERM_WIDTH), ANSI['reset']))  # clear whole terminal line
+        sys.stdout.flush()
+
+    return end
+
+def pretty_path(path):
+    """convert paths like .../ArchiveBox/archiver/../output/abc into output/abc"""
+    return path.replace(REPO_DIR + '/', '')
+
+
+def download_url(url):
+    """download a given url's content into downloads/domain.txt"""
+
+    if not os.path.exists(SOURCES_DIR):
+        os.makedirs(SOURCES_DIR)
+
+    ts = str(datetime.now().timestamp()).split('.', 1)[0]
+
+    source_path = os.path.join(SOURCES_DIR, '{}-{}.txt'.format(domain(url), ts))
+
+    print('[*] [{}] Downloading {} > {}'.format(
+        datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
+        url,
+        pretty_path(source_path),
+    ))
+    end = progress(TIMEOUT, prefix='      ')
+    try:
+        downloaded_xml = requests.get(url).content.decode()
+        end()
+    except Exception as e:
+        end()
+        print('[!] Failed to download {}\n'.format(url))
+        print('    ', e)
+        raise SystemExit(1)
+
+    with open(source_path, 'w', encoding='utf-8') as f:
+        f.write(downloaded_xml)
+
+    return source_path
+
+def str_between(string, start, end=None):
+    """(<abc>12345</def>, <abc>, </def>)  ->  12345"""
+
+    content = string.split(start, 1)[-1]
+    if end is not None:
+        content = content.rsplit(end, 1)[0]
+
+    return content
+
+def get_link_type(link):
+    """Certain types of links need to be handled specially, this figures out when that's the case"""
+
+    if link['base_url'].endswith('.pdf'):
+        return 'PDF'
+    elif link['base_url'].rsplit('.', 1) in ('pdf', 'png', 'jpg', 'jpeg', 'svg', 'bmp', 'gif', 'tiff', 'webp'):
+        return 'image'
+    elif 'wikipedia.org' in link['domain']:
+        return 'wiki'
+    elif 'youtube.com' in link['domain']:
+        return 'youtube'
+    elif 'soundcloud.com' in link['domain']:
+        return 'soundcloud'
+    elif 'youku.com' in link['domain']:
+        return 'youku'
+    elif 'vimeo.com' in link['domain']:
+        return 'vimeo'
+    return None
+
+def merge_links(a, b):
+    """deterministially merge two links, favoring longer field values over shorter,
+    and "cleaner" values over worse ones.
+    """
+    longer = lambda key: a[key] if len(a[key]) > len(b[key]) else b[key]
+    earlier = lambda key: a[key] if a[key] < b[key] else b[key]
+    
+    url = longer('url')
+    longest_title = longer('title')
+    cleanest_title = a['title'] if '://' not in a['title'] else b['title']
+    link = {
+        'timestamp': earlier('timestamp'),
+        'url': url,
+        'domain': domain(url),
+        'base_url': base_url(url),
+        'tags': longer('tags'),
+        'title': longest_title if '://' not in longest_title else cleanest_title,
+        'sources': list(set(a.get('sources', []) + b.get('sources', []))),
+    }
+    link['type'] = get_link_type(link)
+    return link
+
+def find_link(folder, links):
+    """for a given archive folder, find the corresponding link object in links"""
+    url = parse_url(folder)
+    if url:
+        for link in links:
+            if (link['base_url'] in url) or (url in link['url']):
+                return link
+
+    timestamp = folder.split('.')[0]
+    for link in links:
+        if link['timestamp'].startswith(timestamp):
+            if link['domain'] in os.listdir(os.path.join(ARCHIVE_DIR, folder)):
+                return link      # careful now, this isn't safe for most ppl
+            if link['domain'] in parse_url(folder):
+                return link
+    return None
+
+
+def parse_url(folder):
+    """for a given archive folder, figure out what url it's for"""
+    link_json = os.path.join(ARCHIVE_DIR, folder, 'index.json')
+    if os.path.exists(link_json):
+        with open(link_json, 'r') as f:
+            try:
+                link_json = f.read().strip()
+                if link_json:
+                    link = json.loads(link_json)
+                    return link['base_url']
+            except ValueError:
+                print('File contains invalid JSON: {}!'.format(link_json))
+
+    archive_org_txt = os.path.join(ARCHIVE_DIR, folder, 'archive.org.txt')
+    if os.path.exists(archive_org_txt):
+        with open(archive_org_txt, 'r') as f:
+            original_link = f.read().strip().split('/http', 1)[-1]
+            with_scheme = 'http{}'.format(original_link)
+            return with_scheme
+
+    return ''
+
+def manually_merge_folders(source, target):
+    """prompt for user input to resolve a conflict between two archive folders"""
+
+    if not IS_TTY:
+        return
+
+    fname = lambda path: path.split('/')[-1]
+
+    print('    {} and {} have conflicting files, which do you want to keep?'.format(fname(source), fname(target)))
+    print('      - [enter]: do nothing (keep both)')
+    print('      - a:       prefer files from {}'.format(source))
+    print('      - b:       prefer files from {}'.format(target))
+    print('      - q:       quit and resolve the conflict manually')
+    try:
+        answer = input('> ').strip().lower()
+    except KeyboardInterrupt:
+        answer = 'q'
+
+    assert answer in ('', 'a', 'b', 'q'), 'Invalid choice.'
+
+    if answer == 'q':
+        print('\nJust run ArchiveBox again to pick up where you left off.')
+        raise SystemExit(0)
+    elif answer == '':
+        return
+
+    files_in_source = set(os.listdir(source))
+    files_in_target = set(os.listdir(target))
+    for file in files_in_source:
+        if file in files_in_target:
+            to_delete = target if answer == 'a' else source
+            run(['rm', '-Rf', os.path.join(to_delete, file)])
+        run(['mv', os.path.join(source, file), os.path.join(target, file)])
+
+    if not set(os.listdir(source)):
+        run(['rm', '-Rf', source])
+
+def fix_folder_path(archive_path, link_folder, link):
+    """given a folder, merge it to the canonical 'correct' path for the given link object"""
+    source = os.path.join(archive_path, link_folder)
+    target = os.path.join(archive_path, link['timestamp'])
+
+    url_in_folder = parse_url(source)
+    if not (url_in_folder in link['base_url']
+            or link['base_url'] in url_in_folder):
+        raise ValueError('The link does not match the url for this folder.')
+
+    if not os.path.exists(target):
+        # target doesn't exist so nothing needs merging, simply move A to B
+        run(['mv', source, target])
+    else:
+        # target folder exists, check for conflicting files and attempt manual merge
+        files_in_source = set(os.listdir(source))
+        files_in_target = set(os.listdir(target))
+        conflicting_files = files_in_source & files_in_target
+
+        if not conflicting_files:
+            for file in files_in_source:
+                run(['mv', os.path.join(source, file), os.path.join(target, file)])
+
+    if os.path.exists(source):
+        files_in_source = set(os.listdir(source))
+        if files_in_source:
+            manually_merge_folders(source, target)
+        else:
+            run(['rm', '-R', source])
+
+
+def migrate_data():
+    # migrate old folder to new OUTPUT folder
+    old_dir = os.path.join(REPO_DIR, 'html')
+    if os.path.exists(old_dir):
+        print('[!] WARNING: Moved old output folder "html" to new location: {}'.format(OUTPUT_DIR))
+        run(['mv', old_dir, OUTPUT_DIR], timeout=10)
+
+
+def cleanup_archive(archive_path, links):
+    """move any incorrectly named folders to their canonical locations"""
+    
+    # for each folder that exists, see if we can match it up with a known good link
+    # if we can, then merge the two folders (TODO: if not, move it to lost & found)
+
+    unmatched = []
+    bad_folders = []
+
+    if not os.path.exists(archive_path):
+        return
+
+    for folder in os.listdir(archive_path):
+        try:
+            files = os.listdir(os.path.join(archive_path, folder))
+        except NotADirectoryError:
+            continue
+        
+        if files:
+            link = find_link(folder, links)
+            if link is None:
+                unmatched.append(folder)
+                continue
+            
+            if folder != link['timestamp']:
+                bad_folders.append((folder, link))
+        else:
+            # delete empty folders
+            run(['rm', '-R', os.path.join(archive_path, folder)])
+    
+    if bad_folders and IS_TTY and input('[!] Cleanup archive? y/[n]: ') == 'y':
+        print('[!] Fixing {} improperly named folders in archive...'.format(len(bad_folders)))
+        for folder, link in bad_folders:
+            fix_folder_path(archive_path, folder, link)
+    elif bad_folders:
+        print('[!] Warning! {} folders need to be merged, fix by running ArchiveBox.'.format(len(bad_folders)))
+
+    if unmatched:
+        print('[!] Warning! {} unrecognized folders in html/archive/'.format(len(unmatched)))
+        print('    '+ '\n    '.join(unmatched))
+
+
+def wget_output_path(link, look_in=None):
+    """calculate the path to the wgetted .html file, since wget may
+    adjust some paths to be different than the base_url path.
+
+    See docs on wget --adjust-extension (-E)
+    """
+
+    # if we have it stored, always prefer the actual output path to computed one
+    if link.get('latest', {}).get('wget'):
+        return link['latest']['wget']
+
+    urlencode = lambda s: quote(s, encoding='utf-8', errors='replace')
+
+    if link['type'] in ('PDF', 'image'):
+        return urlencode(link['base_url'])
+
+    # Since the wget algorithm to for -E (appending .html) is incredibly complex
+    # instead of trying to emulate it here, we just look in the output folder
+    # to see what html file wget actually created as the output
+    wget_folder = link['base_url'].rsplit('/', 1)[0].split('/')
+    look_in = os.path.join(ARCHIVE_DIR, link['timestamp'], *wget_folder)
+
+    if look_in and os.path.exists(look_in):
+        html_files = [
+            f for f in os.listdir(look_in)
+            if re.search(".+\\.[Hh][Tt][Mm][Ll]?$", f, re.I | re.M)
+        ]
+        if html_files:
+            return urlencode(os.path.join(*wget_folder, html_files[0]))
+
+    return None
+
+    # If finding the actual output file didn't work, fall back to the buggy
+    # implementation of the wget .html appending algorithm
+    # split_url = link['url'].split('#', 1)
+    # query = ('%3F' + link['url'].split('?', 1)[-1]) if '?' in link['url'] else ''
+
+    # if re.search(".+\\.[Hh][Tt][Mm][Ll]?$", split_url[0], re.I | re.M):
+    #     # already ends in .html
+    #     return urlencode(link['base_url'])
+    # else:
+    #     # .html needs to be appended
+    #     without_scheme = split_url[0].split('://', 1)[-1].split('?', 1)[0]
+    #     if without_scheme.endswith('/'):
+    #         if query:
+    #             return urlencode('#'.join([without_scheme + 'index.html' + query + '.html', *split_url[1:]]))
+    #         return urlencode('#'.join([without_scheme + 'index.html', *split_url[1:]]))
+    #     else:
+    #         if query:
+    #             return urlencode('#'.join([without_scheme + '/index.html' + query + '.html', *split_url[1:]]))
+    #         elif '/' in without_scheme:
+    #             return urlencode('#'.join([without_scheme + '.html', *split_url[1:]]))
+    #         return urlencode(link['base_url'] + '/index.html')
+
+
+def derived_link_info(link):
+    """extend link info with the archive urls and other derived data"""
+
+    link_info = {
+        **link,
+        'date': datetime.fromtimestamp(float(link['timestamp'])).strftime('%Y-%m-%d %H:%M'),
+        'google_favicon_url': 'https://www.google.com/s2/favicons?domain={domain}'.format(**link),
+        'favicon_url': 'archive/{timestamp}/favicon.ico'.format(**link),
+        'files_url': 'archive/{timestamp}/index.html'.format(**link),
+        'archive_url': 'archive/{}/{}'.format(link['timestamp'], wget_output_path(link) or 'index.html'),
+        'pdf_link': 'archive/{timestamp}/output.pdf'.format(**link),
+        'screenshot_link': 'archive/{timestamp}/screenshot.png'.format(**link),
+        'dom_link': 'archive/{timestamp}/output.html'.format(**link),
+        'archive_org_url': 'https://web.archive.org/web/{base_url}'.format(**link),
+    }
+
+    # PDF and images are handled slightly differently
+    # wget, screenshot, & pdf urls all point to the same file
+    if link['type'] in ('PDF', 'image'):
+        link_info.update({
+            'archive_url': 'archive/{timestamp}/{base_url}'.format(**link),
+            'pdf_link': 'archive/{timestamp}/{base_url}'.format(**link),
+            'screenshot_link': 'archive/{timestamp}/{base_url}'.format(**link),
+            'dom_link': 'archive/{timestamp}/{base_url}'.format(**link),
+            'title': '{title} ({type})'.format(**link),
+        })
+    return link_info
@@ -0,0 +1 @@
+../archiver/archive.py
@@ -0,0 +1 @@
+archivebox
@@ -0,0 +1,35 @@
+#!/bin/bash
+
+REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )"; cd .. && pwd )"
+
+if [[ "$1" == "--chrome" ]]; then
+    # Google Chrome / Chromium
+    default=$(ls ~/Library/Application\ Support/Google/Chrome/Default/History)
+    if [[ -e "$2" ]]; then
+        cp "$2" "$REPO_DIR/output/sources/chrome_history.db.tmp"
+    else
+        echo "Defaulting to history db: $default"
+        echo "Optionally specify the path to a different sqlite history database as the 2nd argument."
+        cp "$default" "$REPO_DIR/output/sources/chrome_history.db.tmp"
+    fi
+    sqlite3 "$REPO_DIR/output/sources/chrome_history.db.tmp" "SELECT \"[\" || group_concat(json_object('timestamp', last_visit_time, 'description', title, 'href', url)) || \"]\" FROM urls;" > "$REPO_DIR/output/sources/chrome_history.json"
+    rm "$REPO_DIR"/output/sources/chrome_history.db.*
+    echo "Chrome history exported to:"
+    echo "    output/sources/chrome_history.json"
+fi
+
+if [[ "$1" == "--firefox" ]]; then
+    # Firefox
+    default=$(ls ~/Library/Application\ Support/Firefox/Profiles/*.default/places.sqlite)
+    if [[ -e "$2" ]]; then
+        cp "$2" "$REPO_DIR/output/sources/firefox_history.db.tmp"
+    else
+        echo "Defaulting to history db: $default"
+        echo "Optionally specify the path to a different sqlite history database as the 2nd argument."
+        cp "$default" "$REPO_DIR/output/sources/firefox_history.db.tmp"
+    fi
+    sqlite3 "$REPO_DIR/output/sources/firefox_history.db.tmp" "SELECT \"[\" || group_concat(json_object('timestamp', last_visit_date, 'description', title, 'href', url)) || \"]\" FROM moz_places;" > "$REPO_DIR/output/sources/firefox_history.json"
+    rm "$REPO_DIR"/output/sources/firefox_history.db.*
+    echo "Firefox history exported to:"
+    echo "    output/sources/firefox_history.json"
+fi
@@ -0,0 +1 @@
+../archiver/purge.py
@@ -1,9 +1,9 @@
 #!/bin/bash
-# Bookmark Archiver Setup Script
+# ArchiveBox Setup Script
 # Nick Sweeting 2017 | MIT License
-# https://github.com/pirate/bookmark-archiver
+# https://github.com/pirate/ArchiveBox

-echo "[i] Installing bookmark-archiver dependencies. 📦"
+echo "[i] Installing ArchiveBox dependencies. 📦"
 echo ""
 echo "    You may be prompted for a password in order to install the following dependencies:"
 echo "        - Chromium Browser   (see README for Google-Chrome instructions instead)"
@@ -28,12 +28,13 @@ if which apt-get > /dev/null; then
        sudo ln -s "$(which google-chrome)" /usr/bin/chromium-browser
    elif which chromium-browser; then
        echo "[i] chromium-browser already installed, using existing installation."
+        chromium-browser --version
    else
        echo "[+] Installing chromium-browser..."
        apt install chromium-browser -y
    fi
    echo "[+] Installing python3, wget, curl..."
-    apt install python3 wget curl
+    apt install -y python3 python3-distutils python3-requests wget curl

 # On Mac:
 elif which brew > /dev/null; then   # 🐍 eye of newt
@@ -43,6 +44,7 @@ elif which brew > /dev/null; then   # 🐍 eye of newt
        chmod +x /usr/local/bin/chromium-browser

    elif which chromium-browser; then
+        brew cask upgrade chromium-browser
        echo "[+] Linking /usr/local/bin/chromium-browser -> /Applications/Chromium.app"
        echo -e '#!/bin/bash\n/Applications/Chromium.app/Contents/MacOS/Chromium "$@"' > /usr/local/bin/chromium-browser
        chmod +x /usr/local/bin/chromium-browser
@@ -82,5 +84,5 @@ echo ""
 echo "[X] Failed to install some dependencies! ‼️"
 echo "    - Try the Manual Setup instructions in the README.md"
 echo "    - Try the Troubleshooting: Dependencies instructions in the README.md"
-echo "    - Open an issue on github to get help: https://github.com/pirate/bookmark-archiver/issues"
+echo "    - Open an issue on github to get help: https://github.com/pirate/ArchiveBox/issues"
 exit 1
@@ -0,0 +1 @@
+setup-archivebox
@@ -0,0 +1,10 @@
+server {
+    listen                      80 default_server;
+    server_name                 _;
+
+    root                        /data/output/;
+    index                       index.html;
+    autoindex                   on;
+
+    try_files                   $uri $uri/ $uri.html =404;
+}
@@ -1,90 +0,0 @@
-<html lang="en">
-    <head>
-        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
-        <title>Archived Sites</title>
-        <style>
-            html, body {{
-                width: 100%;
-                height: 100%;
-                font-size: 20px;
-                font-weight: 200;
-                text-align: center;
-                margin: 0px;
-                padding: 0px;
-                font-family: "Gill Sans", Helvetica, sans-serif;
-            }}
-            header {{
-                background-color: #aa1e55;
-                color: white;
-                padding: 10px;
-            }}
-            header h1 {{
-                font-weight: 300;
-                color: black;
-                margin-top: 10px;
-                margin-bottom: 12px;
-            }}
-            header h1 small {{
-                color: white;
-                font-size:0.5em;
-            }}
-            header h1 small a {{
-                text-decoration: none;
-                color: orange;
-                opacity: 0.6
-                font-weight: 300;
-            }}
-            header h1 small a:hover {{
-                opacity: 1;
-            }}
-            table {{
-                padding: 6px;
-                width: 100%;
-            }}
-            table thead th {{
-                font-weight: 400;
-            }}
-            tbody tr:nth-child(odd) {{
-               background-color: #ffebeb;
-            }}
-            table tr td {{
-                white-space: nowrap;
-                overflow: hidden;
-                padding-bottom: 0.4em;
-                padding-top: 0.4em;
-                padding-left: 2px;
-            }}
-            table tr td img {{
-                height: 24px;
-                padding: 0px;
-                padding-right: 5px;
-                text-indent: -10000px;
-            }}
-        </style>
-    </head>
-    <body>
-        <header>
-            <h1 title="Last modified {}">
-                <img src="https://nicksweeting.com/images/archive.png" height="36px">
-                Archived Sites <img src="https://getpocket.com/favicon.ico" height="36px"> <br/>
-                <small>
-                    Exported with: <a href="https://github.com/pirate/bookmark-archiver">Bookmark Archiver</a>
-                </small>
-            </h1>
-        </header>
-        <table style="width:100%;height: 90%; overflow-y: scroll;table-layout: fixed">
-            <thead>
-                <tr>
-                    <th style="width: 120px;"><img src="https://getpocket.com/favicon.ico" height="12px"> Starred</th>
-                    <th style="width: 45vw;">Saved Article</th>
-                    <th style="width: 50px">Files</th>
-                    <th style="width: 50px">PDF</th>
-                    <th style="width: 60px;font-size:0.8em;">Screenshot</th>
-                    <th style="width: 50px">A.org</th>
-                    <th style="width: 100px;whitespace:nowrap;overflow-x:scroll;display:block">Original URL</th>
-                </tr>
-            </thead>
-            <tbody>{}</tbody>
-        </table>
-    </body>
-</html>
@@ -0,0 +1 @@
+bin/setup-archivebox
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+import json
+import os
+from os.path import dirname, pardir, join
+from subprocess import check_output, check_call
+from tempfile import TemporaryDirectory
+from typing import List
+
+import pytest
+
+
+ARCHIVER_BIN = join(dirname(__file__), pardir, 'archive')
+
+
+class Helper:
+    def __init__(self, output_dir: str):
+        self.output_dir = output_dir
+
+    def run(self, links, env=None, env_defaults=None):
+        if env_defaults is None:
+            env_defaults = {
+                # we don't wanna spam archive.org witin our tests..
+                'SUBMIT_ARCHIVE_DOT_ORG': 'False',
+            }
+        if env is None:
+            env = {}
+
+        env = dict(**env_defaults, **env)
+
+        jj = []
+        for url in links:
+            jj.append({
+                'href': url,
+                'description': url,
+            })
+        input_json = join(self.output_dir, 'input.json')
+        with open(input_json, 'w') as fo:
+            json.dump(jj, fo)
+
+        if env is None:
+            env = {}
+        env['OUTPUT_DIR'] = self.output_dir
+        check_call(
+            [ARCHIVER_BIN, input_json],
+            env={**os.environ.copy(), **env},
+        )
+
+
+class TestArchiver:
+    def setup(self):
+        # self.tdir = TemporaryDirectory(dir='hello')
+        class AAA:
+            name = 'hello'
+        self.tdir = AAA()
+
+    def teardown(self):
+        pass
+        # self.tdir.cleanup()
+
+    @property
+    def output_dir(self):
+        return self.tdir.name
+
+    def test_fetch_favicon_false(self):
+        h = Helper(self.output_dir)
+
+        h.run(links=[
+            'https://google.com',
+        ], env={
+            'FETCH_FAVICON': 'False',
+        })
+        # for now no asserts, good enough if it isn't failing
+
+    def test_3000_links(self):
+        """
+        The pages are deliberatly unreachable. The tool should gracefully process all of them even though individual links are failing.
+        """
+        h = Helper(self.output_dir)
+
+        h.run(links=[
+            f'https://localhost:123/whatever_{i}.html' for i in range(3000)
+        ], env={
+            'FETCH_FAVICON': 'False',
+            'FETCH_SCREENSHOT': 'False',
+            'FETCH_PDF': 'False',
+            'FETCH_DOM': 'False',
+            'CHECK_SSL_VALIDITY': 'False',
+        })
+
+
+if __name__ == '__main__':
+    pytest.main([__file__])
				`@@ -0,0 +1 @@`
				table.dataTable{width:100%;margin:0 auto;clear:both;border-collapse:separate;border-spacing:0}table.dataTable thead th,table.dataTable tfoot th{font-weight:bold}table.dataTable thead th,table.dataTable thead td{padding:10px 18px;border-bottom:1px solid #111}table.dataTable thead th:active,table.dataTable thead td:active{outline:none}table.dataTable tfoot th,table.dataTable tfoot td{padding:10px 18px 6px 18px;border-top:1px solid #111}table.dataTable thead .sorting,table.dataTable thead .sorting_asc,table.dataTable thead .sorting_desc,table.dataTable thead .sorting_asc_disabled,table.dataTable thead .sorting_desc_disabled{cursor:pointer;*cursor:hand;background-repeat:no-repeat;background-position:center right}table.dataTable thead .sorting{background-image:url("sort_both.png")}table.dataTable thead .sorting_asc{background-image:url("sort_asc.png")}table.dataTable thead .sorting_desc{background-image:url("sort_desc.png")}table.dataTable thead .sorting_asc_disabled{background-image:url("sort_asc_disabled.png")}table.dataTable thead .sorting_desc_disabled{background-image:url("sort_desc_disabled.png")}table.dataTable tbody tr{background-color:#ffffff}table.dataTable tbody tr.selected{background-color:#B0BED9}table.dataTable tbody th,table.dataTable tbody td{padding:8px 10px}table.dataTable.row-border tbody th,table.dataTable.row-border tbody td,table.dataTable.display tbody th,table.dataTable.display tbody td{border-top:1px solid #ddd}table.dataTable.row-border tbody tr:first-child th,table.dataTable.row-border tbody tr:first-child td,table.dataTable.display tbody tr:first-child th,table.dataTable.display tbody tr:first-child td{border-top:none}table.dataTable.cell-border tbody th,table.dataTable.cell-border tbody td{border-top:1px solid #ddd;border-right:1px solid #ddd}table.dataTable.cell-border tbody tr th:first-child,table.dataTable.cell-border tbody tr td:first-child{border-left:1px solid #ddd}table.dataTable.cell-border tbody tr:first-child th,table.dataTable.cell-border tbody tr:first-child td{border-top:none}table.dataTable.stripe tbody tr.odd,table.dataTable.display tbody tr.odd{background-color:#f9f9f9}table.dataTable.stripe tbody tr.odd.selected,table.dataTable.display tbody tr.odd.selected{background-color:#acbad4}table.dataTable.hover tbody tr:hover,table.dataTable.display tbody tr:hover{background-color:#f6f6f6}table.dataTable.hover tbody tr:hover.selected,table.dataTable.display tbody tr:hover.selected{background-color:#aab7d1}table.dataTable.order-column tbody tr>.sorting_1,table.dataTable.order-column tbody tr>.sorting_2,table.dataTable.order-column tbody tr>.sorting_3,table.dataTable.display tbody tr>.sorting_1,table.dataTable.display tbody tr>.sorting_2,table.dataTable.display tbody tr>.sorting_3{background-color:#fafafa}table.dataTable.order-column tbody tr.selected>.sorting_1,table.dataTable.order-column tbody tr.selected>.sorting_2,table.dataTable.order-column tbody tr.selected>.sorting_3,table.dataTable.display tbody tr.selected>.sorting_1,table.dataTable.display tbody tr.selected>.sorting_2,table.dataTable.display tbody tr.selected>.sorting_3{background-color:#acbad5}table.dataTable.display tbody tr.odd>.sorting_1,table.dataTable.order-column.stripe tbody tr.odd>.sorting_1{background-color:#f1f1f1}table.dataTable.display tbody tr.odd>.sorting_2,table.dataTable.order-column.stripe tbody tr.odd>.sorting_2{background-color:#f3f3f3}table.dataTable.display tbody tr.odd>.sorting_3,table.dataTable.order-column.stripe tbody tr.odd>.sorting_3{background-color:whitesmoke}table.dataTable.display tbody tr.odd.selected>.sorting_1,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_1{background-color:#a6b4cd}table.dataTable.display tbody tr.odd.selected>.sorting_2,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_2{background-color:#a8b5cf}table.dataTable.display tbody tr.odd.selected>.sorting_3,table.dataTable.order-column.stripe tbody tr.odd.selected>.sorting_3{background-color:#a9b7d1}table.dataTable.display tbody tr.even>.sorting_1,table.dataTable.order-column.stripe tbody tr.even>.sorting_1{background-color:#fafafa}table.dataTable.display tbody tr.even>.sorting_2,table.dataTable.order-column.stripe tbody tr.even>.sorting_2{background-color:#fcfcfc}table.dataTable.display tbody tr.even>.sorting_3,table.dataTable.order-column.stripe tbody tr.even>.sorting_3{background-color:#fefefe}table.dataTable.display tbody tr.even.selected>.sorting_1,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_1{background-color:#acbad5}table.dataTable.display tbody tr.even.selected>.sorting_2,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_2{background-color:#aebcd6}table.dataTable.display tbody tr.even.selected>.sorting_3,table.dataTable.order-column.stripe tbody tr.even.selected>.sorting_3{background-color:#afbdd8}table.dataTable.display tbody tr:hover>.sorting_1,table.dataTable.order-column.hover tbody tr:hover>.sorting_1{background-color:#eaeaea}Line truncated