template_service.py held port allocation, magic-variable resolution and
the template catalog alongside install/update in one 2,800-line class.
Those three groups move verbatim into TemplatePortsMixin,
TemplateMagicMixin and TemplateCatalogMixin, each in its own module;
TemplateService inherits them, so its surface (106 attributes, same
classmethod/staticmethod kinds) and every caller are unchanged. Tests
that patched module globals now patch the module the code lives in.
With the remote index memoized, re-parsing every bundled YAML on each call
was the entire remaining cost of a Templates page load. Entries are now
memoized per file and keyed on (st_mtime_ns, st_size), so a sync or an edit
invalidates exactly the file that changed: measured on the real catalog,
list_local_templates() goes from 304ms to 0.89ms (118 templates), and the
full catalog listing from 1707ms to 1.26ms.
The writers need no invalidation call - mtime keying covers create, overwrite
and delete, the last because the listing only returns what the directory scan
finds. Cache entries are pruned to what is still on disk so a long-running
panel that churns templates cannot grow it without bound.
Failed parses are cached too, keyed on the same stat tuple, so one malformed
YAML no longer costs a parse on every render; it is retried the moment the
file changes. A file that fails to parse still does not claim its id, so a
valid file of the same name in the fallback directory can supply it - pinned
by a test, since that is easy to lose in a refactor.
Entries are deep-copied out: a shallow copy would share the nested
categories/repo/engine containers, so a caller appending to categories would
corrupt every later listing.
Also swaps os.listdir for os.scandir (DirEntry.stat() reuses data the scan
already returned) and sorts directory entries, which makes a same-id
collision (foo.yaml vs foo.yml) resolve identically on every platform instead
of depending on filesystem order.
Both bugs were latent only while DEFAULT_REPOS pointed at a 404. Now that
serverkit.ai proxies a live registry, they bite on every Templates page load.
list_all_templates() concatenated local + remote with no id merge, so every
bundled template was listed twice the moment a real registry answered.
Measured against the live registry: 224 entries for 116 unique ids. It now
merges by id with local winning, matching get_template(), which already
resolves local directories before any repo; among repos, the first enabled
one to claim an id keeps it.
fetch_remote_templates() ran inline on every listing with a 30s timeout and
no memoization, so one unreachable repo stalled the whole catalog on every
render. It is now memoized with a TTL (5m success, 60s failure), a 10s
timeout, and a last-good fallback so a transient blip cannot blank a catalog
the panel has already shown. The last-good payload is carried forward across
repeated failures rather than being overwritten by the first one. The cache
is dropped in save_config(), the single choke point for repo add/remove/
enable (sync_templates routes through it too), and force=True bypasses it.
11 proving tests, including the repeated-failure case and a regression test
that a cached entry cannot be corrupted by a caller mutating it.
Juan Denis