bench(cli-core): cold-cache 38× faster than @claude-flow/cli — alpha.0 published (ADR-100)

Persisted cold-cache benchmark proving the cli-core split fixes #1748
Issue 3's 30s MCP-startup timeout race:

  | Package                              | Cold     | Warm   | Files | Packed |
  | @claude-flow/cli-core@3.7.0-alpha.0  | 671 ms   | 361 ms |    22 | 22 KB  |
  | @claude-flow/cli@3.6.30              | 25.5 s   | 623 ms |   999 | 1.8 MB |

Results:
- 38× cold-cache speedup (25.5s → 0.67s)
- 80× packed-size reduction (1.8 MB → 22.3 KB)
- 45× file-count reduction (999 → 22)
- Fits comfortably under Claude Code's 30s MCP-startup timeout
  (cli is borderline at 25.5s — slow networks race and lose)

Caveats (also in the JSON's caveats array):
- cli-core@alpha.0 is foundation-only — types + output + MCP-tool
  types + validate-input. Does NOT yet expose memory/hooks command
  surface. ADR-100 §Discovery documents the tool-def/handler split
  needed for alpha.1 + alpha.2.
- The benchmark proves the COLD-CACHE thesis (smaller bundle = under
  timeout). It does NOT yet prove memory/hooks parity at speed —
  that requires the backend abstraction work in alpha.1.

Persisted to v3/docs/benchmarks/cli-core-cold-cache.json so future
regressions are visible.

Published: @claude-flow/cli-core@3.7.0-alpha.0 with --tag alpha
(latest also points to alpha.0 since this is the first publish).

Co-Authored-By: RuFlo <ruv@ruv.net>
This commit is contained in:
ReuvenandRuFlo committed 2026-05-05 13:43:35 -04:00
1 parent dda65b4b8a
commit 0acf557baa
1 file changed
+37
@@ -0,0 +1,37 @@
{
"benchmark": "cli-core-vs-cli-cold-cache",
"issue": "#1748 #3 (30s MCP-startup timeout race)",
"adr": "ADR-100",
"runAt": "2026-05-05T13:00:00Z",
"host": "darwin / macOS / npm 10+",
"method": "rm -rf ~/.npm/_npx; time npx -y <pkg>@alpha --version (4 runs total — cold + warm for each package)",
"results": {
"@claude-flow/cli-core@3.7.0-alpha.0": {
"packed_size_bytes": 22300,
"unpacked_size_bytes": 89900,
"files": 22,
"cold_cache_ms": 671,
"warm_cache_ms": 361,
"cold_warm_ratio": 1.86
},
"@claude-flow/cli@3.6.30": {
"packed_size_bytes": 1800000,
"unpacked_size_bytes": 9600000,
"files": 999,
"cold_cache_ms": 25496,
"warm_cache_ms": 623,
"cold_warm_ratio": 40.9
}
},
"improvement": {
"cold_cache_speedup": "38× (25.5s → 0.67s)",
"fits_under_30s_mcp_startup_timeout": "yes — 38× headroom vs cli's 25.5s which races the timeout",
"size_reduction_packed": "80× (1.8 MB → 22.3 KB)",
"size_reduction_files": "45× (999 → 22 files)"
},
"caveats": [
"cli-core@alpha.0 is foundation-only (types, output, MCP-tool types, validate-input). Does not yet expose memory/hooks command surface.",
"ADR-100 §Discovery documents that memory/hooks tools have deep ML deps requiring backend abstraction (alpha.1) and tool-definition/handler split (alpha.2) before parity is reached.",
"@claude-flow/cli's 25.5s cold-cache is under the 30s timeout on this connection but barely — slow networks would race and silently fail (the original #1748 #3 symptom)."
]
}