mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-10-28 07:38:37 +09:00
bench(cli-core): cold-cache 38× faster than @claude-flow/cli — alpha.0 published (ADR-100)
Persisted cold-cache benchmark proving the cli-core split fixes #1748 Issue 3's 30s MCP-startup timeout race: | Package | Cold | Warm | Files | Packed | | @claude-flow/cli-core@3.7.0-alpha.0 | 671 ms | 361 ms | 22 | 22 KB | | @claude-flow/cli@3.6.30 | 25.5 s | 623 ms | 999 | 1.8 MB | Results: - 38× cold-cache speedup (25.5s → 0.67s) - 80× packed-size reduction (1.8 MB → 22.3 KB) - 45× file-count reduction (999 → 22) - Fits comfortably under Claude Code's 30s MCP-startup timeout (cli is borderline at 25.5s — slow networks race and lose) Caveats (also in the JSON's caveats array): - cli-core@alpha.0 is foundation-only — types + output + MCP-tool types + validate-input. Does NOT yet expose memory/hooks command surface. ADR-100 §Discovery documents the tool-def/handler split needed for alpha.1 + alpha.2. - The benchmark proves the COLD-CACHE thesis (smaller bundle = under timeout). It does NOT yet prove memory/hooks parity at speed — that requires the backend abstraction work in alpha.1. Persisted to v3/docs/benchmarks/cli-core-cold-cache.json so future regressions are visible. Published: @claude-flow/cli-core@3.7.0-alpha.0 with --tag alpha (latest also points to alpha.0 since this is the first publish). Co-Authored-By: RuFlo <ruv@ruv.net>
This commit is contained in:
1 parent
dda65b4b8a
commit
0acf557baa
1 file changed
+37
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"benchmark": "cli-core-vs-cli-cold-cache",
|
||||
"issue": "#1748 #3 (30s MCP-startup timeout race)",
|
||||
"adr": "ADR-100",
|
||||
"runAt": "2026-05-05T13:00:00Z",
|
||||
"host": "darwin / macOS / npm 10+",
|
||||
"method": "rm -rf ~/.npm/_npx; time npx -y <pkg>@alpha --version (4 runs total — cold + warm for each package)",
|
||||
"results": {
|
||||
"@claude-flow/cli-core@3.7.0-alpha.0": {
|
||||
"packed_size_bytes": 22300,
|
||||
"unpacked_size_bytes": 89900,
|
||||
"files": 22,
|
||||
"cold_cache_ms": 671,
|
||||
"warm_cache_ms": 361,
|
||||
"cold_warm_ratio": 1.86
|
||||
},
|
||||
"@claude-flow/cli@3.6.30": {
|
||||
"packed_size_bytes": 1800000,
|
||||
"unpacked_size_bytes": 9600000,
|
||||
"files": 999,
|
||||
"cold_cache_ms": 25496,
|
||||
"warm_cache_ms": 623,
|
||||
"cold_warm_ratio": 40.9
|
||||
}
|
||||
},
|
||||
"improvement": {
|
||||
"cold_cache_speedup": "38× (25.5s → 0.67s)",
|
||||
"fits_under_30s_mcp_startup_timeout": "yes — 38× headroom vs cli's 25.5s which races the timeout",
|
||||
"size_reduction_packed": "80× (1.8 MB → 22.3 KB)",
|
||||
"size_reduction_files": "45× (999 → 22 files)"
|
||||
},
|
||||
"caveats": [
|
||||
"cli-core@alpha.0 is foundation-only (types, output, MCP-tool types, validate-input). Does not yet expose memory/hooks command surface.",
|
||||
"ADR-100 §Discovery documents that memory/hooks tools have deep ML deps requiring backend abstraction (alpha.1) and tool-definition/handler split (alpha.2) before parity is reached.",
|
||||
"@claude-flow/cli's 25.5s cold-cache is under the 30s timeout on this connection but barely — slow networks would race and silently fail (the original #1748 #3 symptom)."
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user