mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-11-09 22:13:39 +09:00
rUv
cd8a9c0988
* feat(memory): distillation service — mine memory_entries into intelligence substrate (ADR-174 M0+M1) Root cause the goal-planner surfaced: the daemon's consolidate worker (worker-daemon.ts:1443) is a STUB writing hardcoded zeros — 6000+ commits recorded to memory_entries but never distilled. reasoning_patterns / episodes / pattern_embeddings / causal_edges were all empty (hence 🧠 0%). M0 (safety) + M1 (service): runDistillation() mines memory_entries into episodes → reasoning_patterns (+pattern_embeddings) → causal_edges. - $0: reuses structured-distill.ts extractor + the embeddings already on each row; no LLM, no re-embedding. - Incremental: distill_state rowid cursor per namespace; never rescans. - Non-destructive: only inserts into the empty target tables; never touches memory_entries. Per-batch transactions. - Safe on the just-recovered DB: quick_check gate skips corrupt DBs; better-sqlite3 optional (silent no-op). - Provenance (ADR-171): feedback outcomes = oracle:test-exec (promotable); everything else = proxy:structural (written, NEVER promoted). Measured on a copy of the real 7,900-entry DB: 7,899 → 4,260 patterns (182 promoted, 4,078 proxy), 4,259 embeddings, 4,258 edges; source unchanged; dry-run wrote nothing; 2nd run processed 0. 6 tests green. ADR-174 records the decision, root cause, param surface, self-opt plan, alternatives, rollback. Follow-ups on branch: M2 CLI, M3 daemon (replace the stub), M4 self-optimization, M5 platform default. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(memory): distill CLI, daemon auto-distill, self-optimization (ADR-174 M2+M3+M4) M2 — CLI: 'memory distill run|status|config' consuming the frozen M1 runDistillation. Full param surface: --dry-run, --namespace, --batch-size, --dedup-distance, --max-entries, --since, --budget-usd, --judge (fable gated on budget per ADR-172), --aggressive/--conservative presets, --config, --db. status reports substrate counts + cursor + provenance breakdown; degrades gracefully. 12 tests. M3 — daemon: replaced the runConsolidateWorker STUB (the root cause — it wrote hardcoded zeros) with a real runDistillation call every 30-min tick, bounded to 1000 entries/tick (cursor drains a backlog across ticks, stays well under the 16-min worker timeout). Real DistillReport written to the metrics file. Opt-out via --no-distill / RUFLO_DAEMON_NO_DISTILL and the existing 'daemon enable -w consolidate --disable'. Never throws (corrupt/ skipped surfaced, not fatal). 7 tests. The loop is now self-sustaining. M4 — self-optimization ('ruflo tuning ruflo'): tuneDistillation grid- searches {batchSize, dedupDistance, promoteThreshold} on ISOLATED copies, scored by MRR@10/recall@10 on a rowid-based held-out split (train/held-out disjointness asserted; held-out scored once). Source checksum unchanged. Honest measured result on the real 7.9k-entry corpus: winner batchSize=200 dedupDistance=0.2 promoteThreshold=execution+corroborated, held-out MRR@10 0.753 vs 0.749 baseline — roughly on par (the substrate/training value is the payoff, not raw retrieval uplift; measured-not-marketing). 11 tests. Combined M1+M2+M3+M4: 30 tests green. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(browser): page-agent natural-language intent layer — browser_act (ADR-175) Adds an in-page NL intent tool on top of the selector-based browser tools: browser_act injects page-agent (Alibaba, MIT) into the agent-browser-driven page and runs 'execute(task)' — one natural-language call instead of manual selector orchestration. Verified against the real page-agent@1.11.0 (ESM-only, split @page-agent/* packages) — three load-bearing deviations handled: - import('page-agent') throws 'window is not defined' in Node even when installed → availability probed via require.resolve (path only, no eval), like browser-session-tools.ts. - the injectable IIFE auto-constructs a second PageAgent pointed at Alibaba's PUBLIC test endpoint (data leak on every call) → stripped before injection (stripDemoAutoInit). - execute() returns { success, data, history } → mapped to { result, steps }. Safety + routing: - Optional dependency (~1.11.0) + v3 pnpm-lock regenerated same change; browser_act returns { degraded:true } (never throws) when absent. - LLM routed through this CLI's Tier-3 ladder (OPENROUTER/OLLAMA, OpenAI- compatible); bare ANTHROPIC_API_KEY degrades (wrong API shape). - KEY SAFETY: page-agent runs in page context, so the real key never enters the injected string — startLocalLLMProxy holds it server-side on a loopback http server and injects Authorization itself; page sees 127.0.0.1 + a placeholder. Tests assert no key substring in the injection. - Trajectory → memory namespace 'browser' (feeds ADR-174 distillation). - AIDefence gates returned content (optional; passes through if absent). ADR-175 records the decision, the outer-harness/in-page split, constraints, the 6 follow-on use cases, alternatives, and rollback. 16 tool tests + 145 mcp-client regression tests green; full build clean. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(browser): fail-closed demo-endpoint firewall for page-agent injection The demo auto-connect (page content → Alibaba's public sandbox at page-ag-testing-*.fcapp.run) was only neutralized by a best-effort text strip that fell OPEN — stripDemoAutoInit returned the source unchanged if the DEMO_MODEL marker moved, silently re-enabling the leak on an upstream bundle change. Verified via the real tarball: the demo endpoint lives ONLY in dist/iife/page-agent.demo.js (6 refs); the library core ESM entry is clean (0 refs). So no fork is needed — the leak is a property of the demo bundle, not the library. Fix: findDemoLeak() firewall + DemoLeakError. buildPageAgentInjection now scans the stripped bundle for known demo-endpoint signatures (page-ag-testing, .fcapp.run, DEMO_MODEL) and THROWS if any survive; the browser_act handler catches it and degrades rather than injecting. This is version-independent and fail-CLOSED — a moved/renamed marker can no longer leak, because the guarantee is 'endpoint absent', not 'marker found'. +4 tests (signature detection, clean-bundle pass, moved-marker refusal, clean-bundle injection). 20/20 green. ADR-175 updated with the fork-vs-fix rationale. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(browser): add ADR-112 discoverability guidance to browser_act description CI failed 4 checks (tool-description audit + 3 smoke contracts that run it): browser_act was the 1 tool of 361 without a 'Use when …' guidance signal. The audit regex captures only the FIRST quoted segment of a multi-line description, so the guidance phrase must lead. Reworded to open with 'Use when …' + 'Prefer … over …' + 'pair with …' + 'Falls back …'. Audit now 361/361, exit 0. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(memory): M5 — promote M4 winning distill config as daemon default (ADR-174) The daemon's consolidate worker now runs the M4-tuned config (dedupDistance=0.2, chosen by the self-optimization grid-search on the real corpus). Honest note in code + ADR: held-out MRR@10 0.753 vs 0.749 baseline — on-par, the substrate/training value is the payoff. Overridable per-run via 'memory distill'. Completes ADR-174 M0-M5. Co-Authored-By: RuFlo <ruv@ruv.net> * docs+fix(memory): address ADR-174 review — honest default, edge contract, invariants (ADR-174) Review fixes: - STATUS: Proposed → Accepted (M0-M5 implemented+tested, pending #2570 merge). - TUNED DEFAULT honestly justified: the M4 retrieval delta (0.753 vs 0.749) is a TIE, so dedupDistance=0.2 is sold as COMPRESSION, not retrieval lift — measured 2,723 vs 4,350 patterns (37% fewer) at 11.6ms/batch, 0 violations. Added an explicit promotion rule (retrieval-neutral within 0.002 AND >=15% compression AND <250ms/batch AND 0 proxy violations). - PARAM TABLE / DEFAULT consistency: dedup default is 0.2 everywhere (service, CLI, daemon). Presets now bracket it: --aggressive 0.3 / --conservative 0.1. - CAUSAL_EDGES OVERCLAIM: edges now carry an explicit contract (edge_type=cooccurrence, provenance_tier=proxy:structural, confidence=0.3, promoted=false) with the rule 'may rank retrieval, must NOT justify autonomous action'. Also fixed a latent bug: edges linked epId-1 (assumed consecutive rowids) → now the actual previous episode id. - EMBEDDING 1:1 invariant enforced: a cluster with no parseable vector is skipped, so every pattern has exactly one embedding (was off-by-one on a malformed source vector). - ADR adds operational invariants + audit SQL, business-value framing, and an acceptance test. +2 invariant tests (embedding coverage, edge contract). 32 distill tests green. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(hooks): capture edit/task FAILURES, not just successes (ADR-174) The DB analysis found 898 feedback records — 100% success, 0 failures — because the post-edit/post-task hooks recorded a hardcoded success:true and never inspected the tool outcome. With no negative examples the oracle tier can't teach good-vs-bad, which caps what distillation can learn. Fix: read Claude Code's PostToolUse tool_response and record the REAL outcome. - isToolFailure(hookInput): conservative detector (positive error signal only) — tool_response.is_error / error / success:false, non-zero Bash exit code, or an error string. Exported + unit-tested (5 cases). - Generated hook (helpers-generator.ts) + the shipped .claude/helpers/ hook-handler.cjs both inline it: post-edit → recordEdit(file, !toolFailed), post-task → feedback(!toolFailed) (feedback() already boosts on success / decays on failure — it was just never called with false). - intelligence.cjs recordEdit(file, success) persists success to pending-insights so consolidation/distillation gets the negative example. The distillation JUDGE (memory-distillation.ts judgeFeedback) already maps success:false → reward 0 / not-promoted, so the substrate is failure-ready; this closes the upstream gap that never produced failures. End-to-end: a failed-edit PostToolUse payload now logs '[LEARN] Edit FAILURE recorded' through the actual shipped hook; a success stays '[OK] Edit recorded'. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(init): version-stamped auto-refresh of critical helpers — hook fixes reach all users, no re-init (ADR-174) The propagation gap: Claude Code hooks run the PROJECT-LOCAL .claude/helpers/*.cjs copies, not the npm package — so npx ruflo@latest never updated them, and users didn't know to re-run init. Any hook fix (e.g. the ADR-174 failure-capture change) reached only people who happened to re-init. Fix: stamp the critical helpers (hook-handler.cjs, intelligence.cjs, auto-memory-hook.mjs) with the installed CLI version; on the next CLI command, if a project's stamp is older than the installed version, silently re-copy them. New lightweight src/init/helper-refresh.ts (fs/path/module only at load; lazily imports the heavy generators only on the rare source-unresolvable fallback). Wired AWAITED into index.ts startup so a fast command can't exit before the copy lands — fast path is a stamp read + string compare (sub-ms); the copy runs at most once per version bump. No-op outside a ruflo project (never scaffolds files in an unrelated dir). Skipped for init/update. executeUpgrade also stamps so an explicit upgrade is current. Verified end-to-end: a stale-stamped project with an OLD (no-failure-capture) hook silently upgrades to the toolFailed logic on a plain 'ruflo --help', and a current project is a no-op (a marker file survives). +4 tests. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(security): Ed25519 signed provenance gate for the helper auto-refresh (ADR-174) The auto-refresh copies auto-EXECUTING hook code from the installed package into projects. npm verifies the tarball at install, not the files on disk after — a sibling package's postinstall (or disk tamper) could overwrite the helpers and the refresh would faithfully propagate the malicious code. Fix (fail-closed): every helper is verified against a ruflo-signed manifest before install. - helper-signing.ts: baked RUFLO_HELPERS_PUBKEY + verifyHelpersManifest() (native Ed25519, same primitive as rvfa-signing.ts). Fail-closed — any bad signature / tampered hash / wrong algorithm → null → refuse. - scripts/sign-helpers.mjs: publish-time signer; private key from $RUFLO_HELPERS_SIGNING_KEY, NEVER committed (kept out of the repo tree). Not shipped (scripts/ is narrowed to postinstall.cjs in files). - helper-refresh.writeCriticalHelpers: verifies the signed manifest + every source helper's SHA-256 BEFORE copying anything; a mismatch is refused and the project's existing helpers are left intact. The generator fallback needs no manifest (it's the CLI's own compiled code = trust root). - index.ts warns loudly on a blocked refresh (potential tampering signal). Threat model: closes post-install/on-disk tampering of the helper files; does not defend a wholesale-compromised CLI (which could swap the baked key too — but then it owns the binary you run). A CI test ('hashes match shipped files') fails if a helper changes without re-signing, so a stale manifest can't ship. Verified end-to-end: a tampered shipped helper → refresh REFUSED (blocked, project helper untouched); restored → refresh succeeds. +6 signing tests. Co-Authored-By: RuFlo <ruv@ruv.net> * chore(security): source helper-signing key from GCP Secret Manager sign-helpers.mjs now resolves the private key from GCP Secret Manager (RUFLO_HELPERS_SIGNING_SECRET, via gcloud secrets versions access) as the preferred source for CI/publish, with the local-PEM fallback (RUFLO_HELPERS_SIGNING_KEY) for air-gapped signing. Key never committed. ADR-174 security note updated. Co-Authored-By: RuFlo <ruv@ruv.net>
Ruflo Plugins
32 Claude Code plugins for agent-powered development workflows. Load with --plugin-dir.
Quick Start
# Load specific plugins
claude --plugin-dir plugins/ruflo-core --plugin-dir plugins/ruflo-swarm
# Load all plugins
claude $(ls -d plugins/ruflo-*/ | sed 's|^|--plugin-dir |' | tr '\n' ' ')
Plugin Catalog
Core & Coordination
| Plugin | Description |
|---|---|
| ruflo-core | MCP server, status, doctor, coder/researcher/reviewer agents |
| ruflo-swarm | Swarm topologies (hierarchical, mesh), Monitor streaming |
| ruflo-autopilot | Autonomous /loop task completion with prediction |
| ruflo-loop-workers | 12 background workers via /loop or CronCreate |
| ruflo-workflows | Workflow templates, parallel execution, branching |
Memory & Intelligence
| Plugin | Description |
|---|---|
| ruflo-agentdb | AgentDB with HNSW vector search (150x-12,500x faster) |
| ruflo-rag-memory | SOTA RAG — hybrid search, Graph RAG, MMR diversity, memory bridge |
| ruflo-rvf | Portable RVF memory format, session persistence |
| ruflo-ruvector | ruvector — FlashAttention-3, Graph RAG, hybrid search, 103 MCP tools, Brain AGI |
| ruflo-knowledge-graph | Entity extraction, relation mapping, pathfinder traversal |
| ruflo-intelligence | SONA neural patterns, trajectory learning, model routing |
| ruflo-daa | Dynamic Agentic Architecture, cognitive patterns |
Architecture & Methodology
| Plugin | Description |
|---|---|
| ruflo-adr | ADR lifecycle — create, index, supersede, compliance checking |
| ruflo-ddd | DDD scaffolding — bounded contexts, aggregates, domain events |
| ruflo-sparc | SPARC methodology with 5 phases and quality gates |
Quality & Security
| Plugin | Description |
|---|---|
| ruflo-security-audit | CVE scanning, dependency vulnerability checks |
| ruflo-aidefence | Prompt injection detection, PII scanning |
| ruflo-testgen | Test gap detection, TDD London School workflow |
| ruflo-browser | Playwright browser automation and testing |
Development Tools
| Plugin | Description |
|---|---|
| ruflo-jujutsu | Diff analysis, risk scoring, reviewer recommendations |
| ruflo-docs | Doc generation, drift detection, API docs |
| ruflo-ruvllm | Local LLM inference, MicroLoRA, chat formatting |
| ruflo-agent | WASM agent sandboxing and gallery |
| ruflo-plugin-creator | Scaffold and validate new plugins |
| ruflo-migrations | Database schema migration management |
| ruflo-observability | Structured logging, tracing, metrics correlation |
| ruflo-cost-tracker | Token usage tracking, budget alerts, cost optimization |
Domain-Specific
| Plugin | Description |
|---|---|
| ruflo-goals | GOAP planning, deep research, horizon tracking |
| ruflo-federation | Zero-trust cross-installation agent federation |
| ruflo-iot-cognitum | Cognitum Seed IoT — trust scoring, anomaly detection, fleet management |
| ruflo-neural-trader | neural-trader — 4 agents, LSTM/Transformer, Rust/NAPI backtesting, 112+ MCP tools |
| ruflo-market-data | Market data ingestion, OHLCV vectorization, pattern matching |
Recommended Stacks
| Use Case | Plugins |
|---|---|
| Feature development | ruflo-core + ruflo-swarm + ruflo-testgen + ruflo-ddd |
| Security audit | ruflo-core + ruflo-security-audit + ruflo-aidefence |
| Architecture work | ruflo-core + ruflo-adr + ruflo-ddd + ruflo-sparc |
| Deep research | ruflo-core + ruflo-goals + ruflo-rag-memory + ruflo-intelligence |
| Vector search | ruflo-core + ruflo-ruvector + ruflo-rag-memory + ruflo-knowledge-graph |
| IoT development | ruflo-core + ruflo-iot-cognitum + ruflo-agentdb |
| Trading systems | ruflo-core + ruflo-neural-trader + ruflo-market-data + ruflo-ruvector |
| Full stack | All 32 plugins |
npm Package Integration
Several plugins wrap standalone npm packages for deeper functionality:
| Plugin | npm Package | What It Adds |
|---|---|---|
ruflo-neural-trader |
neural-trader |
112+ MCP tools, Rust/NAPI engine, LSTM/Transformer models |
ruflo-ruvector |
ruvector |
103 MCP tools, FlashAttention-3, Graph RAG, Brain AGI |
# Install backing packages
npm install neural-trader ruvector
# Add as MCP servers (optional, for direct tool access)
claude mcp add neural-trader -- npx neural-trader mcp start
claude mcp add ruvector -- npx ruvector mcp start
Plugin Structure
Each plugin follows the Claude Code plugin specification:
ruflo-<name>/
.claude-plugin/plugin.json # Plugin manifest
agents/<name>.md # Agent definitions (frontmatter: name, description, model)
commands/<name>.md # CLI command mappings
skills/<name>/SKILL.md # Interactive skills (frontmatter: name, description, argument-hint, allowed-tools)
README.md # Plugin documentation
Creating a Plugin
claude --plugin-dir plugins/ruflo-plugin-creator
# Then: /create-plugin my-new-plugin
Or manually: copy any existing plugin directory and modify.
Validation
claude plugin validate plugins/ruflo-<name>
Verification & Discoverability
Every MCP tool description across the 32 plugins must answer "use this over native (Bash/Read/Grep/Glob/Task/TodoWrite) when?" per ADR-112. The rule is enforced by CI:
# Run the audit (scans all MCPTool definitions across all plugins)
node scripts/audit-tool-descriptions.mjs
# Gates: every description must include "Use when …" guidance,
# be ≥ 80 chars, and be unique. Baseline at verification/mcp-tool-baseline.json
# is monotone-decreasing — CI fails on any regression.
Combined with verification/ (Ed25519-signed witness manifest, 103+ documented fixes attested), the plugin surface is regression-protected at three layers: install smoke (npm i), behavioral smoke (paired-tool round-trips), and presence attestation (every load-bearing line of every documented fix). See verification/README.md for the full stack.
License
MIT