mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-11-10 06:23:28 +09:00
12ec7e72dc09d580554dca9a0c59ddbb07c1923c
353
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rUv
|
cd8a9c0988 |
feat: memory distillation self-learning loop (ADR-174) + page-agent browser intent (ADR-175) (#2570)
* feat(memory): distillation service — mine memory_entries into intelligence substrate (ADR-174 M0+M1) Root cause the goal-planner surfaced: the daemon's consolidate worker (worker-daemon.ts:1443) is a STUB writing hardcoded zeros — 6000+ commits recorded to memory_entries but never distilled. reasoning_patterns / episodes / pattern_embeddings / causal_edges were all empty (hence 🧠 0%). M0 (safety) + M1 (service): runDistillation() mines memory_entries into episodes → reasoning_patterns (+pattern_embeddings) → causal_edges. - $0: reuses structured-distill.ts extractor + the embeddings already on each row; no LLM, no re-embedding. - Incremental: distill_state rowid cursor per namespace; never rescans. - Non-destructive: only inserts into the empty target tables; never touches memory_entries. Per-batch transactions. - Safe on the just-recovered DB: quick_check gate skips corrupt DBs; better-sqlite3 optional (silent no-op). - Provenance (ADR-171): feedback outcomes = oracle:test-exec (promotable); everything else = proxy:structural (written, NEVER promoted). Measured on a copy of the real 7,900-entry DB: 7,899 → 4,260 patterns (182 promoted, 4,078 proxy), 4,259 embeddings, 4,258 edges; source unchanged; dry-run wrote nothing; 2nd run processed 0. 6 tests green. ADR-174 records the decision, root cause, param surface, self-opt plan, alternatives, rollback. Follow-ups on branch: M2 CLI, M3 daemon (replace the stub), M4 self-optimization, M5 platform default. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(memory): distill CLI, daemon auto-distill, self-optimization (ADR-174 M2+M3+M4) M2 — CLI: 'memory distill run|status|config' consuming the frozen M1 runDistillation. Full param surface: --dry-run, --namespace, --batch-size, --dedup-distance, --max-entries, --since, --budget-usd, --judge (fable gated on budget per ADR-172), --aggressive/--conservative presets, --config, --db. status reports substrate counts + cursor + provenance breakdown; degrades gracefully. 12 tests. M3 — daemon: replaced the runConsolidateWorker STUB (the root cause — it wrote hardcoded zeros) with a real runDistillation call every 30-min tick, bounded to 1000 entries/tick (cursor drains a backlog across ticks, stays well under the 16-min worker timeout). Real DistillReport written to the metrics file. Opt-out via --no-distill / RUFLO_DAEMON_NO_DISTILL and the existing 'daemon enable -w consolidate --disable'. Never throws (corrupt/ skipped surfaced, not fatal). 7 tests. The loop is now self-sustaining. M4 — self-optimization ('ruflo tuning ruflo'): tuneDistillation grid- searches {batchSize, dedupDistance, promoteThreshold} on ISOLATED copies, scored by MRR@10/recall@10 on a rowid-based held-out split (train/held-out disjointness asserted; held-out scored once). Source checksum unchanged. Honest measured result on the real 7.9k-entry corpus: winner batchSize=200 dedupDistance=0.2 promoteThreshold=execution+corroborated, held-out MRR@10 0.753 vs 0.749 baseline — roughly on par (the substrate/training value is the payoff, not raw retrieval uplift; measured-not-marketing). 11 tests. Combined M1+M2+M3+M4: 30 tests green. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(browser): page-agent natural-language intent layer — browser_act (ADR-175) Adds an in-page NL intent tool on top of the selector-based browser tools: browser_act injects page-agent (Alibaba, MIT) into the agent-browser-driven page and runs 'execute(task)' — one natural-language call instead of manual selector orchestration. Verified against the real page-agent@1.11.0 (ESM-only, split @page-agent/* packages) — three load-bearing deviations handled: - import('page-agent') throws 'window is not defined' in Node even when installed → availability probed via require.resolve (path only, no eval), like browser-session-tools.ts. - the injectable IIFE auto-constructs a second PageAgent pointed at Alibaba's PUBLIC test endpoint (data leak on every call) → stripped before injection (stripDemoAutoInit). - execute() returns { success, data, history } → mapped to { result, steps }. Safety + routing: - Optional dependency (~1.11.0) + v3 pnpm-lock regenerated same change; browser_act returns { degraded:true } (never throws) when absent. - LLM routed through this CLI's Tier-3 ladder (OPENROUTER/OLLAMA, OpenAI- compatible); bare ANTHROPIC_API_KEY degrades (wrong API shape). - KEY SAFETY: page-agent runs in page context, so the real key never enters the injected string — startLocalLLMProxy holds it server-side on a loopback http server and injects Authorization itself; page sees 127.0.0.1 + a placeholder. Tests assert no key substring in the injection. - Trajectory → memory namespace 'browser' (feeds ADR-174 distillation). - AIDefence gates returned content (optional; passes through if absent). ADR-175 records the decision, the outer-harness/in-page split, constraints, the 6 follow-on use cases, alternatives, and rollback. 16 tool tests + 145 mcp-client regression tests green; full build clean. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(browser): fail-closed demo-endpoint firewall for page-agent injection The demo auto-connect (page content → Alibaba's public sandbox at page-ag-testing-*.fcapp.run) was only neutralized by a best-effort text strip that fell OPEN — stripDemoAutoInit returned the source unchanged if the DEMO_MODEL marker moved, silently re-enabling the leak on an upstream bundle change. Verified via the real tarball: the demo endpoint lives ONLY in dist/iife/page-agent.demo.js (6 refs); the library core ESM entry is clean (0 refs). So no fork is needed — the leak is a property of the demo bundle, not the library. Fix: findDemoLeak() firewall + DemoLeakError. buildPageAgentInjection now scans the stripped bundle for known demo-endpoint signatures (page-ag-testing, .fcapp.run, DEMO_MODEL) and THROWS if any survive; the browser_act handler catches it and degrades rather than injecting. This is version-independent and fail-CLOSED — a moved/renamed marker can no longer leak, because the guarantee is 'endpoint absent', not 'marker found'. +4 tests (signature detection, clean-bundle pass, moved-marker refusal, clean-bundle injection). 20/20 green. ADR-175 updated with the fork-vs-fix rationale. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(browser): add ADR-112 discoverability guidance to browser_act description CI failed 4 checks (tool-description audit + 3 smoke contracts that run it): browser_act was the 1 tool of 361 without a 'Use when …' guidance signal. The audit regex captures only the FIRST quoted segment of a multi-line description, so the guidance phrase must lead. Reworded to open with 'Use when …' + 'Prefer … over …' + 'pair with …' + 'Falls back …'. Audit now 361/361, exit 0. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(memory): M5 — promote M4 winning distill config as daemon default (ADR-174) The daemon's consolidate worker now runs the M4-tuned config (dedupDistance=0.2, chosen by the self-optimization grid-search on the real corpus). Honest note in code + ADR: held-out MRR@10 0.753 vs 0.749 baseline — on-par, the substrate/training value is the payoff. Overridable per-run via 'memory distill'. Completes ADR-174 M0-M5. Co-Authored-By: RuFlo <ruv@ruv.net> * docs+fix(memory): address ADR-174 review — honest default, edge contract, invariants (ADR-174) Review fixes: - STATUS: Proposed → Accepted (M0-M5 implemented+tested, pending #2570 merge). - TUNED DEFAULT honestly justified: the M4 retrieval delta (0.753 vs 0.749) is a TIE, so dedupDistance=0.2 is sold as COMPRESSION, not retrieval lift — measured 2,723 vs 4,350 patterns (37% fewer) at 11.6ms/batch, 0 violations. Added an explicit promotion rule (retrieval-neutral within 0.002 AND >=15% compression AND <250ms/batch AND 0 proxy violations). - PARAM TABLE / DEFAULT consistency: dedup default is 0.2 everywhere (service, CLI, daemon). Presets now bracket it: --aggressive 0.3 / --conservative 0.1. - CAUSAL_EDGES OVERCLAIM: edges now carry an explicit contract (edge_type=cooccurrence, provenance_tier=proxy:structural, confidence=0.3, promoted=false) with the rule 'may rank retrieval, must NOT justify autonomous action'. Also fixed a latent bug: edges linked epId-1 (assumed consecutive rowids) → now the actual previous episode id. - EMBEDDING 1:1 invariant enforced: a cluster with no parseable vector is skipped, so every pattern has exactly one embedding (was off-by-one on a malformed source vector). - ADR adds operational invariants + audit SQL, business-value framing, and an acceptance test. +2 invariant tests (embedding coverage, edge contract). 32 distill tests green. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(hooks): capture edit/task FAILURES, not just successes (ADR-174) The DB analysis found 898 feedback records — 100% success, 0 failures — because the post-edit/post-task hooks recorded a hardcoded success:true and never inspected the tool outcome. With no negative examples the oracle tier can't teach good-vs-bad, which caps what distillation can learn. Fix: read Claude Code's PostToolUse tool_response and record the REAL outcome. - isToolFailure(hookInput): conservative detector (positive error signal only) — tool_response.is_error / error / success:false, non-zero Bash exit code, or an error string. Exported + unit-tested (5 cases). - Generated hook (helpers-generator.ts) + the shipped .claude/helpers/ hook-handler.cjs both inline it: post-edit → recordEdit(file, !toolFailed), post-task → feedback(!toolFailed) (feedback() already boosts on success / decays on failure — it was just never called with false). - intelligence.cjs recordEdit(file, success) persists success to pending-insights so consolidation/distillation gets the negative example. The distillation JUDGE (memory-distillation.ts judgeFeedback) already maps success:false → reward 0 / not-promoted, so the substrate is failure-ready; this closes the upstream gap that never produced failures. End-to-end: a failed-edit PostToolUse payload now logs '[LEARN] Edit FAILURE recorded' through the actual shipped hook; a success stays '[OK] Edit recorded'. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(init): version-stamped auto-refresh of critical helpers — hook fixes reach all users, no re-init (ADR-174) The propagation gap: Claude Code hooks run the PROJECT-LOCAL .claude/helpers/*.cjs copies, not the npm package — so npx ruflo@latest never updated them, and users didn't know to re-run init. Any hook fix (e.g. the ADR-174 failure-capture change) reached only people who happened to re-init. Fix: stamp the critical helpers (hook-handler.cjs, intelligence.cjs, auto-memory-hook.mjs) with the installed CLI version; on the next CLI command, if a project's stamp is older than the installed version, silently re-copy them. New lightweight src/init/helper-refresh.ts (fs/path/module only at load; lazily imports the heavy generators only on the rare source-unresolvable fallback). Wired AWAITED into index.ts startup so a fast command can't exit before the copy lands — fast path is a stamp read + string compare (sub-ms); the copy runs at most once per version bump. No-op outside a ruflo project (never scaffolds files in an unrelated dir). Skipped for init/update. executeUpgrade also stamps so an explicit upgrade is current. Verified end-to-end: a stale-stamped project with an OLD (no-failure-capture) hook silently upgrades to the toolFailed logic on a plain 'ruflo --help', and a current project is a no-op (a marker file survives). +4 tests. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(security): Ed25519 signed provenance gate for the helper auto-refresh (ADR-174) The auto-refresh copies auto-EXECUTING hook code from the installed package into projects. npm verifies the tarball at install, not the files on disk after — a sibling package's postinstall (or disk tamper) could overwrite the helpers and the refresh would faithfully propagate the malicious code. Fix (fail-closed): every helper is verified against a ruflo-signed manifest before install. - helper-signing.ts: baked RUFLO_HELPERS_PUBKEY + verifyHelpersManifest() (native Ed25519, same primitive as rvfa-signing.ts). Fail-closed — any bad signature / tampered hash / wrong algorithm → null → refuse. - scripts/sign-helpers.mjs: publish-time signer; private key from $RUFLO_HELPERS_SIGNING_KEY, NEVER committed (kept out of the repo tree). Not shipped (scripts/ is narrowed to postinstall.cjs in files). - helper-refresh.writeCriticalHelpers: verifies the signed manifest + every source helper's SHA-256 BEFORE copying anything; a mismatch is refused and the project's existing helpers are left intact. The generator fallback needs no manifest (it's the CLI's own compiled code = trust root). - index.ts warns loudly on a blocked refresh (potential tampering signal). Threat model: closes post-install/on-disk tampering of the helper files; does not defend a wholesale-compromised CLI (which could swap the baked key too — but then it owns the binary you run). A CI test ('hashes match shipped files') fails if a helper changes without re-signing, so a stale manifest can't ship. Verified end-to-end: a tampered shipped helper → refresh REFUSED (blocked, project helper untouched); restored → refresh succeeds. +6 signing tests. Co-Authored-By: RuFlo <ruv@ruv.net> * chore(security): source helper-signing key from GCP Secret Manager sign-helpers.mjs now resolves the private key from GCP Secret Manager (RUFLO_HELPERS_SIGNING_SECRET, via gcloud secrets versions access) as the preferred source for CI/publish, with the local-PEM fallback (RUFLO_HELPERS_SIGNING_KEY) for air-gapped signing. Key never committed. ADR-174 security note updated. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
rUv
|
d012042800 |
feat: review-driven upgrades — perf, security, SOTA capabilities (6-agent concurrent implementation) (#2547)
* chore(arch): untrack the prepublish plugin mirror — build artifact, not source
v3/@claude-flow/cli/plugins/ is regenerated by prepublishOnly from
plugins/ruflo-metaharness/ at publish time. Keeping it committed caused
silent drift between publishes (4 files diverged this week alone) and
every source edit required a manual mirror sync. npm pack still bundles
it — the files array + prepublish copy are unaffected by gitignore.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(gaia): submission integrity checklist — fail-closed pre-submit audit (Berkeley RDI hardening)
Adds a 4-part integrity section to the gaia-validate pre-submit gate,
motivated by UC Berkeley RDI's 'Agents' Last Exam' (April 2026), which
gamed all 8 major agent benchmarks to near-100% without solving tasks.
- scripts/gaia-integrity.mjs: (A) answer-key-shaped path reads outside
the sanctioned dataset dir — FAIL-CLOSED exit 2; (B) eval()/new
Function()/exec-of-non-literal strings in gaia-bench runner code
paths — FAIL-CLOSED exit 2; (C) judge-prompt-injection markers in
produced answers/trajectories — WARN; (D) provenance stamp (results
+ git SHA + dataset content hash) written for gaia-submit to embed.
- --allow-integrity-override suppresses the exit code but records
'overridden: true' in the stamp — never silent.
- --self-test plants a fake answer-key read, an eval-of-task runner,
and injection markers in temp fixtures and proves fail-closed/pass/warn.
- gaia-validate.md: new check 7 documenting the checklist + flags.
- gaia-submit.md: package now embeds integrity.json under the Ed25519
witness manifest (tamper-evident attestation); stale/failed stamps block
packaging.
- smoke-gaia.sh: 3 new steps (script wired, self-test green, repo scan
clean); plugin bumped to 0.5.0.
Co-Authored-By: RuFlo <ruv@ruv.net>
* perf(cli): lazy ONNX embedder init + local-first ruvector resolution
Fix 1 — lazy ONNX init (measured, biggest win):
neural-tools.ts had a module-level try/catch with TOP-LEVEL AWAIT that ran
`await import('ruvector')` + `initOnnxEmbedder()` + a probe embed at import
time. Because mcp-client statically imports neural-tools, this added the
full ONNX startup to every CLI command. The block is now a memoized
`ensureEmbeddings()` (module-level promise guard) invoked from
generateEmbedding() and from the three handlers that report the provider
without embedding (neural_compress / neural_status / neural_optimize), so
degraded/fallback semantics and provider reporting are byte-identical.
The probe embed moved inside ensureEmbeddings() — it still guards the
ADR-086 type-load-success-but-runtime-fails trap, but only on first use.
Measured (warm, darwin-arm64):
node bin/cli.js --help 0.50s -> 0.09s
node bin/cli.js status 0.64s -> 0.24s
import(mcp-client.js) 438ms -> 30ms
"Loading ONNX model" no longer appears on non-embedding commands;
appears exactly once on first embed (neural_predict / memory search).
Fix 2 — ruvector pin + local-first resolution:
browser-session-tools.ts pinned `npx -y ruvector@0.2.25` while 0.2.27 is
what the package installs, forcing cold npx downloads of a second version
(4 per browser session). The pin is now 0.2.27 and a memoized
resolveLocalRuvectorCli() (createRequire + package.json bin walk) spawns
`node <local-cli>` directly, falling back to the npx pin only when no
local install exists.
While verifying the 0.2.27 subcommand surface (required before bumping),
found the trajectory-* flags passed here were never valid on 0.2.25 OR
0.2.27 (`--session-id/--task/--args/--verdict` vs the real
`--context/--agent`, `--action/--result`, `--success/--quality`) — every
trajectory call exited with commander "unknown option", same bug class as
the #2015 `rvf create --kind` fix documented in this file. Flags are now
mapped correctly; browser_session_record and browser_session_end verified
end-to-end (rvf create + trajectory-begin/step/end + rvf compact all pass).
Fix 3 — lazy heavy MCP tool groups: profiled and NOT converted.
Per-module timed dynamic imports of all 36 tool modules show neural-tools
was 469.6ms and every other group is 0.3-5.5ms (hooks-tools 3.5ms,
agentdb-tools 2.5ms, embeddings 0.9ms, wasm-agent 0.6ms). After fix 1 the
whole registry builds in ~30ms; converting handler dispatch to lazy
import() would add risk for <6ms of residual win, so mcp-client.ts is
intentionally untouched.
Validation:
- tsc build clean
- plugins/ruflo-metaharness/scripts/test-mcp-tools.mjs: 171/171 passed
- memory search / neural predict still work; ONNX loads lazily then
Co-Authored-By: RuFlo <ruv@ruv.net>
* security(deps): lift OTel floor-became-ceiling pins — core >=2.8.0, otlp family >=0.220.0
The exact pins @opentelemetry/{core,resources,sdk-trace-base}@1.25.1 and
the otlp exporter/transformer family@0.52.1 (root + ruflo wrapper
overrides) were added in #1997/#2112 as a 'fewer-knobs-for-arborist'
defense against the npm arborist Invalid Version crash (empty-version
placeholder from optionalDependencies/peerDependencies overlap). They
were never API-compat pins — but they became a ceiling that forces the
tree onto @opentelemetry/core <2.8.0, which npm audit flags for
GHSA-8988-4f7v-96qf (unbounded memory allocation in W3C Baggage
parsing), and they blocked every upstream fix.
Convert to floors while keeping the lockstep families coherent:
- stable line: core / resources / sdk-trace-base -> >=2.8.0
- otlp line: sdk-node, exporter-prometheus, the three
exporter-trace-otlp-* packages, otlp-exporter-base,
otlp-grpc-exporter-base, otlp-transformer -> >=0.220.0
(sdk-node@0.220.0 declares exactly core@2.9.0 + otlp@0.220.0, so the
floors resolve to one mutually consistent set; verified in both the
root lockfile and a fresh ruflo/ resolution: core 2.9.0 everywhere,
otlp 0.220.0 everywhere.)
The arborist defense is preserved — every package in both families is
still forced to a single resolution via overrides; the override set is
now broader (whole otlp family at root too), which gives arborist fewer
ambiguous nodes than before, not more.
Also adds the exporter family overrides to the root manifest so root
and the published ruflo wrapper stay aligned (#2112 lesson: root
overrides do not propagate to the published wrapper).
Validation:
- root: npm audit --omit=dev 30 moderate -> 0 vulnerabilities
- ruflo: npm install --package-lock-only succeeds, 0 vulnerabilities
- runtime probe: new NodeSDK({}) via @opentelemetry/sdk-node@0.220.0
with core@2.9.0 loads and exposes start(); OTLPTraceExporter loads
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): npm audit fix on v3 workspace lock — grpc-js, hono/node-server, brace-expansion
Plain `npm audit fix --omit=dev --package-lock-only` (no --force) on the
v3 npm workspace lockfile. Clears the three targeted advisories:
- @grpc/grpc-js 1.14.3 -> 1.14.4 GHSA-5375-pq7m-f5r2 (+GHSA-99f4-grh7-6pcq)
- @hono/node-server 1.19.9 -> 1.19.14 GHSA-wc8c-qw6v-h7f6 (+GHSA-92pp-h63x-v22m)
- @isaacs/brace-expansion 5.0.0 removed/deduped GHSA-7h2j-956f-4vf2
Audit summary: 82 vulnerabilities (2 low, 56 moderate, 24 high)
-> 55 vulnerabilities (50 moderate, 5 high).
The remainder needs --force / dependency swaps and is out of scope for
this pass.
Note: the reify was run with the two pnpm-style `workspace:*` devDep
specs (plugin-agent-federation, plugin-iot-cognitum) temporarily
substituted with the registry pins the lockfile already records
(@claude-flow/shared@3.0.0-alpha.8, @claude-flow/security@3.0.0-alpha.10),
because npm's arborist does not support the workspace: protocol; the
manifests themselves are unchanged. The v3 workspace is otherwise
pnpm-driven (`pnpm -r build`).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): align agentdb floor in @claude-flow/memory — ^3.0.0-alpha.17
Root overrides force agentdb >=3.0.0-alpha.17, but @claude-flow/memory
declared ^3.0.0-alpha.16, so its nested/published tree could still
resolve alpha.16 (the #2112 lesson: overrides do not propagate to
nested or published trees). The v3 pnpm lock indeed had memory pinned
to agentdb 3.0.0-alpha.16 — bumping the package's own floor makes both
branches resolve the same version regardless of the consumer's
override set.
pnpm-lock.yaml is regenerated by `pnpm install`; besides the
agentdb 3.0.0-alpha.16 -> 3.0.0-alpha.17 move under memory, it also
reconciles previously committed manifest bumps (metaharness family)
that had drifted from the stale lock.
Verified: `npm ls agentdb` from root shows a single deduped
agentdb@3.0.0-alpha.17 across direct, @claude-flow/memory,
@claude-flow/neural, and agentic-flow branches.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): correct FEDERATION_BIND_HOST doc default — 127.0.0.1 (ADR-166 Phase 3d)
The bin.ts header JSDoc still advertised `default: 0.0.0.0`, but the
code (makeContext) has defaulted to 127.0.0.1 since the ADR-166
Phase 3d loopback-by-default hardening. Doc-only change.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* fix(memory): drop vitest-4 style vi.fn generic that breaks tsc build
The hoisted vitest 1.x types Mock as Mock<Args extends any[], Returns>,
so the single-generic vi.fn<SearchFn> form fails the package build
(pre-existing baseline failure). Annotate the variable instead — the
inferred mock signature is identical.
Co-Authored-By: RuFlo <ruv@ruv.net>
* perf(memory): batch embeddings in the migration import path
MemoryMigrator.processBatch batched entries (batchSize=100) but then
awaited this.embeddingGenerator(content) ONE ENTRY AT A TIME inside the
loop — N sequential ONNX inferences per batch on the
memory_import_claude hot path.
Now:
- New BatchEmbeddingGenerator type ((contents: string[]) =>
Promise<Float32Array[]>) accepted as an optional 4th constructor /
factory param — one true batch call per migration batch when the
embedder supports it (e.g. bge-embedder's single padded forward pass).
- Fallback: bounded concurrency over the single-text generator
(MigrationConfig.embeddingConcurrency, default 8 — never unbounded;
set 1 to restore sequential behavior).
- If the batch call fails, falls back to per-entry embedding so one
backend hiccup doesn't strand the batch.
Per-entry error semantics preserved: validation failures skip, store
failures fail, embedding failures warn and store the entry without a
vector — one bad entry never fails the batch.
Measured (200 synthetic entries, 5ms simulated single-embed latency):
sequential 1220ms -> concurrency-8 156ms (7.8x) -> true batch 115ms
(10.6x).
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(memory): temporal validity on hierarchical memory (Zep/Graphiti-style)
Facts in hierarchical memory can now carry validity windows and are
INVALIDATED on conflict instead of overwritten:
- New TieredMemoryStore module (@claude-flow/memory) promotes the inline
controller-registry stub to a first-class, tested store with optional
validFrom / validUntil (ISO) + supersededBy fields.
- store accepts supersedes=<entryId|key>: the old entry is stamped
validUntil=now + supersededBy=<newId> and archived — never deleted.
- recall filters temporally-invalid entries (expired, superseded, or
not-yet-valid) by default; includeExpired=true is the audit escape
hatch.
- Backward compatible: entries without the fields behave exactly as
before (always valid); legacy store(key,value,tier) / recall(query,
topK) call shapes unchanged; same-key stores still overwrite; the
bridge's real-vs-stub detection signal (getStats+promote) is
preserved.
- CLI bridge passes the new optional fields through
bridgeHierarchicalStore/-Recall. On the native agentdb
HierarchicalMemory backend, validFrom/validUntil ride in metadata and
are filtered at recall; supersedes is reported as unsupported there
(no public update API) rather than silently dropped.
- MCP inputSchemas for agentdb_hierarchical-store (validFrom,
validUntil, supersedes) and agentdb_hierarchical-recall
(includeExpired) expose the fields to agents, with ISO-8601
validation.
Tests: supersede flow (including same-key supersede preserving
history), expired + not-yet-valid filtering, includeExpired escape
hatch, legacy entries untouched, unparseable timestamps fail open.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(routing): confidence-gated tier escalation — hooks_model-verify ($0 post-generation verifier)
ADR-026/143 classify BEFORE generation only. This adds the 2026-SOTA
post-generation gate: attempt the cheap tier, verify the output with
CHEAP structural signals (no LLM call), escalate only on failure.
- src/ruvector/output-verifier.ts — pure verifier: empty/truncated
output, refusal patterns, degenerate repetition, unbalanced
delimiters, and a real syntax parse for code/JSON tasks (TypeScript
transpileModule diagnostics / JSON.parse, graceful degradation when
typescript is absent). Returns {confident, reasons[], suggestedTier,
suggestedModel, escalate}; ladder haiku→sonnet→opus, tier 2→3.
- hooks_model-verify MCP tool (hooks-tools.ts, surgical) — wraps the
verifier and feeds the verdict into the SAME learning stream as
hooks_model-outcome (ModelRouter.recordOutcome: success when
confident, escalated when not) so the Thompson priors learn which
task shapes the cheap tier fails on. A dedicated tool (not a verify
mode on hooks_model-outcome) because verify is a mid-loop decision
point returning a verdict the agent acts on, while outcome is a
terminal write — the sequence is route → generate → verify →
(escalate) → outcome.
- __tests__/output-verifier.test.ts — refusal/truncation/parse/
escalation-ladder coverage + MCP handler tests (18 tests).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* feat(federation): A2A Agent Cards — spec-1.0 discovery adapter over the bespoke federation schema
Implements the Agent2Agent (A2A, Linux Foundation) Agent Card layer as an
ADAPTER over the existing federation registration/discovery shapes. Cards
only — A2A Tasks/messaging are deliberately out of scope this iteration.
Spec: A2A Protocol 1.0.0 (a2a-protocol.org; schema source of truth
specification/a2a.proto in a2aproject/A2A). Required AgentCard fields per
spec 4.4.1: name, description, supportedInterfaces, version, capabilities,
defaultInputModes, defaultOutputModes, skills. Well-known discovery path
per 8.2 + IANA registration: /.well-known/agent-card.json.
- src/a2a/agent-card.ts: toAgentCard() maps FederationManifest -> spec-
compliant card (agent types -> AgentSkills; federation endpoint -> open-
form RUFLO-FEDERATION AgentInterface; bespoke identity rides in a spec
4.6 extension urn:ruflo:federation:manifest:v1 for lossless round-trip);
validateAgentCard() enforces the 1.0 required-field shape;
fromAgentCard() maps a remote card into a FederationNode — ALWAYS
TrustLevel.UNTRUSTED (cards are self-asserted metadata; trust is earned
via the normal handshake + TrustEvaluator path).
- src/a2a/well-known.ts: minimal node:http endpoint serving GET/HEAD on
the exact well-known path with spec 8.6 Cache-Control + ETag/304.
Binds 127.0.0.1 by default and REFUSES non-loopback binds without an
explicit allowNonLoopback opt-in (ADR-166 posture preserved — no
loosening of federation bind or auth).
- src/a2a/consume.ts: fetchAgentCard (injectable fetch, timeout, size cap,
structural validation) + consumeAgentCard registering the peer into
DiscoveryService so A2A peers appear in federation discovery.
- DiscoveryService: new 'a2a-card' discovery mechanism +
registerExternalPeer() (refresh-not-replace preserves trust state).
- plugin.ts: opt-in wiring via config.a2aCard/a2aCardPort/a2aCardHost;
bind failure degrades gracefully; server closed on shutdown.
- 40 new unit tests incl. golden-file card fixture
(__tests__/fixtures/a2a-agent-card.golden.json), foreign-card mapping,
round-trip, loopback guard, and a live serve->consume e2e over HTTP.
- plugin.test.ts: fixed pre-existing stale assertion (10 -> 11 CLI
commands; 'federation trust elevate' was added without updating the
count) — baseline suite failed before this change.
Build clean; 600/600 tests green.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(intelligence): execution-state-tree retrieval prototype (MAGE-style positional recall)
Research basis: MAGE (arXiv 2606.06090) — semantic-similarity retrieval
fragments decision trajectories on long-horizon tasks; retrieving by
POSITION in a hierarchical execution-state tree (root→current path)
preserves coherence.
- src/ruvector/trajectory-tree.ts — TrajectoryTree prototype:
trajectory-start opens a node under the session root (nesting under
the deepest open trajectory), steps append children, end closes and
pops the current position. recallPath({sessionId, depth}) returns
the exact root→current path + recent siblings — no embedding search.
Best-effort JSON snapshot at .claude-flow/intelligence/
trajectory-tree.json, ALONGSIDE existing trajectory storage (no
migration). Limitations documented in the module header.
- hooks-tools.ts (surgical): non-fatal tree mirrors in
hooks_intelligence_trajectory-start/step/end (+ optional sessionId
param on start), and an opt-in strategy:"state-tree" on
hooks_intelligence_pattern-search (+ sessionId/depth params).
Default strategy stays "semantic" — zero behavior change unless
opted in.
- __tests__/trajectory-tree.test.ts — synthetic 3-level trajectory
path assertions, nesting/pop, sibling window, persistence
round-trip, and default-strategy-untouched checks (12 tests).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* fix(metaharness): kill third-party @latest, extract shared _invoke.mjs, derive smoke counts
Three of the four converged security/perf/arch review fixes:
1. SECURITY (HIGH) + PERF — _harness.mjs no longer shells to
`npx -y metaharness@latest` (a compromised upstream publish would have
executed arbitrary code on user machines, and @latest forced a registry
check per call). Resolution is now: (a) locally installed metaharness
satisfying the PINNED range ~0.3.0 (walk-up node_modules), then (b) a
one-time `npm install --prefix ~/.ruflo/metaharness-cache-0.3.0`, then
plain `node <abs-path-to-cli>` spawns — zero network per call. Both bins
(metaharness + harness) are read from the resolved package.json bin map.
The {stdout, stderr, exitCode, json, degraded, reason, durationMs}
contract and ADR-150 graceful degradation are preserved; timeouts now
report 'metaharness-timeout' (matching _darwin/_redblue) instead of being
conflated with 'metaharness-not-available'.
Measured: `score.mjs --path .` completes in ~0.35s via the cache.
2. ARCH — new scripts/_invoke.mjs consolidates the copy-pasted plumbing:
DEGRADED_RX (superset incl. `npm ERR`), classifyDegraded (keeps the
-timeout vs -not-available distinction), injectJson, parseTrailingJson
(the LAST-{...}-block variant from _darwin — the old _harness first-block
parse was a live bug), ensureCachedInstall (generalizes gepa/_redblue,
cache-dir naming kept compatible so existing caches keep serving),
importOptionalLibrary, findLocalPackageDir, makeDegradedEmitter.
_harness/_darwin/_redblue/gepa are now thin adapters with IDENTICAL
exported signatures; their genuinely-different parts stay (redblue's
node-direct isMain workaround, darwin's async streaming for evolve,
harness's dual binaries). test-graceful-degradation.mjs gains
RUFLO_METAHARNESS_CACHE_BASE / RUFLO_METAHARNESS_SKIP_LOCAL seams so the
ADR-150 drill still exercises the degraded path on warm-cache machines.
3. ARCH — smoke.sh derives EXPECTED_TOOLS / EXPECTED_SUBS once from
metaharness-tools.ts / metaharness.ts and replaces the 7 independently
hardcoded literals (15 at :750/:829/:2046, 13 at :725/:851, 16 footnote,
15 success-pattern). Cross-surface value preserved: independent surfaces
(CLAUDE.md catalog, test-mcp-tools' own `tools.length === 15` literal,
footnote appends, runScript refs) are compared against the derived count,
never a surface against itself; a >=15/>=13 floor step catches
extraction-regex rot.
Validation: smoke.sh 122/122, test-graceful-degradation 16/16,
test-mcp-tools 171/171.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(metaharness): add --diagnose GEPA failure diagnosis to evolve
GEPA's key trick — natural-language failure diagnosis from execution traces
feeding the next mutation, not just scalar fitness — scoped modestly into
the evolve wrapper as an opt-in flag.
`evolve.mjs --diagnose` now appends a `diagnosis` section to the emitted
JSON after the run completes: losing/failed variants' transcripts are run
through darwin's gepa `analyzeTranscript` + `classifyFailure` ops (via the
shared importGepa resolver, moved to _darwin.mjs because gepa.mjs runs
main() on import) and aggregated into failure classes + counts + dominant
class per variant.
Upstream shape findings (verified against @metaharness/darwin@0.8.0 with a
tiny mock-sandbox evolve):
- `metaharness-darwin evolve --json` emits a TEXT leaderboard — stdout
carries no JSON and no transcripts (r.json has always been null here).
- Per-variant run records live at <repo>/.metaharness/runs/<id>.json as
sandbox exec traces ({taskId, exitCode, stdout, stderr}) — NOT gepa
{actionRaw, obs} transcripts.
Diagnosis therefore layers: gepa-shaped transcripts when a run record
embeds them (agent sandbox / future upstream) → champion-transcript
fallback → `diagnosis: {available: false, reason, traceSummary}` with a
mechanical per-variant tally. Never fails the run — internal errors
degrade to {available: false, reason: 'diagnosis-failed: ...'}.
Both paths exercised end-to-end: mock evolve yields the traceSummary
fallback; a seeded gepa-shaped run record yields
{available: true, scope: 'losing-variants', failureClasses:
{exploration-loop: 1, edit-mechanics: 1}, dominantClass: 'exploration-loop'}.
Documented in skills/harness-evolve/SKILL.md.
Co-Authored-By: RuFlo <ruv@ruv.net>
* Revert "feat(gaia): submission integrity checklist — fail-closed pre-submit audit (Berkeley RDI hardening)"
This reverts commit
|
||
|
rUv
|
e0d0af1cef |
feat(gaia): pre-submission exploit audit + signed attestation (ADR-167) (#2543)
* feat(gaia): pre-submission exploit audit + signed attestation (ADR-167) Motivated by UC Berkeley RDI's April-2026 finding that all 8 major agent benchmarks were gamed to ~98-100% without solving a single task (GAIA leaked ~98% of answers via public answer DBs + normalization collisions; o3/Claude-3.7 monkey-patch graders in 30%+ of runs per METR). Signing proves transport-integrity; this audit proves earning-integrity. The audit report is deterministic ($0, offline, pure-core) and is registered as an ADR-103 witness fix marker so its sha256 + "clean": true marker are signed INTO manifest.md.json. - ADR-167 (v3/docs/adr): threat model, check catalogue mapped to RDI/METR vectors, manifest-attestation design, honest harness-gap limitations. - gaia-audit.mjs: 7 pure checks (answer-leakage, no-work, oracle-leakage, grader-isolation, normalization-collision, voting-disclosure, split-integrity); CLI exit 0 clean / 1 CRITICAL / 2 usage. Schema-blocked checks return skip with a harness_gap note, never a false pass. - Tests: 27 passing; clean + dirty fixtures triggering every vector. - Wired --audit into /gaia validate; pre-sign gate into /gaia submit (refuses CRITICAL fail unless --allow-dirty) + gaia-submission skill. - Bump ruflo-workflows 0.4.0 → 0.5.0. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01V84YJJK7RBF9TuUo4eVSW7 * docs(adr): ADR-168 — GAIA harness evidence recording (implements ADR-167 §7 forward contract) Co-Authored-By: RuFlo <ruv@ruv.net> * docs(adr): ADR-169 — benchmark reporting integrity standard (strict-EM headlines, view-labeled scaling, disclosed gaps) Codifies the discipline the FRAMES ablation self-audit measured against the Berkeley RDI vectors. Completes the integrity family: ADR-167 audits the submission, ADR-168 records the evidence, ADR-169 governs how numbers are reported. Co-Authored-By: RuFlo <ruv@ruv.net> * feat(gaia): port static source-scan checks (AUD-8/9/10) into gaia-audit Ports the unique, non-overlapping checks from the reverted #2547 duplicate gaia-integrity.mjs into gaia-audit.mjs as a new static source-scan family, matching ADR-167's check-registry / status / harness_gap design: - AUD-8 answer-key-reads (CRITICAL, fail-closed): scans runner sources + produced artifacts for answer/gold/solution/ground-truth-shaped paths referenced outside the sanctioned dataset dir(s). - AUD-9 dynamic-eval (CRITICAL, fail-closed): static scan of the gaia-bench runner sources for eval()/new Function()/exec-family calls with a non-literal first arg. Preserves the #2547 false-positive refinements (multi-line aware, RegExp.prototype.exec excluded, fixed-string commands + comments ignored) — passes clean against ruflo's own multi-line `gcloud secrets … execSync`. - AUD-10 judge-injection (WARN): scans the agent's PRODUCED answer/outputs for judge-directed prompt-injection markers. Distinct from AUD-3 oracle-leakage (input-side gold-answer scan), so both run. Unlike AUD-1/3/4 these need no trajectory instrumentation — they enforce today and skip honestly (never false-pass) when the source population is absent (standalone/ejected run or --skip-source-scan). The #2547 provenance stamp and --allow-integrity-override are NOT ported: #2543's signed attestation and the `/gaia submit --allow-dirty` gate already own provenance + override, so the new checks flow through the same attestation.clean/strict_clean path. CLI adds --repo-root, --dataset-dir (repeatable), --skip-source-scan. Fixtures: planted evil runner (eval + interpolated exec), planted answer-key read, judge-injection results. Tests: 44 green (17 new). Docs: ADR-167 §3/§7 check table, gaia-validate.md, gaia-submission SKILL.md. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3 * fix(ci): advance ruflo-workflows smoke version assertion to 0.5.0 The AUD-8/9/10 port bumped plugin.json to 0.5.0 but the plugin's own smoke step 1 still asserted 0.4.0 — the single failing step behind both smoke-all and the meta-smoke leg of smoke-without-metaharness on #2543. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
rUvanddjimit
|
ae9bba523e |
feat(metaharness): learn + gepa integration — metaharness@0.3.0 / darwin@0.8.0 (15 MCP tools) (#2540)
* fix(mcp-bridge): streamable-HTTP session cleanup — DELETE + Mcp-Session-Id + 202 (#2425) Closes #2425. Cherry-picked from djimit's commit |
||
|
rUv
|
19e4277e12 |
feat: ADR-164 agentbbs business autopilot — Phases 1-4 (4 MCP tools, 7 pods, http_fetch, atomic budget tracker) (#2503)
* docs(adr): ADR-164 — AgentBBS federated business-management autopilot Draft ADR proposing agentbbs@0.1.0 integration as a federation-peer kind + business pod scaffolding. Covers the 4 new MCP tools, 7 domain pod templates (sales/marketing/finance/ops/support/hr/exec), upstream changes required in both ruflo and ruvnet/agentbbs, 5-phase rollout, and 7 honest risks. Builds on the optional-dep + graceful-degraded pattern shipped today in PR #2500 (agenticow) and ADR-150 (metaharness). Status: Draft. Implementation gated on upstream confirmation of agentbbs persistence semantics + latency baseline. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * docs(adr): amend ADR-164 + add ADR-164.1 (budget atomicity) ADR-164 amendments (Draft v2, 1193 → 1412 lines, +219 net): - Rewrite agentbbs maturity assessment — it's a 13-crate Rust workspace with 7 existing CI workflows + 30+ release tags + existing federation crate, NOT a 16h-old greenfield. The npm package is the launcher only. - §3.2.4 — two-phase auth: 15-min token is handshake-only; long-lived stream sessions persist past expiry until idle-close. - §3.5.4 — new "Founder-bootstrap trust elevation" admin escape hatch so #exec cross-pod synthesis works on Day 1 without 500-interaction wait. - §4.4 — Ops pod gets aidefence_*, terminal_execute, agent_execute, new http_fetch tool, cloud-provider MCP servers; concrete bench scenario. - §5.1.8 — new MCP tool `http_fetch` (URL allowlist, timeout, size cap). - §5.2 — survey-first upstream changes; many requirements may already exist in crates/agentbbs-federation and crates/agentbbs-mcp. - §7.3 + §9.5 — forward-reference to ADR-164.1 for race resolution. - §9.1 — risk #1 rewritten: real risks are npm/Rust version drift, FSL license implications, cargo first-run requirement, MCP server compatibility. ADR-164.1 (new, 724 lines): - Atomic reserve-and-commit token bucket against AgentDB-style SQLite store with BEGIN IMMEDIATE serialization. - Schema: bbs_budget_reservations + bbs_budget_rooms tables. - Three-op API: reserve / commit / release with discriminated returns. - 5 concurrency tests (vitest-implementable) closing the race window. - Migration plan: feature-flagged rollout behind CLAUDE_FLOW_BBS_ATOMIC_BUDGET, default flip after 1 week zero incidents. - p99 < 5ms target with ~1000 reserves/sec throughput ceiling honestly documented (partition-per-pod as the scale-out path, not replacing SQLite). Both Draft status. Phase 1 implementation gated on these. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * docs(adr): ADR-164.1 amendments — close the Expired Commit Leak Peer review (2026-06-29) identified three issues: 1. CRITICAL — Expired Commit Leak (§8.1, §5.3, §4.1) Original draft rejected commit() of an expired reservation, so the actual API spend went unrecorded. A high-latency loop could silently burn unbounded $ past the monthly cap. Fixed: commit() now ACCEPTS the spend, transitions state to new 'committed_post_expiry' value, records actual_usd, returns { ok: true, warned: 'COMMIT_AFTER_EXPIRY' }, and emits a reservation.committed_post_expiry audit envelope. 2. SQLite locking clarification (§3.2 _lock_bump field note) BEGIN IMMEDIATE itself acquires the RESERVED lock; _lock_bump is now documented as belt-and-suspenders + code-review visibility, not the primary serialization primitive. 3. Per-pod reservationExpiryMs (§3.2 expires_at field note) Default 60s is too short for local large-model pods. Pod templates may now set reservationExpiryMs in [5_000, 300_000] ms; ceiling defends against unbounded runaway. Plus new Test 6 in §9: load-bearing proof that COMMIT_AFTER_EXPIRY captures the real spend and the room's subsequent gate-check reflects it. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * feat(mcp): Phase 1 — agentbbs integration scaffold (4 MCP tools) Implements ADR-164 §3.2 + §5.1 with the same architectural pattern as PR #2500 (agenticow): optional-dep + graceful-degraded fallback. New MCP tools (4): - federation_bbs_register register a BBS room as a federation peer - federation_bbs_publish publish a typed envelope to a room - federation_bbs_watch poll recent envelopes (streaming = Phase 4) - federation_bbs_human_join single-use Ed25519 token + web/SSH handshake Phase 1 scope (scaffolding the surface, not the wire): - loadAgentbbs() checks importability; degrades to {degraded:true, reason:'agentbbs-not-found'} when missing — never throws - Validation runs BEFORE the optional-dep check so path-traversal / label / msgType rejection is deterministic regardless of install state - Backing store: local JSON-lines in <basePath>/room-<id>.jsonl + rooms.json registry. The Rust agentbbs-federation crate will replace this with signed wire envelopes in Phase 2. - Ed25519 keypair is per-process ephemeral; real federation keypair wiring is Phase 2 (documented inline at the call site). Gates (all PASS): - Build: clean (v3/@claude-flow/cli npm run build) - Vitest: 8 pass + 4 skip-conditional (agentbbs binary not built locally) - ADR-112 audit: 349/349 tools with Use-when guidance (was 345; +4 new) - Bash smoke: 8/8 (end-to-end register→publish→watch cycle via dist handlers) Plus: - .github/workflows/no-agentbbs-smoke.yml — CI guard mirroring no-metaharness-smoke.yml; asserts agentbbs stays in optionalDependencies and runtime drill exits 0 with --ignore-optional install - plugins/ruflo-bbs-federation/.claude-plugin/plugin.json — plugin metadata Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * docs(adr-164): replace §5.2 hypotheticals with concrete agentbbs survey findings Phase 1 survey of ruvnet/agentbbs @ ca3c6e0 resolved each compatibility check in §5.2 against the live upstream source. Six "VERIFY FIRST / if- exists / if-missing" branches are replaced with definite answers: - §5.2.1 FederationPayload — already covers ruflo's needs (4 variants, Ed25519-sealed); map publish→ReplicateMessage, register→AnnounceBoard. No upstream PR. - §5.2.2 MCP tool registration — static 4-tool surface (list_boards, read_board, post_message, search_memory). Re-scoped from "web UI rendering" to the actual MCP shape concern. Gap #1 filed for missing create_board tool. No upstream PR for Phase 1. - §5.2.3 Subscription/streaming — genuinely missing (subscribe=false hardcoded), but matches ruflo's Phase 1 polling design. Gap #2 filed for Phase 4 streaming. No upstream PR for Phase 1. - §5.2.4 Integration seam — surprise finding: agentbbs already drives `npx ruflo federation` via RufloAdapter (other direction). Ruflo's CLI now has a contract obligation to keep init/join/status alive. - §5.2.5 Durable retention — postgres backend exists in CI but `agentbbs mcp` defaults to MemoryStore; gap #3 filed for Phase 3. - §5.2.6 CI regression coverage — no existing end-to-end agentbbs↔ruflo test. Closed by ruvnet/AgentBBS#3 regression guard (PR opened upstream in this same Phase 1 cycle). The regression guard at ruvnet/AgentBBS#3 asserts the four MCP tool names, four FederationPayload variants, and three RufloAdapter subcommand names continue to exist on every agentbbs PR + nightly cron. Cross-ref: - Upstream PR: https://github.com/ruvnet/AgentBBS/pull/3 - Survey commit: ruvnet/AgentBBS@ca3c6e0 * feat(pods): Phase 2 — sales pod end-to-end (dry-run + validator + smoke) Implements ADR-164 Phase 2: scaffold the first concrete business pod (sales) end-to-end with a typed validator, a tick runner, and full smoke coverage. Dry-run mode is the Phase 2 default; --live mode exits 3 with a "Phase 3" message until Managed Agent + claude-p wiring lands. New surface (~1,597 LOC): v3/@claude-flow/cli/src/business-pods/pod-schema.ts — PodTemplate type + validatePodTemplate() + KNOWN_AGENT_TYPES mirror v3/@claude-flow/cli/src/mcp-tools/business-pod-tools.ts — 1 new MCP tool (business_pod_validate) plugins/ruflo-business-pods/ .claude-plugin/plugin.json templates/sales.json — Phase 2 sales pod template (matches ADR-164 §4.1 + §3.3 schema) scripts/pod-tick.mjs — dry-run tick runner + file-based budget ledger (TODO(adr-164.1): swap to atomic SQLite tracker) scripts/pod-tick.test.mjs — 15 node:test tests scripts/smoke.sh — 8-step smoke contract skills/pod-sales/SKILL.md .github/workflows/business-pods-smoke.yml — CI gate Gates (all PASS): - Build: clean - vitest business-pod-tools: 19/19 PASS - node:test pod-tick: 15/15 PASS - ADR-112 audit: 350/350 with guidance (was 349; +1 new tool) - business-pods smoke: 8/8 PASS - agentbbs regression smoke: 8/8 PASS (no regression from Phase 1) - agenticow regression smoke: 8/8 PASS (no regression from PR #2500) Honest Phase 2 deviations: - Schema validator duplicated TS / JS (single canonical TS in cli/src/, JS copy in pod-tick.mjs so script runs without prebuilt CLI). Phase 3 consolidates via shared agents/registry.ts. - Budget ledger is file-based JSON stub; ADR-164.1 atomic SQLite tracker swap deferred to Phase 3. - agentbbs subprocess wiring not invoked; envelopes go directly to the Phase 1 JSONL backing store (matches ADR-164 §5.2.2 phase plan). Phase 3 follow-ups inline-marked TODO(adr-164.1) / TODO(phase-3). Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * feat(pods): Phase 3 — remaining 6 pods + domain-affinity router policy Implements ADR-164 Phase 3: templates for marketing/finance/ops/support/hr/exec + the @metaharness/router domain-affinity policy hook. New templates (6) at plugins/ruflo-business-pods/templates/: marketing.json SOC2, $40/mo, cloud-tolerant finance.json GDPR, $80/mo, local-pinned, expiryMs=180_000 (3min) ops.json SOC2, $50/mo, local-pinned, synthetic-HTTP bench scenario; TODO(phase-3-http-fetch) for the unbuilt http_fetch tool support.json SOC2, $40/mo, cloud-tolerant hr.json GDPR, $30/mo, local-pinned (employee PII = finance-grade) exec.json GDPR, $60/mo, local-pinned, references §3.5.4 founder-bootstrap trust elevation for share-context on Day 1 New code: v3/@claude-flow/cli/src/business-pods/domain-affinity-policy.ts (86 LOC) - selectAgentBackend(pod): {backend, reason} - rules: preferLocalExecution=true → local-stdio else if budget.monthlyUsd >= 50 → cloud-managed else → remote-peer v3/@claude-flow/cli/src/mcp-tools/business-pod-tools.ts (+107 LOC) - new MCP tool: business_pod_route_backend - shared loadTemplate(input: object | string) helper Schema mirror in lockstep: added `base-template-generator` to KNOWN_AGENT_TYPES in BOTH TS (pod-schema.ts) and JS (pod-tick.mjs) copies. Gates (all PASS): - Build: clean - vitest business-pod-tools: 44/44 PASS (was 19; +18 Phase 3 cases) - node:test pod-tick: 21/21 PASS (was 15; +6 happy-path tests) - ADR-112 audit: 351/351 with guidance (was 350; +1 new tool) - business-pods smoke: 11/11 PASS (was 8; +3 Phase 3 steps) - agentbbs regression smoke: 8/8 PASS - agenticow regression smoke: 8/8 PASS Phase 4 follow-ups (inline TODOs): - http_fetch MCP tool implementation (referenced by ops.json) - @metaharness/router wire point in policy-engine.ts - ruflo federation trust elevate ... --to TRUSTED CLI escape hatch - Live-mode dispatch (Managed Agent + claude -p wiring) - Atomic SQLite budget tracker swap (ADR-164.1) Per ADR-164 §3.4, all rules codified verbatim; smoke step 11 + 4 vitest cases assert what the rules produce per template. Live mode still exits 3 with "Phase 4" message — Phase 3 is templates + validator + routing-policy only. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * feat: Phase 4 — http_fetch + atomic budget tracker + federation trust elevate Three Phase 4 deliverables landing per ADR-164: A. http_fetch MCP tool (ADR-164 §5.1.8) - 1 new tool (#352) for ops-pod's synthetic-endpoint bench - Security-default-rejects: private IPs (RFC-1918, localhost), auth headers (Authorization/Cookie/X-Auth-*) unless env vars opt-in - URL allowlist + timeout + response-size cap (256 KB default, 1 MB max) - 25/25 vitest pass (URL allowlist, timeout abort, truncation, auth-header rejection, happy path against node:http mock) B. Atomic SQLite budget tracker (ADR-164.1 §3-§5) - bbs_budget_rooms + bbs_budget_reservations tables - reserve/commit/release with BEGIN IMMEDIATE + _lock_bump - committed_post_expiry state machine (Expired Commit Leak fix from peer review 2026-06-29) implemented - expires_at clamped to [5_000, 300_000] ms per-pod override - Behind CLAUDE_FLOW_BBS_ATOMIC_BUDGET=1 — file-based stub remains the default until next flip; pod-tick.mjs migration deferred to Phase 5 - 15/15 vitest pass (Tests 1, 2, 3, 6 from ADR-164.1 §9, skipping Test 5 clock-skew which would require system-clock manipulation) - reservationId is randomUUID v4 (ADR-164.1 §3.2 notes v7 is purely aesthetic — v7 generator would add a transitive dep we don't need) C. ruflo federation trust elevate CLI (ADR-164 §3.5.4) - New subcommand: bypasses 500-interaction TRUST_TRANSITION_THRESHOLDS - Mandatory --reason + --audit flags (both required by spec) - Refuses unknown peer node-ids - Reuses existing trust_level_changed audit envelope with metadata.tag='bootstrap_elevation' + metadata.operatorBypass=true (cheapest path — keeps SEVERITY_BY_EVENT_TYPE / CATEGORY_BY_EVENT_TYPE stable rather than introducing a new event-type that needs propagation through downstream consumers) Gates (all PASS): - Build: clean (tsc) - vitest http-fetch + bbs-budget-tracker: 40/40 PASS (25 + 15) - vitest business-pod-tools regression: 44/44 PASS (Phase 3 still green) - ADR-112 audit: 352/352 with guidance (was 351; +1 http_fetch) - business-pods smoke: 11/11 PASS - agentbbs smoke: 8/8 PASS - agenticow smoke: 8/8 PASS Phase 5 follow-ups (explicit deferrals): - Live-mode dispatch (pod-tick --live still exits 3) - Sweeper timer integration site (daemon worker, not pod-tick) - Spend-reporter audit adapter (write to federation_spend namespace BEFORE BEGIN IMMEDIATE per ADR-164.1 §6.1) - pod-tick.mjs migration from file-based to atomic tracker - ACL gate on `federation trust elevate` (currently consistent with `federation evict` / `reactivate` — no founder-only crypto check yet) - http_fetch wiring into pod-tick live-mode for ops bench Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01WbBa4nccx5aGhXphoHxcni * chore(lock): regen v3/pnpm-lock.yaml with agentbbs entry (PR #2503 CI fix) Same fix as we did on the agenticow PR #2500 push — pnpm install at root regenerates pnpm-lock.yaml (the npm workspace), but the real workspace is v3/pnpm-workspace.yaml and its sibling v3/pnpm-lock.yaml. CI uses --frozen-lockfile so the v3 lockfile must be in sync with v3/@claude-flow/cli's package.json (which declares agentbbs: ~0.1.0 as optional). * fix(ci): re-export-tolerant agentbbs guard + skip business-pods smoke under --ignore-optional PR #2503 CI surfaced 2 ADR-164 issues: 1. no-agentbbs-smoke Rule 2 false-positive on mcp-tools/index.ts. The static-grep treated `export { agentbbsTools } from './agentbbs-tools.js';` as an unguarded agentbbs reference. The agentbbsTools symbol itself contains the loadAgentbbs() guard — the index.ts re-export is benign. Fixed: strip re-export + spread + import-of-agentbbsTools patterns + all comments before the unguarded-reference check. 2. ruflo-business-pods smoke 9/11 under no-metaharness-smoke's runtime drill (which does `npm install --ignore-optional`). Without optional deps, vitest may be absent AND the cli dist may not exist. Steps 4 (vitest) and 11 (call into dist) now SKIP cleanly when those are missing instead of failing — the main test-suite jobs still gate the full vitest + build runs. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(ci): business-pods-smoke uses pnpm (root npm chokes on workspace:*) The root package.json declares some workspace deps with the pnpm `workspace:*` protocol that npm doesn't support. Symptom: `npm install` step exited with EUNSUPPORTEDPROTOCOL. The v3/ tree is a real pnpm workspace (v3/pnpm-workspace.yaml). Switch the workflow to install + build from there using pnpm + frozen-lockfile, then fall through to vitest / node:test / smoke contract. Pattern matches the existing v3-ci.yml install steps. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(ci): pnpm --filter '@claude-flow/cli...' to include workspace deps The previous `pnpm --filter @claude-flow/cli build` didn't build workspace dependencies first, so the cli's tsc step failed with TS2307 errors looking for @claude-flow/memory, @claude-flow/neural, @claude-flow/cli-core types. The `...` suffix tells pnpm to include all of the cli's workspace dependencies in the build graph. Applied to both business-pods-smoke.yml and no-agentbbs-smoke.yml's runtime drill. Co-Authored-By: RuFlo <ruv@ruv.net> * fix(ci): pnpm -r build (whole workspace, dep-first) — pnpm filter can't discover cross-package dist/ imports Round 4 surfaced two pre-existing fragilities in the workspace's build wiring: 1. business-pods-smoke: @claude-flow/cli imports from @claude-flow/swarm/dist/ not via package.json deps but via direct file path. `--filter "@claude-flow/cli..."` doesn't traverse those. Fixed: `pnpm -r --no-bail build` builds the whole workspace dep-first, with a fallback to a direct cli build. 2. no-agentbbs-smoke runtime drill: `--no-optional` stripped agentdb, which is a transitive consumer of @claude-flow/neural and breaks neural's tsc even though neural has its own optional-degradation path internally. Fixed: install everything, then SURGICALLY rm node_modules/agentbbs only, so only the dep we're testing graceful-degradation for is missing. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
a63cdf0526 |
feat(metaharness): integrate @metaharness/redblue@~0.1.1 — adversarial red/blue LLM testing
ADR-150 sibling-package integration. `@metaharness/redblue` runs an
adversarial red-team → judge → blue-team patch → retest → report loop
against an LLM target you own. Covers OWASP LLM Top-10 / NIST AI RMF
families (prompt injection, tool misuse, data leakage, jailbreaks,
denial-of-wallet). Joins score/genome/mcp-scan/threat-model/oia-audit
as the run-time red/blue layer of the security plugin surface.
Why integrate here vs ruflo-aidefence or ruflo-security-audit:
- aidefence is detection (PII / prompt-injection scan); redblue is
*generation* — they complement, redblue doesn't replace it
- security-audit is dep CVEs / static scan; redblue is run-time LLM eval
- ruflo-metaharness already carries the @metaharness/* wrapper pattern
(`_darwin.mjs`, `_harness.mjs`), the locator, the MCP-tool registration
surface, and the smoke/CI gates — redblue slots in as one more sibling
Surface added:
- `npx ruflo metaharness redblue {init|run|patch|attack|report}` CLI
- `metaharness_redblue` MCP tool (subcommand + family + tests + patch +
mockJudge + alertOnFail + timeoutMs)
- Skill discoverability via CLAUDE.md (both CLI catalog + MCP tool catalog)
- 11 metaharness subcommands (was 10), 13 MCP tools (was 12)
Safety posture (inherited verbatim from upstream; wrapper does NOT relax):
- upstream `src/config/safety.ts` hard-enforces no real credentials, no
live external targets (loopback / *.test / *.internal only), no shell,
no arbitrary network, no eval at config-load time — a config that
enables any of these errors at load
- `--mock-judge` is the $0 marker-fixture path for CI / offline; the real
model judge requires `OPENROUTER_API_KEY` and is capped by
`max_cost_usd` / `max_runtime_minutes` / `max_tests`
- wrapper shells out with `shell: false` argv arrays; never string interp
- ADR-150 graceful-degradation rule #3 — on install/network failure the
wrapper returns `{degraded: true, reason: ...}` and exits 0 so ruflo
stays operational
Upstream bug workaround (track for revert):
`@metaharness/redblue@0.1.1`'s CLI bootstrap uses
`isMain = import.meta.url === \`file://\${process.argv[1]}\``. When npx
links `redblue` → `dist/cli/index.js`, argv[1] is the symlink path but
import.meta.url is the resolved real path, so isMain is false and the
binary silently exits 0 with no output, no file, no error. Darwin's CLI
calls `main()` unconditionally and is unaffected. Workaround: install the
package once into `~/.ruflo/redblue-cache` via `npm install --prefix` and
invoke `node <abs_path>/dist/cli/index.js` directly — argv[1] then equals
the real path and isMain becomes true. First install hits npm's cache so
the cold path is ~200ms; warm invocations are ~20ms.
Locator hardening (prevents recurrence of artifact-mirror staleness):
The publish artifact at `v3/@claude-flow/cli/plugins/ruflo-metaharness/`
is regenerated only at `prepublishOnly`. On a dev tree with a fresh
script in source but a stale mirror, the locator would pick the mirror
because `_harness.mjs` exists in both. Locators in `commands/metaharness.ts`
and `mcp-tools/metaharness-tools.ts` now accept an optional `requiredScript`
that narrows the match to dirs containing the dispatched script. The
artifact mirror is also synced with the two new redblue scripts so the
publish artifact stays usable until the next `prepublishOnly`.
Gates updated to lock new state:
- smoke.sh: 9 subcommands in dispatcher (was 8), 11 in description list
(was 10), 13 MCP tools (was 12), 13 mcp-script-count (was 12), 14
MCP_SUCCESS_SEMANTIC footnotes (was 13), 13 `success: r.success`
occurrences (was 12), tilde-pin check covers @metaharness/redblue
- test-mcp-tools.mjs: `tools.length === 13`, `metaharness_redblue`
enumerated, attack-preview Phase-3 input, 180s timeout (cold-cache npx
fetch class)
- CLAUDE.md: surfaced under both CLI catalog and MCP-tool catalog (gated
by smoke 17z56 + 17z57)
Verification:
- `bash plugins/ruflo-metaharness/scripts/smoke.sh` → 120 passed, 0 failed
- `node plugins/ruflo-metaharness/scripts/test-mcp-tools.mjs` → 153 passed, 0 failed
- `node v3/@claude-flow/cli/bin/cli.js metaharness redblue init --out <path>`
→ writes the safety-asserting sample config, exits 0
- attack preview, mock-judge run (3-test → real findings report), and
CLI dispatcher all exercised end-to-end before commit
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
12ed081f6b |
darwin-plugins iter 4 — ruflo-security-audit: --depth full → --depth deep (CLI doesnt support full)
This iter-4 edit was made by the headless agent but its commit step didnt land in the branch (rate-limit at end of workflow). Cleanly landing it now alongside the lockfile fix. |
||
|
ruv
|
0b5badac7e | darwin-plugins iter 3 — ruflo-security-audit: align memory_store namespace to documented security-findings | ||
|
ruv
|
4b07145a3f | darwin-plugins iter 3 — ruflo-ruvector: fix stale 103 MCP tool count in agent (missed by iter 2) | ||
|
ruv
|
b5cb847b70 | darwin-plugins iter 3 — ruflo-testgen: smoke step 2 covers new tdd-repair skill | ||
|
ruv
|
c82b3621bf | darwin-plugins iter 3 — ruflo-swarm: align swarm-init argument-hint with 6 supported topologies | ||
|
ruv
|
53369d1c78 | darwin-plugins iter 3 — ruflo-sparc: correct sparc-implement description to name actual SPARC phases (Pseudocode + Architecture) | ||
|
ruv
|
0024355145 | darwin-plugins iter 3 — ruflo-rag-memory: replace remaining unverified HNSW perf table with measured audit numbers | ||
|
ruv
|
73793eb91f | darwin-plugins iter 3 — ruflo-plugin-creator: align smoke check-count claim in SKILL.md (10+ → ≥8) with ADR/README | ||
|
ruv
|
473e473779 | darwin-plugins iter 3 — ruflo-observability: ADR Decision text says 'Accepted' (matches frontmatter + smoke check #9) | ||
|
ruv
|
c005412254 | darwin-plugins iter 3 — ruflo-knowledge-graph: flag semantic-route as disabled in agent tool list (matches kg.md + ADR-0001) | ||
|
ruv
|
3e532063e0 | darwin-plugins iter 3 — ruflo-market-data: align market patterns command with SKILL.md dual-path store | ||
|
ruv
|
e4ecb88ae3 | darwin-plugins iter 3 — ruflo-intelligence: align plugin.json tool-count breakdown with README inventory | ||
|
ruv
|
a29dcc522d | darwin-plugins iter 3 — ruflo-jujutsu: ADR Decision text now matches smoke check #9 (Accepted, not Proposed) | ||
|
ruv
|
1b4c92f462 | darwin-plugins iter 3 — ruflo-goals: sync smoke-check description with Accepted ADR status | ||
|
ruv
|
00f5fa7c49 | darwin-plugins iter 3 — ruflo-cost-tracker: fix subcommand count heading (20→23) | ||
|
ruv
|
fb7faef781 | darwin-plugins iter 3 — ruflo-adr: fix misleading verify.mjs exit-code header comment | ||
|
ruv
|
6c17d59256 | darwin-plugins iter 3 — ruflo-daa: sync ADR-0001 Context version to v0.2.0 (matches plugin.json) | ||
|
ruv
|
7fcdede483 | darwin-plugins iter 3 — ruflo-agentdb: fix stale '10 checks' count (smoke has 13 + 3 INV) | ||
|
ruv
|
44cf7fe82c | darwin-plugins iter 3 — ruflo-core: fix plugin.json description count (3→4 agents/skills) | ||
|
ruv
|
4fea343905 | darwin-plugins iter 3 — ruflo-arena: align README evolve example with CLI default (300, not 400) | ||
|
ruv
|
f251906d47 | darwin-plugins iter 2 — ruflo-metaharness: README listed 5 skills but plugin exposes 11 | ||
|
ruv
|
6ed682d931 | darwin-plugins iter 2 — ruflo-security-audit: fix invalid CLI flags (depth=deep not full, drop nonexistent --include-dev / cve --fix / security report) | ||
|
ruv
|
0ec7e7c6aa | darwin-plugins iter 2 — ruflo-sparc: fix namespace count (4→3 owned + patterns consumed) | ||
|
ruv
|
e3b8cf00ce | darwin-plugins iter 2 — ruflo-swarm: fix nonexistent Agent tool reference in swarm-init skill | ||
|
ruv
|
dc4b2d86d9 | darwin-plugins iter 2 — ruflo-ruvector: correct MCP tool count 103→91 (verified via mcp tools) | ||
|
ruv
|
51d0b3c4db | darwin-plugins iter 2 — ruflo-testgen: fix stale namespace name in ADR (testgen-gaps → test-gaps) | ||
|
ruv
|
de1436bf6f | darwin-plugins iter 2 — ruflo-observability: fix Skills table misrepresenting skills as slash-commands | ||
|
ruv
|
dcf269f7b5 | darwin-plugins iter 2 — ruflo-ruvllm: align chat-format argument-hint with 5 supported providers | ||
|
ruv
|
99dd600e27 | darwin-plugins iter 2 — ruflo-plugin-creator: fix validate-plugin contradiction (plugin.json must not have skills/commands/agents arrays) | ||
|
ruv
|
1432cd642b | darwin-plugins iter 2 — ruflo-rag-memory: replace unverified 150x-12,500x HNSW claim with measured numbers from intelligence audit | ||
|
ruv
|
ecd3a4ca80 | darwin-plugins iter 2 — ruflo-neural-trader: fix stale skill count (7→9) and document missing trader-explain / trader-portfolio-cg | ||
|
ruv
|
7d65f932d7 | darwin-plugins iter 2 — ruflo-loop-workers: drop invalid --train-neural flag from post-task example | ||
|
ruv
|
4be61d8f27 | darwin-plugins iter 2 — ruflo-jujutsu: invoke analyze_diff-reviewers in /jujutsu command | ||
|
ruv
|
ef87d00b6c | darwin-plugins iter 2 — ruflo-iot-cognitum: correct stale source-file count (32→39) | ||
|
ruv
|
b0af258b2b |
darwin-plugins iter 2 — ruflo-ddd: fix malformed Bash() permission patterns in skill frontmatter
Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01EYpeWgqLx3Q3yL2R2ePfmG |
||
|
ruv
|
0253936f4b | darwin-plugins iter 2 — ruflo-knowledge-graph: fix README namespace claim (kg-graph → knowledge-graph) to match actual code | ||
|
ruv
|
6b735a1537 | darwin-plugins iter 2 — ruflo-goals: fix invalid --train-neural flag on post-task (belongs to post-edit) | ||
|
ruv
|
16748510c6 | darwin-plugins iter 2 — ruflo-daa: tighten /daa command to use detailed:true and remove unmet knowledge-share aggregate claim | ||
|
ruv
|
0afaf63bc1 |
darwin-plugins iter 2 — ruflo-federation: fix misleading commands table (slash commands vs skills)
Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01EYpeWgqLx3Q3yL2R2ePfmG |
||
|
ruv
|
1df7d4e0d3 | darwin-plugins iter 2 — ruflo-autopilot: fix /autopilot config flag names to match MCP signature (maxIterations/timeoutMinutes) | ||
|
ruv
|
09eac92096 | darwin-plugins iter 2 — ruflo-graph-intelligence: remove broken bench script (missing scripts/benchmark-substrate.mjs) | ||
|
ruv
|
39bb83557c | darwin-plugins iter 2 — ruflo-browser: fix stale tool count (18 interaction + 5 lifecycle, not 23+5) | ||
|
ruv
|
4d7515cd06 | darwin-plugins iter 2 — ruflo-agent: document nested-subagents skill in README | ||
|
ruv
|
6ae2746ee4 | darwin-plugins iter 2 — ruflo-agentdb: replace unverified 150x-12,500x HNSW claims with measured ~1.9x-4.7x |