* dream(memory): wire embedding near-duplicate detection into MemoryConsolidator.dedup()
MemoryConsolidator.dedup() only ever deduplicated entries via byte-exact
SHA-256 content hashing, even though every entry already carries a
computed .embedding and the adapter's HNSWIndex (already held in scope
for removal bookkeeping) is incrementally kept in sync. Paraphrases and
reformattings of the same memory were never caught.
Adds a second pass: for hash-pass survivors with an embedding, query the
already-populated HNSWIndex for cosine-similarity neighbors above a
configurable similarityThreshold (default 0.95, matching the unwired
domain-layer consolidator's own constant), and merge using the same
keeper-selection strategies (now factored into shared selectKeeper/
mergeGroup helpers). Guarded to cosine-metric indexes only; disabled via
similarityThreshold >= 1.
Evaluation: 471/472 passing in @claude-flow/memory (1 pre-existing,
unrelated, environmental failure); baseline (git-stash-isolated source)
fails exactly the 2 new discriminating tests and passes the rest,
confirming the fix is additive and non-regressive. tsc --noEmit clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): fix-forward — converge near-dup pass to a fixed point in one dedup() call
Independent adversarial critique (STEP 10) reproduced a real, bounded gap
in the prior commit: a near-duplicate cluster larger than
NEAR_DUP_SEARCH_K (8) split into multiple leftover sub-group survivors
within a single dedup() call, because each round's `consumed` bookkeeping
permanently excluded a group's keeper from further matching even though
it was still fully present in the index. 15 pairwise-identical-embedding
entries collapsed to 2 survivors (merged: 13) instead of 1 (merged: 14).
It self-healed across repeated runAll() calls (background timer /
nightlyLearner), so this was never a permanent-data-loss bug, but a
single manual dedup() call under-converged relative to its documented
"collapse duplicates" contract.
Fix: loop pass 2 to a fixed point (re-scan until a full round produces
zero merges) instead of a single scan. Always terminates — entries
strictly decrease each round that merges anything. `groups` now counts
merge operations across all rounds, which can exceed the number of
underlying duplicate clusters when one needed more than one round;
documented in the method's doc comment.
Added a discriminating regression test reproducing the critic's exact
15-entry scenario; confirmed it fails against the pre-fix single-round
code (merged: 13) and passes against this fix (merged: 14, single
survivor). Full package suite: 472/473 passing (1 pre-existing,
unrelated, environmental failure, unchanged from before this commit).
tsc --noEmit clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): 2026-09-08 research gist — embedding near-dup consolidation
Full SOTA report: 5-role parallel research fan-out, ledger check with
GitHub-verified fates for the last 14 rows (3 merged since 09-03, one PR
now crossing the 14-day stale threshold for the first time this cycle),
competitor comparison (CrewAI/Mem0/Zep/LangMem/Qdrant/SemDedup), plugins
and automation scan findings, adversarial critique with a real bug found
and fixed, and witness stamp.
No gist-creation MCP tool available in this session (GitHub MCP tools
cover issues/PRs/repos, not gists) — committing to docs/dream-cycle/
instead, matching every dream-cycle night since 2026-08-14.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): #3231 update LEDGER.md for 2026-09-08
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): #3231 empirical receipts for dedup() near-dup pass (response to review)
Response to PR #3232's INCONCLUSIVE review, which asked for a threshold
contract rather than a smoke fixture: a contamination-checked embedding
corpus with pinned model, precision/recall including hard negatives and
cross-namespace cases, baseline vs. candidate latency/memory at realistic
cardinality, and deterministic replay + unchanged exact-dedup behavior.
Real pinned-model corpus: attempted directly against both ONNX backends
this monorepo ships. Both fail for reasons unrelated to this candidate,
reproduced on an unmodified checkout before writing anything:
- @huggingface/transformers@4.2.0: transformers.node.mjs does
`import { Tensor } from "onnxruntime-common"` against a CJS module — a
real ESM/CJS interop break under this sandbox's Node version.
- @xenova/transformers@2.17.0: pulls in sharp@0.32.6, whose native
binding (sharp-linux-x64.node) isn't present for this platform — the
same failure independently reproduced moments later by this package's
own nightlyLearner test, which falls back to mock embeddings.
Neither is fixable within this PR's scope without touching unrelated,
pre-existing native/module-resolution infrastructure. Documented in
detail at the top of the new test file rather than silently worked
around or omitted.
What's delivered instead, in
v3/@claude-flow/memory/src/consolidator-embedding-benchmark.test.ts:
- A deterministic synthetic corpus where pairwise cosine similarity is
constructed EXACTLY via vectorAtSimilarity() (Gram-Schmidt against a
random orthogonal vector), not measured after the fact — a rigorous
instrument for threshold mechanics specifically, disclosed as not
validating real-world semantic accuracy.
- Precision/recall/false-merge-rate across a 5-point threshold sweep,
every point at a deliberate margin from every corpus similarity value
after an exact-boundary collision (sweep value == corpus value) proved
float32-jitter-sensitive while writing this — documented as a finding,
not silently patched around.
- Namespace isolation: proved cross-namespace near-dup merging matches
cross-namespace hash-exact merging exactly (both pre-existing,
unscoped-by-namespace behavior — this candidate doesn't change it).
- Determinism: 3 independent runs, compared by deterministic entry KEY
(not raw id, which is randomly generated per store() and was a second
bug this file's own first draft had to fix).
- Latency/memory at N=5000 (matching this repo's own HNSW-benchmark
convention): initially measured 9.8s for the candidate pass, which
traced to an unset `ef` search parameter defaulting to efConstruction
(200) — every dedup() search traversed a 200-candidate list for a
duplicate-detection task that only needs very-close neighbors. Fixed
by passing an explicit NEAR_DUP_SEARCH_EF=32 in consolidator.ts,
re-measured at 3.7s (~2.7x), precision/recall unchanged (still exact).
Real numbers logged in the test output either way, not asserted away.
Full package suite: 478/479 passing (472 pre-existing + 7 new; same 1
pre-existing unrelated environmental failure as every prior commit on
this branch). tsc --noEmit clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): #3231 add new benchmark file to CI test-ratchet baseline
CI on a2e99cf failed with "1 unexpected failing file(s): +
v3/@claude-flow/memory/src/consolidator-embedding-benchmark.test.ts".
Root-caused via the vitest JSON report artifact (test-results-ubuntu-latest,
downloaded and parsed directly, not guessed): a suite-level collection
failure, "Cannot find package '@claude-flow/security' imported from
'.../v3/@claude-flow/memory/src/agentdb-retrieval-guard.ts'" — the same
unbuilt-sibling-package gap already accepted in scripts/ci-test-baseline.txt
for 9 other @claude-flow/memory test files, including consolidator.test.ts
itself (line 95). Every file in this package fails identically in this CI
checkout state; my new file just wasn't in the baseline yet because it's
new. Not a regression this PR introduced — confirmed by the failure being
a pre-import-time module-resolution error, not a test assertion in the
new file's own logic (which passes locally in a checkout with @claude-flow/
memory's dependencies actually built: see the 6/6 local run in the prior
commit's message).
Fix: add the new file to the baseline, alongside its 9 memory-package
siblings already there, matching this repo's own established convention
for this exact gap rather than working around it some other way.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
* dream(memory): #3231 baseline 2 more federation/gateway plugin test files
CI on 20afdc8 failed (run 34905889339) after main was rebased in — 3
unexpected failing files, all pre-existing, unrelated to this PR's diff:
plugins/ruflo-chatgpt-federation/test/oauth.test.mjs
Cannot find package 'jose'
plugins/ruflo-chatgpt-federation/test/publisher.test.mjs
Cannot find package 'nostr-tools/pure'
plugins/ruflo-x-gateway/test/oauth.test.mjs
Cannot find package 'jose'
Verified via the vitest.json artifact (downloaded and parsed, not
guessed): all three are plugin-local-dependency collection failures,
identical in kind to the already-baselined sibling
plugins/ruflo-x-gateway/test/gateway.test.mjs ("node:test suite with its
own deps... root vitest cannot resolve them"). Confirmed these are new
on main itself (main's own ci-test-baseline.txt at the rebase point does
not list them either) and unrelated to v3/@claude-flow/memory — nothing
this PR touches.
Re-verified locally: re-ran the full @claude-flow/memory suite against
the rebased main (496/497 passing, same 1 pre-existing unrelated
failure as always) and recomputed the ratchet's unexpected-set directly
against the CI run's vitest.json artifact — 0 unexpected after this
change.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG
---------
Co-authored-by: Claude <noreply@anthropic.com>
Claude Flow V3
Modular AI Agent Coordination System - A complete reimagining of Claude-Flow with 15-agent hierarchical mesh swarm coordination.
Introduction
Claude Flow V3 is a next-generation AI agent coordination system built on 10 Architecture Decision Records (ADRs). It provides a modular, security-first, high-performance platform for orchestrating multi-agent swarms with hierarchical mesh topology.
V3 represents a complete architectural overhaul:
- 10x faster testing with Vitest
- 150x-12,500x faster search with HNSW indexing
- 2.49x-7.47x Flash Attention speedup
- 50-75% memory reduction
Features
Core Capabilities
- 15-Agent Hierarchical Mesh - Queen-led coordination with specialized worker agents
- Domain-Driven Design - Clean bounded contexts with separation of concerns
- Plugin Architecture - Microkernel pattern for extensibility
- MCP-First API - Consistent interfaces across all modules
- Event Sourcing - Full audit trail for state changes
- Hybrid Memory Backend - SQLite + AgentDB for optimal performance
Security
- CVE Remediation - All known vulnerabilities addressed
- Input Validation - Zod-based schema validation
- Secure ID Generation - Cryptographic random IDs
- Path Security - Traversal protection
- SQL Injection Prevention - Parameterized queries
Performance
| Metric | Target | Achieved |
|---|---|---|
| Event Bus (100k events) | <50ms | ~6ms |
| Map Lookup (100k gets) | <20ms | ~16ms |
| Array.find vs Map O(1) | N/A | 978x speedup |
| Flash Attention | 2.49x-7.47x | Validated |
| AgentDB Search | 150x-12,500x | HNSW indexed |
Architecture
Architecture Decision Records (ADRs)
| ADR | Decision |
|---|---|
| ADR-001 | Adopt agentic-flow as core foundation |
| ADR-002 | Domain-Driven Design structure |
| ADR-003 | Single coordination engine (UnifiedSwarmCoordinator) |
| ADR-004 | Plugin-based architecture (microkernel) |
| ADR-005 | MCP-first API design |
| ADR-006 | Unified memory service (AgentDB) |
| ADR-007 | Event sourcing for state changes |
| ADR-008 | Vitest over Jest (10x faster) |
| ADR-009 | Hybrid memory backend default |
| ADR-010 | Remove Deno support (Node.js 20+ only) |
Module Architecture
┌─────────────────────────────────────────────────────────────────┐
│ @claude-flow/v3-monorepo │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ security │ │ memory │ │ swarm │ │
│ │ CVE fixes │ │ AgentDB │ │ 15-agent │ │
│ │ validation │ │ HNSW │ │ coordination │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ integration │ │ performance │ │ neural │ │
│ │ agentic-flow │ │ Flash Attn │ │ SONA │ │
│ │ bridge │ │ benchmarks │ │ learning │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ cli │ │ testing │ │ deployment │ │
│ │ commands │ │ TDD London │ │ release │ │
│ │ prompts │ │ School │ │ CI/CD │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ shared │ │
│ │ types • events • core • hooks • resilience • plugins │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
Directory Structure
v3/
├── @claude-flow/ # Modular packages
│ ├── security/ # Security module
│ │ └── src/
│ │ ├── index.ts # Password hashing, validators
│ │ └── ...
│ │
│ ├── memory/ # Memory module
│ │ ├── src/
│ │ │ ├── agentdb-backend.ts # AgentDB integration
│ │ │ ├── hnsw-index.ts # HNSW vector indexing
│ │ │ ├── hybrid-backend.ts # SQLite + AgentDB
│ │ │ ├── sqlite-backend.ts # SQLite backend
│ │ │ ├── cache-manager.ts # Caching layer
│ │ │ └── domain/ # DDD entities
│ │ ├── benchmarks/ # Performance benchmarks
│ │ └── examples/ # Usage examples
│ │
│ ├── swarm/ # Swarm coordination
│ │ └── src/
│ │ ├── unified-coordinator.ts # Main coordinator
│ │ ├── topology-manager.ts # Topology management
│ │ ├── consensus/ # Consensus protocols
│ │ └── domain/ # DDD entities
│ │
│ ├── integration/ # agentic-flow integration
│ │ └── src/
│ │ ├── agentic-flow-bridge.ts # Core bridge
│ │ ├── agent-adapter.ts # Agent adaptation
│ │ └── sona-adapter.ts # SONA learning
│ │
│ ├── performance/ # Performance module
│ │ ├── src/
│ │ │ └── framework/ # Benchmark framework
│ │ └── benchmarks/
│ │ ├── startup/ # Startup benchmarks
│ │ └── attention/ # Flash Attention
│ │
│ ├── neural/ # Neural/SONA module
│ │ └── src/
│ │ ├── algorithms/ # Learning algorithms
│ │ └── modes/ # Neural modes
│ │
│ ├── cli/ # CLI module
│ │ ├── bin/ # Executable
│ │ └── src/
│ │ └── commands/ # Command handlers
│ │
│ ├── testing/ # Testing framework
│ │ └── src/
│ │ ├── fixtures/ # Test fixtures
│ │ ├── mocks/ # Mock services
│ │ ├── helpers/ # Test helpers
│ │ └── regression/ # Regression tests
│ │
│ ├── shared/ # Shared utilities
│ │ └── src/
│ │ ├── types/ # Shared types
│ │ ├── events/ # Event system
│ │ ├── core/ # Core interfaces
│ │ ├── hooks/ # Hook system
│ │ ├── resilience/ # Retry, circuit breaker
│ │ ├── plugins/ # Plugin system
│ │ └── security/ # Security utilities
│ │
│ └── deployment/ # Deployment module
│ └── src/ # Release management
│
├── mcp/ # MCP Server
│ ├── server.ts # Main server
│ ├── tools/ # MCP tools
│ │ ├── agent-tools.ts
│ │ ├── swarm-tools.ts
│ │ ├── memory-tools.ts
│ │ └── hooks-tools.ts
│ └── transport/ # Transport layers
│ ├── stdio.ts
│ ├── http.ts
│ └── websocket.ts
│
├── __tests__/ # Integration tests
│ └── integration/
│ ├── memory-integration.test.ts
│ ├── swarm-integration.test.ts
│ ├── mcp-integration.test.ts
│ └── workflow-integration.test.ts
│
├── docs/ # Documentation
│ ├── README.md # Docs overview
│ ├── guides/ # User guides
│ └── implementation/ # Implementation details
│
├── helpers/ # Cross-platform helpers
│ ├── claude-flow-v3.sh # Master helper (Linux/macOS)
│ ├── claude-flow-v3.ps1 # Master helper (Windows)
│ └── templates/ # Helper templates
│
├── scripts/ # Utility scripts
│ └── quick-benchmark.mjs # Quick perf test
│
├── index.ts # Main entry point
├── swarm.config.ts # Swarm configuration
├── vitest.config.ts # Test configuration
└── package.json # Monorepo package
Modules
@claude-flow/security
Security-first implementation with CVE fixes, input validation, and credential management.
import { PasswordHasher, validateInput, sanitizePath } from '@claude-flow/security';
const hasher = new PasswordHasher();
const hash = await hasher.hash('password');
const valid = await hasher.verify('password', hash);
@claude-flow/memory
Unified memory service with AgentDB, HNSW indexing, and 150x-12,500x faster search.
import { HybridMemoryRepository, HNSWIndex } from '@claude-flow/memory';
const memory = new HybridMemoryRepository({
backend: 'agentdb',
vectorSearch: true
});
await memory.store({ key: 'knowledge', value: 'context', embedding: [...] });
const results = await memory.search({ query: 'knowledge', limit: 10 });
@claude-flow/swarm
15-agent hierarchical mesh coordination with consensus protocols.
import { UnifiedSwarmCoordinator } from '@claude-flow/swarm';
const coordinator = new UnifiedSwarmCoordinator({
topology: 'hierarchical-mesh',
maxAgents: 15
});
await coordinator.initialize();
await coordinator.spawnAgent({ type: 'queen-coordinator' });
@claude-flow/integration
Deep integration with agentic-flow@alpha per ADR-001.
import { AgenticFlowBridge } from '@claude-flow/integration';
const bridge = new AgenticFlowBridge();
await bridge.initialize();
const agent = await bridge.createAgent({ type: 'coder' });
@claude-flow/performance
Benchmarking framework with Flash Attention validation.
import { BenchmarkRunner, formatTime } from '@claude-flow/performance';
const runner = new BenchmarkRunner();
const result = await runner.run('map-lookup', () => map.get(key), {
iterations: 100000,
targetTime: 20
});
@claude-flow/neural
SONA learning integration for self-optimizing agents.
import { SONAAdapter } from '@claude-flow/neural';
const sona = new SONAAdapter();
await sona.train({ patterns: learningData });
const prediction = await sona.predict(context);
@claude-flow/cli
Modern CLI with interactive prompts and formatted output.
npx @claude-flow/cli swarm init --topology hierarchical-mesh
npx @claude-flow/cli agent spawn --type queen-coordinator
npx @claude-flow/cli memory search "knowledge"
@claude-flow/testing
TDD London School framework with mocks, fixtures, and regression testing.
import { createMockAgent, createTestFixture } from '@claude-flow/testing';
const mockAgent = createMockAgent({ type: 'coder' });
const fixture = createTestFixture('swarm-coordination');
@claude-flow/shared
Common types, events, utilities, and core interfaces.
import { EventBus, Result, success, failure } from '@claude-flow/shared';
import type { AgentId, TaskStatus } from '@claude-flow/shared/types';
@claude-flow/deployment
Release management and CI/CD automation.
import { ReleaseManager } from '@claude-flow/deployment';
const release = new ReleaseManager();
await release.prepare({ version: '3.0.0', changelog: '...' });
Usage
Quick Start
import { initializeV3Swarm } from '@claude-flow/v3';
// Initialize the swarm
const swarm = await initializeV3Swarm();
// Spawn agents
await swarm.spawnAllAgents();
// Submit a task
const task = swarm.submitTask({
type: 'implementation',
title: 'Implement feature X',
description: 'Detailed description...',
domain: 'core',
phase: 'phase-2-core',
priority: 'high'
});
// Wait for completion
const result = await swarm.waitForTask(task.id);
Import Specific Modules
// Import everything
import * as claudeFlow from '@claude-flow/v3';
// Or import specific modules for tree-shaking
import { UnifiedSwarmCoordinator } from '@claude-flow/swarm';
import { PasswordHasher } from '@claude-flow/security';
import { HNSWIndex } from '@claude-flow/memory';
MCP Server
import { createMCPServer } from '@claude-flow/v3/mcp';
const server = createMCPServer({
transport: 'stdio',
tools: ['agent', 'swarm', 'memory', 'hooks']
});
await server.start();
Helper System
Cross-platform automation for V3 development:
# Linux/macOS
./helpers/claude-flow-v3.sh init
./helpers/claude-flow-v3.sh status
./helpers/claude-flow-v3.sh update domain 3
# Windows (PowerShell)
.\helpers\claude-flow-v3.ps1 init
.\helpers\claude-flow-v3.ps1 status
.\helpers\claude-flow-v3.ps1 update domain 3
Features:
- Progress Tracking: Real-time domain/agent/performance metrics
- Checkpointing: Auto-commit with development milestones
- Validation: Environment and configuration verification
- GitHub Integration: PR management and issue tracking
Installation
# Clone the repository
git clone https://github.com/ruvnet/claude-flow.git
cd claude-flow/v3
# Install dependencies
pnpm install
# Build all modules
pnpm build
Testing
# Run all tests
pnpm test
# Run integration tests
pnpm test:integration
# Run specific module tests
pnpm test:memory
pnpm test:swarm
pnpm test:security
# Run benchmarks
pnpm bench
# Quick benchmark (no dependencies)
node scripts/quick-benchmark.mjs
# Coverage report
pnpm test:coverage
Performance Targets
| Category | Metric | Target |
|---|---|---|
| Search | AgentDB HNSW | 150x-12,500x faster |
| Attention | Flash Attention | 2.49x-7.47x speedup |
| Memory | Reduction | 50-75% |
| Code | Total lines | <5,000 |
| Startup | Cold start | <500ms |
| Learning | SONA adaptation | <0.05ms |
Links
Documentation
Modules
- @claude-flow/security
- @claude-flow/memory
- @claude-flow/swarm
- @claude-flow/integration
- @claude-flow/performance
- @claude-flow/neural
- @claude-flow/cli
- @claude-flow/testing
- @claude-flow/shared
- @claude-flow/deployment
Examples
MCP Tools
External
Requirements
- Node.js: >=20.0.0
- pnpm: >=8.0.0
- TypeScript: >=5.3.0
License
MIT License - See LICENSE for details.
Built with the SPARC methodology and 15-agent hierarchical mesh coordination.