Files
ruflo/v3
rUvandClaude e036e35dd7 dream(memory): #3231 wire embedding near-dup detection into MemoryConsolidator.dedup() (evaluated, ACCEPT) (#3232)
* dream(memory): wire embedding near-duplicate detection into MemoryConsolidator.dedup()

MemoryConsolidator.dedup() only ever deduplicated entries via byte-exact
SHA-256 content hashing, even though every entry already carries a
computed .embedding and the adapter's HNSWIndex (already held in scope
for removal bookkeeping) is incrementally kept in sync. Paraphrases and
reformattings of the same memory were never caught.

Adds a second pass: for hash-pass survivors with an embedding, query the
already-populated HNSWIndex for cosine-similarity neighbors above a
configurable similarityThreshold (default 0.95, matching the unwired
domain-layer consolidator's own constant), and merge using the same
keeper-selection strategies (now factored into shared selectKeeper/
mergeGroup helpers). Guarded to cosine-metric indexes only; disabled via
similarityThreshold >= 1.

Evaluation: 471/472 passing in @claude-flow/memory (1 pre-existing,
unrelated, environmental failure); baseline (git-stash-isolated source)
fails exactly the 2 new discriminating tests and passes the rest,
confirming the fix is additive and non-regressive. tsc --noEmit clean.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): fix-forward — converge near-dup pass to a fixed point in one dedup() call

Independent adversarial critique (STEP 10) reproduced a real, bounded gap
in the prior commit: a near-duplicate cluster larger than
NEAR_DUP_SEARCH_K (8) split into multiple leftover sub-group survivors
within a single dedup() call, because each round's `consumed` bookkeeping
permanently excluded a group's keeper from further matching even though
it was still fully present in the index. 15 pairwise-identical-embedding
entries collapsed to 2 survivors (merged: 13) instead of 1 (merged: 14).

It self-healed across repeated runAll() calls (background timer /
nightlyLearner), so this was never a permanent-data-loss bug, but a
single manual dedup() call under-converged relative to its documented
"collapse duplicates" contract.

Fix: loop pass 2 to a fixed point (re-scan until a full round produces
zero merges) instead of a single scan. Always terminates — entries
strictly decrease each round that merges anything. `groups` now counts
merge operations across all rounds, which can exceed the number of
underlying duplicate clusters when one needed more than one round;
documented in the method's doc comment.

Added a discriminating regression test reproducing the critic's exact
15-entry scenario; confirmed it fails against the pre-fix single-round
code (merged: 13) and passes against this fix (merged: 14, single
survivor). Full package suite: 472/473 passing (1 pre-existing,
unrelated, environmental failure, unchanged from before this commit).
tsc --noEmit clean.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): 2026-09-08 research gist — embedding near-dup consolidation

Full SOTA report: 5-role parallel research fan-out, ledger check with
GitHub-verified fates for the last 14 rows (3 merged since 09-03, one PR
now crossing the 14-day stale threshold for the first time this cycle),
competitor comparison (CrewAI/Mem0/Zep/LangMem/Qdrant/SemDedup), plugins
and automation scan findings, adversarial critique with a real bug found
and fixed, and witness stamp.

No gist-creation MCP tool available in this session (GitHub MCP tools
cover issues/PRs/repos, not gists) — committing to docs/dream-cycle/
instead, matching every dream-cycle night since 2026-08-14.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): #3231 update LEDGER.md for 2026-09-08

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): #3231 empirical receipts for dedup() near-dup pass (response to review)

Response to PR #3232's INCONCLUSIVE review, which asked for a threshold
contract rather than a smoke fixture: a contamination-checked embedding
corpus with pinned model, precision/recall including hard negatives and
cross-namespace cases, baseline vs. candidate latency/memory at realistic
cardinality, and deterministic replay + unchanged exact-dedup behavior.

Real pinned-model corpus: attempted directly against both ONNX backends
this monorepo ships. Both fail for reasons unrelated to this candidate,
reproduced on an unmodified checkout before writing anything:
- @huggingface/transformers@4.2.0: transformers.node.mjs does
  `import { Tensor } from "onnxruntime-common"` against a CJS module — a
  real ESM/CJS interop break under this sandbox's Node version.
- @xenova/transformers@2.17.0: pulls in sharp@0.32.6, whose native
  binding (sharp-linux-x64.node) isn't present for this platform — the
  same failure independently reproduced moments later by this package's
  own nightlyLearner test, which falls back to mock embeddings.
Neither is fixable within this PR's scope without touching unrelated,
pre-existing native/module-resolution infrastructure. Documented in
detail at the top of the new test file rather than silently worked
around or omitted.

What's delivered instead, in
v3/@claude-flow/memory/src/consolidator-embedding-benchmark.test.ts:
- A deterministic synthetic corpus where pairwise cosine similarity is
  constructed EXACTLY via vectorAtSimilarity() (Gram-Schmidt against a
  random orthogonal vector), not measured after the fact — a rigorous
  instrument for threshold mechanics specifically, disclosed as not
  validating real-world semantic accuracy.
- Precision/recall/false-merge-rate across a 5-point threshold sweep,
  every point at a deliberate margin from every corpus similarity value
  after an exact-boundary collision (sweep value == corpus value) proved
  float32-jitter-sensitive while writing this — documented as a finding,
  not silently patched around.
- Namespace isolation: proved cross-namespace near-dup merging matches
  cross-namespace hash-exact merging exactly (both pre-existing,
  unscoped-by-namespace behavior — this candidate doesn't change it).
- Determinism: 3 independent runs, compared by deterministic entry KEY
  (not raw id, which is randomly generated per store() and was a second
  bug this file's own first draft had to fix).
- Latency/memory at N=5000 (matching this repo's own HNSW-benchmark
  convention): initially measured 9.8s for the candidate pass, which
  traced to an unset `ef` search parameter defaulting to efConstruction
  (200) — every dedup() search traversed a 200-candidate list for a
  duplicate-detection task that only needs very-close neighbors. Fixed
  by passing an explicit NEAR_DUP_SEARCH_EF=32 in consolidator.ts,
  re-measured at 3.7s (~2.7x), precision/recall unchanged (still exact).
  Real numbers logged in the test output either way, not asserted away.

Full package suite: 478/479 passing (472 pre-existing + 7 new; same 1
pre-existing unrelated environmental failure as every prior commit on
this branch). tsc --noEmit clean.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): #3231 add new benchmark file to CI test-ratchet baseline

CI on a2e99cf failed with "1 unexpected failing file(s): +
v3/@claude-flow/memory/src/consolidator-embedding-benchmark.test.ts".

Root-caused via the vitest JSON report artifact (test-results-ubuntu-latest,
downloaded and parsed directly, not guessed): a suite-level collection
failure, "Cannot find package '@claude-flow/security' imported from
'.../v3/@claude-flow/memory/src/agentdb-retrieval-guard.ts'" — the same
unbuilt-sibling-package gap already accepted in scripts/ci-test-baseline.txt
for 9 other @claude-flow/memory test files, including consolidator.test.ts
itself (line 95). Every file in this package fails identically in this CI
checkout state; my new file just wasn't in the baseline yet because it's
new. Not a regression this PR introduced — confirmed by the failure being
a pre-import-time module-resolution error, not a test assertion in the
new file's own logic (which passes locally in a checkout with @claude-flow/
memory's dependencies actually built: see the 6/6 local run in the prior
commit's message).

Fix: add the new file to the baseline, alongside its 9 memory-package
siblings already there, matching this repo's own established convention
for this exact gap rather than working around it some other way.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

* dream(memory): #3231 baseline 2 more federation/gateway plugin test files

CI on 20afdc8 failed (run 34905889339) after main was rebased in — 3
unexpected failing files, all pre-existing, unrelated to this PR's diff:

  plugins/ruflo-chatgpt-federation/test/oauth.test.mjs
    Cannot find package 'jose'
  plugins/ruflo-chatgpt-federation/test/publisher.test.mjs
    Cannot find package 'nostr-tools/pure'
  plugins/ruflo-x-gateway/test/oauth.test.mjs
    Cannot find package 'jose'

Verified via the vitest.json artifact (downloaded and parsed, not
guessed): all three are plugin-local-dependency collection failures,
identical in kind to the already-baselined sibling
plugins/ruflo-x-gateway/test/gateway.test.mjs ("node:test suite with its
own deps... root vitest cannot resolve them"). Confirmed these are new
on main itself (main's own ci-test-baseline.txt at the rebase point does
not list them either) and unrelated to v3/@claude-flow/memory — nothing
this PR touches.

Re-verified locally: re-ran the full @claude-flow/memory suite against
the rebased main (496/497 passing, same 1 pre-existing unrelated
failure as always) and recomputed the ratchet's unexpected-set directly
against the CI run's vitest.json artifact — 0 unexpected after this
change.

Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01H4kfasS4XkYaSGYXFLSSEG

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-14 23:13:14 +00:00
..
2026-01-13 00:06:11 +00:00
2026-01-04 17:41:42 +00:00
2026-01-04 17:39:24 +00:00
2026-01-05 20:52:37 +00:00
2026-01-04 17:40:48 +00:00
2026-01-05 19:12:36 +00:00

Claude Flow V3

Modular AI Agent Coordination System - A complete reimagining of Claude-Flow with 15-agent hierarchical mesh swarm coordination.

Version Node.js TypeScript License

Introduction

Claude Flow V3 is a next-generation AI agent coordination system built on 10 Architecture Decision Records (ADRs). It provides a modular, security-first, high-performance platform for orchestrating multi-agent swarms with hierarchical mesh topology.

V3 represents a complete architectural overhaul:

  • 10x faster testing with Vitest
  • 150x-12,500x faster search with HNSW indexing
  • 2.49x-7.47x Flash Attention speedup
  • 50-75% memory reduction

Features

Core Capabilities

  • 15-Agent Hierarchical Mesh - Queen-led coordination with specialized worker agents
  • Domain-Driven Design - Clean bounded contexts with separation of concerns
  • Plugin Architecture - Microkernel pattern for extensibility
  • MCP-First API - Consistent interfaces across all modules
  • Event Sourcing - Full audit trail for state changes
  • Hybrid Memory Backend - SQLite + AgentDB for optimal performance

Security

  • CVE Remediation - All known vulnerabilities addressed
  • Input Validation - Zod-based schema validation
  • Secure ID Generation - Cryptographic random IDs
  • Path Security - Traversal protection
  • SQL Injection Prevention - Parameterized queries

Performance

Metric Target Achieved
Event Bus (100k events) <50ms ~6ms
Map Lookup (100k gets) <20ms ~16ms
Array.find vs Map O(1) N/A 978x speedup
Flash Attention 2.49x-7.47x Validated
AgentDB Search 150x-12,500x HNSW indexed

Architecture

Architecture Decision Records (ADRs)

ADR Decision
ADR-001 Adopt agentic-flow as core foundation
ADR-002 Domain-Driven Design structure
ADR-003 Single coordination engine (UnifiedSwarmCoordinator)
ADR-004 Plugin-based architecture (microkernel)
ADR-005 MCP-first API design
ADR-006 Unified memory service (AgentDB)
ADR-007 Event sourcing for state changes
ADR-008 Vitest over Jest (10x faster)
ADR-009 Hybrid memory backend default
ADR-010 Remove Deno support (Node.js 20+ only)

Module Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     @claude-flow/v3-monorepo                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │   security   │  │    memory    │  │    swarm     │          │
│  │  CVE fixes   │  │   AgentDB    │  │ 15-agent     │          │
│  │  validation  │  │   HNSW       │  │ coordination │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │ integration  │  │  performance │  │    neural    │          │
│  │ agentic-flow │  │ Flash Attn   │  │   SONA       │          │
│  │  bridge      │  │ benchmarks   │  │  learning    │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
│                                                                 │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │     cli      │  │   testing    │  │  deployment  │          │
│  │  commands    │  │ TDD London   │  │   release    │          │
│  │  prompts     │  │   School     │  │    CI/CD     │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
│                                                                 │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                       shared                             │   │
│  │  types • events • core • hooks • resilience • plugins   │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Directory Structure

v3/
├── @claude-flow/                    # Modular packages
│   ├── security/                    # Security module
│   │   └── src/
│   │       ├── index.ts             # Password hashing, validators
│   │       └── ...
│   │
│   ├── memory/                      # Memory module
│   │   ├── src/
│   │   │   ├── agentdb-backend.ts   # AgentDB integration
│   │   │   ├── hnsw-index.ts        # HNSW vector indexing
│   │   │   ├── hybrid-backend.ts    # SQLite + AgentDB
│   │   │   ├── sqlite-backend.ts    # SQLite backend
│   │   │   ├── cache-manager.ts     # Caching layer
│   │   │   └── domain/              # DDD entities
│   │   ├── benchmarks/              # Performance benchmarks
│   │   └── examples/                # Usage examples
│   │
│   ├── swarm/                       # Swarm coordination
│   │   └── src/
│   │       ├── unified-coordinator.ts  # Main coordinator
│   │       ├── topology-manager.ts     # Topology management
│   │       ├── consensus/              # Consensus protocols
│   │       └── domain/                 # DDD entities
│   │
│   ├── integration/                 # agentic-flow integration
│   │   └── src/
│   │       ├── agentic-flow-bridge.ts  # Core bridge
│   │       ├── agent-adapter.ts        # Agent adaptation
│   │       └── sona-adapter.ts         # SONA learning
│   │
│   ├── performance/                 # Performance module
│   │   ├── src/
│   │   │   └── framework/           # Benchmark framework
│   │   └── benchmarks/
│   │       ├── startup/             # Startup benchmarks
│   │       └── attention/           # Flash Attention
│   │
│   ├── neural/                      # Neural/SONA module
│   │   └── src/
│   │       ├── algorithms/          # Learning algorithms
│   │       └── modes/               # Neural modes
│   │
│   ├── cli/                         # CLI module
│   │   ├── bin/                     # Executable
│   │   └── src/
│   │       └── commands/            # Command handlers
│   │
│   ├── testing/                     # Testing framework
│   │   └── src/
│   │       ├── fixtures/            # Test fixtures
│   │       ├── mocks/               # Mock services
│   │       ├── helpers/             # Test helpers
│   │       └── regression/          # Regression tests
│   │
│   ├── shared/                      # Shared utilities
│   │   └── src/
│   │       ├── types/               # Shared types
│   │       ├── events/              # Event system
│   │       ├── core/                # Core interfaces
│   │       ├── hooks/               # Hook system
│   │       ├── resilience/          # Retry, circuit breaker
│   │       ├── plugins/             # Plugin system
│   │       └── security/            # Security utilities
│   │
│   └── deployment/                  # Deployment module
│       └── src/                     # Release management
│
├── mcp/                             # MCP Server
│   ├── server.ts                    # Main server
│   ├── tools/                       # MCP tools
│   │   ├── agent-tools.ts
│   │   ├── swarm-tools.ts
│   │   ├── memory-tools.ts
│   │   └── hooks-tools.ts
│   └── transport/                   # Transport layers
│       ├── stdio.ts
│       ├── http.ts
│       └── websocket.ts
│
├── __tests__/                       # Integration tests
│   └── integration/
│       ├── memory-integration.test.ts
│       ├── swarm-integration.test.ts
│       ├── mcp-integration.test.ts
│       └── workflow-integration.test.ts
│
├── docs/                            # Documentation
│   ├── README.md                    # Docs overview
│   ├── guides/                      # User guides
│   └── implementation/              # Implementation details
│
├── helpers/                         # Cross-platform helpers
│   ├── claude-flow-v3.sh            # Master helper (Linux/macOS)
│   ├── claude-flow-v3.ps1           # Master helper (Windows)
│   └── templates/                   # Helper templates
│
├── scripts/                         # Utility scripts
│   └── quick-benchmark.mjs          # Quick perf test
│
├── index.ts                         # Main entry point
├── swarm.config.ts                  # Swarm configuration
├── vitest.config.ts                 # Test configuration
└── package.json                     # Monorepo package

Modules

@claude-flow/security

Security-first implementation with CVE fixes, input validation, and credential management.

import { PasswordHasher, validateInput, sanitizePath } from '@claude-flow/security';

const hasher = new PasswordHasher();
const hash = await hasher.hash('password');
const valid = await hasher.verify('password', hash);

@claude-flow/memory

Unified memory service with AgentDB, HNSW indexing, and 150x-12,500x faster search.

import { HybridMemoryRepository, HNSWIndex } from '@claude-flow/memory';

const memory = new HybridMemoryRepository({
  backend: 'agentdb',
  vectorSearch: true
});

await memory.store({ key: 'knowledge', value: 'context', embedding: [...] });
const results = await memory.search({ query: 'knowledge', limit: 10 });

@claude-flow/swarm

15-agent hierarchical mesh coordination with consensus protocols.

import { UnifiedSwarmCoordinator } from '@claude-flow/swarm';

const coordinator = new UnifiedSwarmCoordinator({
  topology: 'hierarchical-mesh',
  maxAgents: 15
});

await coordinator.initialize();
await coordinator.spawnAgent({ type: 'queen-coordinator' });

@claude-flow/integration

Deep integration with agentic-flow@alpha per ADR-001.

import { AgenticFlowBridge } from '@claude-flow/integration';

const bridge = new AgenticFlowBridge();
await bridge.initialize();
const agent = await bridge.createAgent({ type: 'coder' });

@claude-flow/performance

Benchmarking framework with Flash Attention validation.

import { BenchmarkRunner, formatTime } from '@claude-flow/performance';

const runner = new BenchmarkRunner();
const result = await runner.run('map-lookup', () => map.get(key), {
  iterations: 100000,
  targetTime: 20
});

@claude-flow/neural

SONA learning integration for self-optimizing agents.

import { SONAAdapter } from '@claude-flow/neural';

const sona = new SONAAdapter();
await sona.train({ patterns: learningData });
const prediction = await sona.predict(context);

@claude-flow/cli

Modern CLI with interactive prompts and formatted output.

npx @claude-flow/cli swarm init --topology hierarchical-mesh
npx @claude-flow/cli agent spawn --type queen-coordinator
npx @claude-flow/cli memory search "knowledge"

@claude-flow/testing

TDD London School framework with mocks, fixtures, and regression testing.

import { createMockAgent, createTestFixture } from '@claude-flow/testing';

const mockAgent = createMockAgent({ type: 'coder' });
const fixture = createTestFixture('swarm-coordination');

@claude-flow/shared

Common types, events, utilities, and core interfaces.

import { EventBus, Result, success, failure } from '@claude-flow/shared';
import type { AgentId, TaskStatus } from '@claude-flow/shared/types';

@claude-flow/deployment

Release management and CI/CD automation.

import { ReleaseManager } from '@claude-flow/deployment';

const release = new ReleaseManager();
await release.prepare({ version: '3.0.0', changelog: '...' });

Usage

Quick Start

import { initializeV3Swarm } from '@claude-flow/v3';

// Initialize the swarm
const swarm = await initializeV3Swarm();

// Spawn agents
await swarm.spawnAllAgents();

// Submit a task
const task = swarm.submitTask({
  type: 'implementation',
  title: 'Implement feature X',
  description: 'Detailed description...',
  domain: 'core',
  phase: 'phase-2-core',
  priority: 'high'
});

// Wait for completion
const result = await swarm.waitForTask(task.id);

Import Specific Modules

// Import everything
import * as claudeFlow from '@claude-flow/v3';

// Or import specific modules for tree-shaking
import { UnifiedSwarmCoordinator } from '@claude-flow/swarm';
import { PasswordHasher } from '@claude-flow/security';
import { HNSWIndex } from '@claude-flow/memory';

MCP Server

import { createMCPServer } from '@claude-flow/v3/mcp';

const server = createMCPServer({
  transport: 'stdio',
  tools: ['agent', 'swarm', 'memory', 'hooks']
});

await server.start();

Helper System

Cross-platform automation for V3 development:

# Linux/macOS
./helpers/claude-flow-v3.sh init
./helpers/claude-flow-v3.sh status
./helpers/claude-flow-v3.sh update domain 3

# Windows (PowerShell)
.\helpers\claude-flow-v3.ps1 init
.\helpers\claude-flow-v3.ps1 status
.\helpers\claude-flow-v3.ps1 update domain 3

Features:

  • Progress Tracking: Real-time domain/agent/performance metrics
  • Checkpointing: Auto-commit with development milestones
  • Validation: Environment and configuration verification
  • GitHub Integration: PR management and issue tracking

Installation

# Clone the repository
git clone https://github.com/ruvnet/claude-flow.git
cd claude-flow/v3

# Install dependencies
pnpm install

# Build all modules
pnpm build

Testing

# Run all tests
pnpm test

# Run integration tests
pnpm test:integration

# Run specific module tests
pnpm test:memory
pnpm test:swarm
pnpm test:security

# Run benchmarks
pnpm bench

# Quick benchmark (no dependencies)
node scripts/quick-benchmark.mjs

# Coverage report
pnpm test:coverage

Performance Targets

Category Metric Target
Search AgentDB HNSW 150x-12,500x faster
Attention Flash Attention 2.49x-7.47x speedup
Memory Reduction 50-75%
Code Total lines <5,000
Startup Cold start <500ms
Learning SONA adaptation <0.05ms

Documentation

Modules

Examples

MCP Tools

External

Requirements

  • Node.js: >=20.0.0
  • pnpm: >=8.0.0
  • TypeScript: >=5.3.0

License

MIT License - See LICENSE for details.


Built with the SPARC methodology and 15-agent hierarchical mesh coordination.