diff --git a/docs/dream-cycle/LEDGER.md b/docs/dream-cycle/LEDGER.md index e6bd5b207f..538828534f 100644 --- a/docs/dream-cycle/LEDGER.md +++ b/docs/dream-cycle/LEDGER.md @@ -132,6 +132,7 @@ alerting. | 2026-08-18 | memory | hybridSearch controller reachable via explicit opt-in (was silently null-returning despite config flag) | #3056 | #3057 | yes | ACCEPT-scoped | overall recall@10 +0.267; category B (pure-paraphrase) regresses -0.133 | b28714fb... | recovered-live | | 2026-08-19 | swarm | MessageBus retry-attempts silently reset to 0 on every re-queue, unbounded redelivery, message.failed unreachable | #3061 | #3062 | yes | ACCEPT | invocations 96-98/0-failed (baseline) → 3/2/1 stable (candidate); 220/220 tests | 62c4fdf7... | 08-14..08-18 all OPEN, none merged yet | + **Recovery note (2026-09-05):** rows for 2026-08-24 through 2026-09-03 were again never appended live — same failure class as the 2026-08-19 recovery above, now confirmed for a 3rd distinct occurrence. Verified via direct @@ -159,11 +160,15 @@ finding" to an explicit recommendation: a future `automation`/`meta` DEEP night should treat "harden STEP 25's ledger-append against silent failure" as a first-class, high-priority candidate, not a background note. +| 2026-08-20 | — | **no-run gap** — no branch/PR/issue exists for this date | — | — | no | — | — | — | no-run gap (external, not diagnosed) | +| 2026-08-21 | — | **no-run gap** — no branch/PR/issue exists for this date | — | — | no | — | — | — | no-run gap (external, not diagnosed) | +| 2026-08-22 | — | **no-run gap** — no branch/PR/issue exists for this date | — | — | no | — | — | — | no-run gap (external, not diagnosed) | +| 2026-08-23 | — | **no-run gap** — no branch/PR/issue exists for this date | — | — | no | — | — | — | no-run gap (external, not diagnosed) | | 2026-08-24 | swarm | weightedConsensus() trust weights now reach raft/byzantine/gossip vote tally (was architecturally unreachable — no field existed to carry them); [0,1] clamp preserves BFT safety margin | #3085 | #3086 | yes | ACCEPT | gossip flip: flat approved=false → weighted approved=true (0.905 vs 0.50); 226/226 tests, 0 regressions; adversarial critic CONFIRMED-WITH-CAVEATS | 08d44c58... | MERGED 2026-09-14 | | 2026-08-25 | performance | HNSW product-quantization: productQuantizeDistance() correctly implemented but zero callers — generic distance ran on raw PQ centroid indices, numerically meaningless; now dispatched, with a disclosed residual isValidPQEncoding() plausibility-heuristic caveat | #3093 | #3094 | yes | ACCEPT | recall@10 0.097 (pre-fix) → 0.270 (post-fix) on deterministic synthetic corpus, ~2.8x; 461/462 tests, 0 regressions | 300da158... | awaiting final CI/merge as of 2026-09-14 | -| 2026-08-26 | security | authorizeMcpTool() trusted unsigned caller identity (ASI07) — ADR-377 verifyInvocationToken bound into live chokepoint | #3102 | #3103 | yes | ACCEPT-with-caveats | see issue #3102 | n/a (backfill) | unmerged draft as of 09-05 | -| 2026-08-27 | intelligence | distillLearning() EWC gate read 1 of 384 Fisher dims via length-collapsed getPenalty() call; rewired to computeConfidencePenalty()/updateFisherFromConfidences() | #3109 | #3110 | yes | ACCEPT-scoped | 152/152 tests (+4 new), baseline-fails/candidate-passes via stash isolation | 89946458... | unmerged draft as of 09-05 | -| 2026-08-28 | memory | HybridBackend's dead `weights` field now drives real weighted-RRF fusion (was computed, never read by merge helpers) | #3118 | #3119 | yes | ACCEPT | see issue #3118 | n/a (backfill) | unmerged draft as of 09-05 | +| 2026-08-26 | security | authorizeMcpTool() trusted unsigned caller identity (ASI07) — ADR-377 verifyInvocationToken bound into live chokepoint | #3102 | #3103 | yes | ACCEPT-with-caveats | see issue #3102 | fd944eae... | awaiting CI as of 2026-09-14 | +| 2026-08-27 | intelligence | distillLearning() EWC gate read 1 of 384 Fisher dims via length-collapsed getPenalty() call; rewired to computeConfidencePenalty()/updateFisherFromConfidences() | #3109 | #3110 | yes | ACCEPT-scoped | 153/153 memory-ruvector-deep tests (+5 new incl. acceptance test); regression-guard test fails on baseline, passes on candidate | 0df6bc4f... | awaiting CI as of 2026-09-14 | +| 2026-08-28 | memory | `HybridBackend.queryHybrid()`'s dead `weights` field (computed, never consumed by combineUnion) replaced with real weighted-RRF fusion (`combineWeighted`), matching Qdrant/Weaviate/Azure AI Search's weighted-RRF pattern | #3118 | #3119 | yes | ACCEPT | 461/462 suite green (1 pre-existing unrelated env failure, confirmed identical on baseline); weight-sensitivity test fails on baseline, passes on candidate (git-stash isolated); post-review: added finite/non-negative weight validation + deterministic id-based tie-break | 47e2f669... | still open as of 2026-09-14 — INCONCLUSIVE per review pending #3095/#3101 dependency repair, candidate evidence itself reaffirmed strong; 08-20..23 no-run gap still undiagnosed | | 2026-08-29 | swarm | TopologyManager.rebalanceHybrid() one-directional adjacency bug fixed | #3122 | #3123 | yes | ACCEPT | see issue #3122 | n/a (backfill) | unmerged draft as of 09-05 | | 2026-08-30 | performance | unwired diskann-backend.ts removed + stale 150x-12,500x HNSW claim in generated CLAUDE.md corrected | #3129 | #3130 | yes | ACCEPT | see issue #3129 | n/a (backfill) | unmerged draft as of 09-05 | | 2026-08-31 | security | MCP governance policy (.harness/mcp-policy.json) was never enforced at runtime — opt-in enforcement wired in | #3138 | #3139 | yes | ACCEPT | see issue #3138 | n/a (backfill) | unmerged draft as of 09-05 | diff --git a/docs/dream-cycle/dream-gist-2026-08-28.md b/docs/dream-cycle/dream-gist-2026-08-28.md new file mode 100644 index 0000000000..5a71c3863c --- /dev/null +++ b/docs/dream-cycle/dream-gist-2026-08-28.md @@ -0,0 +1,65 @@ +# Memory SOTA Report — 2026-08-28 + +TL;DR: Tonight's memory deep-dive re-verified yesterday's (2026-08-27) flagged finding — `HybridBackend.queryHybrid()`'s `weights` field (`v3/@claude-flow/memory/src/hybrid-backend.ts`) is computed and then discarded, so `combineUnion` was a plain set-union with zero score math — and found it's a *stronger* dead-API claim than reported: `weights` is never populated by any caller in the repo (internal or example), not merely unconsumed once set. Deep research also surfaced a materially different, higher-value finding — a *second*, real, tested, RRF+MMR three-arm hybrid-search engine (`controller-registry.ts`'s `hybridSearch` controller, ADR-125/147) already exists and has **zero production callers** — but wiring it into the CLI's `memory_search` MCP tool turned out, on inspection, to require bridging two architecturally separate memory subsystems (the tool's real path uses a lightweight `memory-initializer.ts` bridge, not `ControllerRegistry`/`AgentDBAdapter` at all), which is a larger, riskier change than initially scored. Tonight's shipped candidate is the smaller, self-contained fix: `combineWeighted`, a weighted reciprocal-rank-fusion (RRF) combiner that makes `weights` actually control result ordering for the `'union'` strategy, matching the fusion shape documented by Qdrant/Weaviate/Azure AI Search. Evaluated via deterministic Vitest (zero LLM calls), baseline-isolated via `git stash`: the new weight-sensitivity test fails on baseline and passes on candidate; full package suite 461/462 green (1 pre-existing, unrelated environmental failure, confirmed identical on both baseline and candidate). + +## What's New in 2026 + +| Finding | Source | Confidence | +|---|---|---| +| RRF (`1/(rank+k)`, k=60 standard) sidesteps score-scale incomparability (BM25 unbounded vs. cosine [-1,1]) by fusing on rank, not raw score | OpenSearch blog, updated 2025-11-11 | A | +| Production hybrid systems apply per-arm weights as a *multiplier on that arm's RRF contribution before summation* ("weighted-RRF"), not as an α-blend of raw scores | Azure AI Search docs, ms.date 2026-06-08 | A | +| Six of eight surveyed vector/search systems (Qdrant, Weaviate, Milvus-WeightedRanker, ES `linear` retriever, OpenSearch, Azure) expose a caller-tunable weight that provably reorders results; the other two (plain RRF retrievers, Mem0) withhold it as a *deliberate* design choice, not an oversight | Live competitor research, 2026-08-28 (official docs, all fetched today) | A | +| Excessive fusion complexity doesn't reliably improve agent-memory retrieval; conservative, localized updates beat aggressive summarization/over-engineering | arXiv:2606.24775 (Jun 2026) | B | +| MMR diversity reranking in Ruflo's own `smart-retrieval.ts` uses a token-Jaccard proxy for the diversity term, not embedding-cosine similarity, despite candidates already carrying embeddings — an explicit code-comment admission, not a hidden bug | Direct code read, `smart-retrieval.ts:241` | A | + +## Ruflo Current Capability + +`hybrid-backend.ts`'s `HybridQuery.weights` (declared L139-143) is read once (L413: `query.weights || {semantic:0.7, structured:0.3}`) and then never referenced again — `combineUnion`/`combineIntersection`/`combineSemanticFirst`/`combineStructuredFirst` are pure set/list operations, no numeric fusion. `querySemantic()` also drops `SearchResult.score` when it flattens to `MemoryEntry[]` (which has no score field at all), so no score survived to the merge step even if it were consulted. Separately — and this is the night's more consequential discovery — `controller-registry.ts`'s `hybridSearch` controller (dense + BM25 + entity arms, real `applyRRF`+`applyMMR` fusion, tested at `graceful-retrieval.test.ts:102-281`) is fully built and correct, but is reached by **zero production call sites** anywhere in `cli/src` or `hooks/src`; the CLI's actual `memory_search` MCP tool path never constructs a `ControllerRegistry` at all. `HybridBackend` (this fix's target) is confirmed live in production: `database-provider.ts:301-302` wires it as `createDatabase`'s default backend. + +## Competitor Comparison + +| System | Fusion mechanism | Caller-tunable weight? | Grade | +|---|---|---|---| +| Qdrant | `fusion: rrf` / `dbsf`, `query.rrf.weights` per prefetch | Yes — direct RRF-sum multiplier | A | +| Weaviate | `relativeScoreFusion` (min-max normalize + weighted sum) | Yes — `alpha` param | A | +| Milvus | `WeightedRanker` (explicit per-field weights) vs. unweighted `RRFRanker` | Yes, when WeightedRanker chosen | A | +| Elasticsearch/OpenSearch | `linear` retriever / `normalization-processor`, explicit `weight`/`weights` | Yes for both | A | +| Azure AI Search | RRF with per-query `weight` multiplier before the RRF sum | Yes (RRF `k` itself is not) | A | +| LanceDB | Default `RRFReranker` (rank-only); custom `Reranker` subclass hook for weighting | Partial — mechanism exists, not first-class | B | +| Mem0 (agent-memory) | Semantic + BM25 + entity fusion, internal/automatic | No — deliberate design choice, not exposed | B | +| Ruflo `HybridBackend` (before tonight) | `weights` field declared, computed, never consumed; pure set-union | No — dead code | — | + +Every competitor surveyed ships a real fusion mechanism; Ruflo's `hybrid-backend.ts` sat below the whole field — worse than even Mem0's "no knob" case, since Mem0 at least fuses on score. + +## Hypothesis + +Given `HybridBackend.queryHybrid()`'s `'union'` combine strategy, when `combineUnion` (plain set-union, `weights` unused) is replaced with `combineWeighted` — a weighted-RRF fusion (`weights.semantic`/`weights.structured` as per-arm multipliers on each arm's `1/(k+rank+1)` contribution, k=60, matching Ruflo's own `smart-retrieval.ts` convention) — then result ordering for `'union'`-strategy hybrid queries should become provably weight-sensitive (increasing `weights.semantic` promotes semantically-favored entries; increasing `weights.structured` promotes structurally-favored entries), subject to: (1) `'intersection'`/`'semantic-first'`/`'structured-first'` strategies unchanged; (2) all existing tests green; (3) $0 evaluation cost. Frozen before evaluation; not modified after seeing results. + +## Benchmarks / Evaluation + +**evaluated: accepted.** Real evaluator: Vitest, deterministic, zero LLM calls. Candidate: `v3/@claude-flow/memory/src/hybrid-backend.ts` (net +59/-24 lines: new `querySemanticScored` preserving score, new `combineWeighted`, dead `combineUnion` removed since it became unreachable) + 3 new tests in `hybrid-backend.test.ts` (+83 lines). Two entries with deliberately opposed per-arm ranks (one closer semantically, the other more-recently-stored so it ranks first under SQLite's `ORDER BY created_at DESC`) let weight configuration flip which one tops the fused result — this is the discriminating test. + +Baseline-isolated via `git stash` of only `hybrid-backend.ts` (test file kept): the structured-weight-dominant test **fails** on baseline (`fusion-sem-fav` wins regardless of weights, since plain concatenation always orders semantic arm first) and **passes** on candidate. Disclosed honestly: of the 3 new tests, only this one discriminates baseline from candidate — the semantic-weight-dominant test and the full-union regression test both happen to pass on baseline too (baseline's plain concatenation already puts semantic first / already returns the full union), so they're regression guards, not proof-of-fix on their own. Full `@claude-flow/memory` suite: 461/462 passing, both with and without the candidate — the 1 failure (`auto-memory-bridge.test.ts`, a chmod-based read-only-file test that can't enforce permissions under a root-owned sandbox) is confirmed pre-existing and unrelated, same documented class as 2026-08-15/18/19's nights. `tsc --noEmit`: zero errors. + +## Darwin Results + +Skipped — scope mismatch, same class as most recent nights (`npx ruvector harness darwin --help` confirms the real interface tunes continuous/categorical parameters against an LLM-scored benchmark corpus). This is a correctness/wiring fix (a field either gets consumed or it doesn't) with no gold-labeled retrieval corpus to search a fitness gradient over — not a Darwin-shaped candidate. + +## SOTA Proof & Witness + +| Field | Value | +|---|---| +| Session commit | `d33ef4bf8ab27a8f9ef08352c9c293b53312a861` | +| Gist SHA-256 (pre-witness content) | `35bf1d00e8647db8b2fb0842c5ecdda4737b9a21b0c4a584461fb1db6edf5278` | +| Witness stamp | `47e2f669e6124a686f8dcbe11828339fb8728009ebc3dad8736e8ddf82628510` | +| Evaluation receipt | deterministic Vitest, `git stash`-isolated baseline vs. candidate (see Evaluation section) | +| Flywheel evidence | none signed — deterministic-test evidence, same class as every accepted night since 2026-08-18 | +| Darwin lineage | none — skipped, scope mismatch (see Darwin Results) | + +Verifier procedure: fetch `docs/dream-cycle/dream-gist-2026-08-28.md` from this branch as it existed before this table was filled in (this section replaced a one-line placeholder sentence), SHA-256 it, concatenate with the session commit above, SHA-256 again — result must equal the witness stamp. + +## Recommended Next Steps + +1. **Wire the real `hybridSearch` (RRF+MMR, dense+BM25+entity) controller into a reachable production surface** — it's fully built and tested but has zero callers. Requires either extending `memory_search`'s MCP tool to construct a compatible `ControllerRegistry`/backend, or bridging `memory-initializer.ts`'s lighter path to duck-type as `HybridCapableBackend`. Scored higher (4.30 vs. 4.05) than tonight's fix in tonight's candidate ranking but was deliberately not selected tonight — it's a larger, cross-subsystem change that needs its own dedicated night, not a same-night addition to a small fix. +2. **Automation-scan finding (see issue)**: the ledger-append backlog (STEP 25 running correctly every night, but rows stuck on unmerged draft PRs) needs a CI backlog-guard, not another manual backfill — concrete workflow sketch in the issue. +3. **Replace `smart-retrieval.ts`'s MMR token-Jaccard diversity proxy with embedding-cosine similarity** — candidates already carry embeddings upstream; the proxy is an explicit, admitted approximation (`smart-retrieval.ts:241`), self-contained fix for a future night. diff --git a/v3/@claude-flow/memory/src/hybrid-backend.test.ts b/v3/@claude-flow/memory/src/hybrid-backend.test.ts index 33ff95656d..228fb23494 100644 --- a/v3/@claude-flow/memory/src/hybrid-backend.test.ts +++ b/v3/@claude-flow/memory/src/hybrid-backend.test.ts @@ -237,6 +237,148 @@ describe('HybridBackend - ADR-009', () => { }); }); + describe('Weighted Union Fusion (Dream Cycle 2026-08-28)', () => { + // Before this fix, `queryHybrid`'s 'union' strategy computed `weights` + // and then never consumed it (combineUnion was a plain dedup, no score + // math) — this suite proves `weights` now actually controls ordering. + const QUERY_CONTENT = 'authentication token verification'; + // Populated in beforeEach — generateMemoryId() is `mem__ + // `, so these two ids' relative order is NOT fixed run-to-run + // (same millisecond is common; the random suffix then decides it). The + // tie-break test below computes its expectation from these actual ids + // rather than assuming id-order matches key-order. + let semFavId = ''; + let structFavId = ''; + + beforeEach(async () => { + // Explicit, distinct `createdAt` values (rather than relying on wall-clock + // gaps between two `store()` calls, which can tie at millisecond + // resolution) so SQLiteBackend's `ORDER BY created_at DESC` deterministically + // ranks 'fusion-struct-fav' first in structured results. + // 'fusion-sem-fav' content closely matches QUERY_CONTENT → ranks BEFORE + // 'struct-fav' in semantic (AgentDB cosine) results. Opposite ranks in + // each arm is what makes the weight-sensitivity assertions below + // meaningful (same-order ranks in both arms couldn't distinguish a real + // weighted fusion from any dedup that happens to prefer one arm). + const semFav = createDefaultEntry({ + key: 'fusion-sem-fav', + content: 'authentication token verification flow', + namespace: 'fusion-test', + }); + semFav.createdAt = 1000; + await backend.store(semFav); + semFavId = semFav.id; + + // Content deliberately shares nothing with QUERY_CONTENT → ranks + // AFTER 'fusion-sem-fav' in semantic results. + const structFav = createDefaultEntry({ + key: 'fusion-struct-fav', + content: 'zzz totally unrelated filler xyz 12345', + namespace: 'fusion-test', + }); + structFav.createdAt = 2000; // strictly newer → ranks FIRST under created_at DESC + await backend.store(structFav); + structFavId = structFav.id; + }); + + async function fusedTopKey(weights: { semantic: number; structured: number }) { + const results = await backend.queryHybrid({ + semantic: { + content: QUERY_CONTENT, + k: 5, + threshold: 0.01, // low: let both entries pass so both arms are populated + filters: { namespace: 'fusion-test' }, + }, + structured: { + namespace: 'fusion-test', + keyPrefix: 'fusion-', + limit: 5, + }, + combineStrategy: 'union', + weights, + }); + expect(results.length).toBe(2); // sanity: both entries present in the fused union + return results[0].key; + } + + it('heavily semantic-weighted fusion ranks the semantically-closer entry first', async () => { + expect(await fusedTopKey({ semantic: 1, structured: 0.01 })).toBe('fusion-sem-fav'); + }); + + it('heavily structured-weighted fusion ranks the more-recently-stored entry first', async () => { + expect(await fusedTopKey({ semantic: 0.01, structured: 1 })).toBe('fusion-struct-fav'); + }); + + it('still returns the full deduped union when weights are omitted (regression guard)', async () => { + const results = await backend.queryHybrid({ + semantic: { + content: QUERY_CONTENT, + k: 5, + threshold: 0.01, + filters: { namespace: 'fusion-test' }, + }, + structured: { + namespace: 'fusion-test', + keyPrefix: 'fusion-', + limit: 5, + }, + combineStrategy: 'union', + }); + const keys = results.map((r) => r.key).sort(); + expect(keys).toEqual(['fusion-sem-fav', 'fusion-struct-fav']); + }); + + // Review (PR #3119): malformed weights previously had undefined ranking + // semantics (NaN poisons the sort comparator, negative/Infinity invert + // or swamp the intended ordering). Each malformed component now falls + // back to its own default (0.7 semantic / 0.3 structured) — same + // ranking as the semantic-heavy default, since 0.7 > 0.3. + it.each([ + ['NaN semantic weight', { semantic: NaN, structured: 0.3 }], + ['negative semantic weight', { semantic: -5, structured: 0.3 }], + ['Infinity structured weight', { semantic: 0.7, structured: Infinity }], + ])('falls back to default weighting for a malformed weight (%s)', async (_label, weights) => { + expect(await fusedTopKey(weights)).toBe('fusion-sem-fav'); + }); + + it('breaks exact rrf ties deterministically by entry id, not iteration order', async () => { + // weights of 0/0 make every entry's rrf contribution 0 — a genuine + // tie between both entries, not just a coincidentally-close score. + const results = await backend.queryHybrid({ + semantic: { + content: QUERY_CONTENT, + k: 5, + threshold: 0.01, + filters: { namespace: 'fusion-test' }, + }, + structured: { + namespace: 'fusion-test', + keyPrefix: 'fusion-', + limit: 5, + }, + combineStrategy: 'union', + weights: { semantic: 0, structured: 0 }, + }); + // Review: the tie-break is `entry.id.localeCompare`, not `entry.key` + // — generateMemoryId() is `mem__`, so id-order + // has no fixed relationship to key-order (the two entries are often + // created in the same millisecond; the random suffix then decides + // it). Asserting a hardcoded key-alphabetical order here made this + // test flake on whichever id happened to sort backwards from its + // key. Compute the expectation from the actual ids captured in + // beforeEach instead, so the assertion matches what the code's + // documented contract (id order) actually guarantees. + const idToKey: Record = { + [semFavId]: 'fusion-sem-fav', + [structFavId]: 'fusion-struct-fav', + }; + const expectedKeys = [semFavId, structFavId] + .sort((a, b) => a.localeCompare(b)) + .map((id) => idToKey[id]); + expect(results.map((r) => r.key)).toEqual(expectedKeys); + }); + }); + describe('CRUD Operations', () => { let testEntry: MemoryEntry; diff --git a/v3/@claude-flow/memory/src/hybrid-backend.ts b/v3/@claude-flow/memory/src/hybrid-backend.ts index d2155319ab..9270377765 100644 --- a/v3/@claude-flow/memory/src/hybrid-backend.ts +++ b/v3/@claude-flow/memory/src/hybrid-backend.ts @@ -361,6 +361,21 @@ export class HybridBackend extends EventEmitter implements IMemoryBackend { * Routes to AgentDB for HNSW-based vector search */ async querySemantic(query: SemanticQuery): Promise { + const scored = await this.querySemanticScored(query); + return scored.map((r) => r.entry); + } + + /** + * Semantic vector search, preserving each result's similarity score. + * + * `querySemantic()` above flattens to `MemoryEntry[]`, discarding score — + * fine for callers that only need entries, but `queryHybrid()`'s weighted + * fusion (`combineWeighted`) needs the score to survive, so it calls this + * variant directly instead. + */ + private async querySemanticScored( + query: SemanticQuery + ): Promise> { this.stats.agentdbQueries++; let embedding = query.embedding; @@ -380,26 +395,26 @@ export class HybridBackend extends EventEmitter implements IMemoryBackend { filters: query.filters as MemoryQuery | undefined, }); - let entries = searchResults.map((r) => r.entry); + let scored = searchResults.map((r) => ({ entry: r.entry, score: r.score })); // Apply tag/namespace/type filters that AgentDB may not enforce if (query.filters) { const f = query.filters as Record; if (f.tags && Array.isArray(f.tags)) { const requiredTags = f.tags as string[]; - entries = entries.filter((e) => - requiredTags.every((t) => e.tags.includes(t)) + scored = scored.filter((r) => + requiredTags.every((t) => r.entry.tags.includes(t)) ); } if (f.namespace && typeof f.namespace === 'string') { - entries = entries.filter((e) => e.namespace === f.namespace); + scored = scored.filter((r) => r.entry.namespace === f.namespace); } if (f.type && typeof f.type === 'string' && f.type !== 'semantic') { - entries = entries.filter((e) => e.type === f.type); + scored = scored.filter((r) => r.entry.type === f.type); } } - return entries.slice(0, query.k || 10); + return scored.slice(0, query.k || 10); } /** @@ -412,16 +427,20 @@ export class HybridBackend extends EventEmitter implements IMemoryBackend { const strategy = query.combineStrategy || 'semantic-first'; const weights = query.weights || { semantic: 0.7, structured: 0.3 }; - // Execute both queries in parallel - const [semanticResults, structuredResults] = await Promise.all([ - this.querySemantic(query.semantic), + // Execute both queries in parallel. The scored semantic variant is + // needed for weighted fusion (combineWeighted); other strategies only + // need plain entries, derived from the same result rather than + // querying AgentDB twice. + const [semanticScored, structuredResults] = await Promise.all([ + this.querySemanticScored(query.semantic), query.structured ? this.queryStructured(query.structured) : Promise.resolve([]), ]); + const semanticResults = semanticScored.map((r) => r.entry); // Combine results based on strategy switch (strategy) { case 'union': - return this.combineUnion(semanticResults, structuredResults); + return this.combineWeighted(semanticScored, structuredResults, weights); case 'intersection': return this.combineIntersection(semanticResults, structuredResults); @@ -433,7 +452,7 @@ export class HybridBackend extends EventEmitter implements IMemoryBackend { return this.combineStructuredFirst(semanticResults, structuredResults); default: - return this.combineUnion(semanticResults, structuredResults); + return this.combineWeighted(semanticScored, structuredResults, weights); } } @@ -654,23 +673,58 @@ export class HybridBackend extends EventEmitter implements IMemoryBackend { } /** - * Combine results using union (all unique results) + * Weighted union: fuses semantic + structured results via reciprocal-rank + * fusion (RRF), each arm scaled by its `weights` multiplier before + * summation — the same shape documented for Azure AI Search/Qdrant/ + * Weaviate (a per-arm weight applied to that arm's RRF contribution, + * not a raw-score blend, since SQLite's structured results carry no + * similarity score comparable to AgentDB's). Dream Cycle 2026-08-28: + * `weights` used to be computed by the caller and discarded — this is + * the fusion that actually consumes it. */ - private combineUnion( - semanticResults: MemoryEntry[], - structuredResults: MemoryEntry[] + private combineWeighted( + semanticScored: Array<{ entry: MemoryEntry; score: number }>, + structuredResults: MemoryEntry[], + weights: { semantic: number; structured: number } ): MemoryEntry[] { - const seen = new Set(); - const combined: MemoryEntry[] = []; + const RRF_K = 60; // matches smart-retrieval.ts's applyRRF default - for (const entry of [...semanticResults, ...structuredResults]) { - if (!seen.has(entry.id)) { - seen.add(entry.id); - combined.push(entry); - } - } + // Review (PR #3119): an unvalidated malformed weight (NaN, negative, + // Infinity) gives RRF undefined ranking semantics — NaN poisons every + // sum it touches (breaking the sort comparator below), a negative + // weight silently inverts that arm's intended preference, and + // Infinity swamps the other arm outright. Sanitize per-component: any + // non-finite or negative value falls back to that component's own + // documented default rather than failing the whole hybrid query (this + // mirrors `weights` itself already being optional with a default — + // a malformed field degrades the same way a missing one does). + const safeWeight = (value: number, fallback: number): number => + Number.isFinite(value) && value >= 0 ? value : fallback; + const safeWeights = { + semantic: safeWeight(weights.semantic, 0.7), + structured: safeWeight(weights.structured, 0.3), + }; - return combined; + const fused = new Map(); + + semanticScored.forEach(({ entry }, rank) => { + const contribution = safeWeights.semantic * (1 / (RRF_K + rank + 1)); + const existing = fused.get(entry.id); + fused.set(entry.id, { entry, rrf: (existing?.rrf ?? 0) + contribution }); + }); + + structuredResults.forEach((entry, rank) => { + const contribution = safeWeights.structured * (1 / (RRF_K + rank + 1)); + const existing = fused.get(entry.id); + fused.set(entry.id, { entry, rrf: (existing?.rrf ?? 0) + contribution }); + }); + + // Review (PR #3119): break rrf ties deterministically by entry.id + // rather than relying on Array.sort's stability + Map insertion order + // (semantic-arm-first) as an undocumented accident of iteration order. + return Array.from(fused.values()) + .sort((a, b) => b.rrf - a.rrf || a.entry.id.localeCompare(b.entry.id)) + .map((r) => r.entry); } /**