Entity-Collision: A Stratified Protocol for Attributing Retrieval Lift in Agent Memory
Not on the map. Why?
Abstract (arXiv)
End-to-end agent-memory benchmarks report a single hit@k per retriever, confounding lexical leakage (uncontrolled query/gold/distractor entity overlap) with tag-mixing (preferences, services, tools averaged together). We propose entity-collision, a system-agnostic protocol that pins the BM25 floor by construction -- every distractor shares the answer's entity tokens -- and stratifies queries by discriminator tag, so any lift over BM25 is attributable to the embedder. Applied to an open-source agent-memory testbed across 5 tags x 3 embedders x 5 collision degrees with paired-bootstrap 95% CIs, the protocol reveals a two-axis pattern: a 256-d hash trigram helps only on closed-vocabulary lexical tags at deep collision; MiniLM-384 dominates both axes; and a 2.7x-parameter BGE-large does not uniformly improve on MiniLM -- it wins on intent-style queries but loses on lexical ones. Encoder capacity alone is not the binding constraint. The synthetic intent-tag null replicates on LongMemEval (n=500) as a single-session-preference recall cliff. Adaptive vector-weight routing on LoCoMo is a measured null: 11.7pp of oracle headroom exists, but no signal we tested recovers it. All 26 result tables and 37 reproduce scripts are version-controlled and verified by a public registry; the protocol is exercised on a deterministically governed memory testbed (event-sourced decision log, DAG-state-machine schema lifecycle) so every reported CI is reproducible byte-for-byte from the ingest stream.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 36 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
19 of 36 references.
References · 19
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
- 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
- 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
- 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 360
- 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
7 more
- 2020Overview of the TREC 2019 deep learning trackcited by 282
- 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
- 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
- 2021Overview of the TREC 2020 deep learning trackcited by 193
- 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
- 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 80
- 2026Governed Memory: A Production Architecture for Multi-Agent Workflowscited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.