mlsanityData to 14 Oct 2026

SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval

arXiv 2603.15599 ↗First posted 16 Mar 2026cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Recent conversational memory systems invest heavily in LLM-based structuring at ingestion time and learned retrieval policies at query time. We show that neither is necessary. SmartSearch retrieves from raw, unstructured conversation history using a fully deterministic pipeline: NER-weighted substring matching for recall, rule-based entity discovery for multi-hop expansion, and a CrossEncoder+ColBERT rank fusion stage -- the only learned component -- running on CPU in ~650ms. Oracle analysis on two benchmarks identifies a compilation bottleneck: retrieval recall reaches 98.6%, but without intelligent ranking only 22.5% of gold evidence survives truncation to the token budget. With score-adaptive truncation and no per-dataset tuning, SmartSearch achieves 93.5% on LoCoMo and 88.4% on LongMemEval-S, exceeding all known memory systems under the same evaluation protocol on both benchmarks while using 8.5x fewer tokens than full-context baselines.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 37 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

32 of 37 references.

References · 32

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  11. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
20 more
  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
  2. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  3. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  4. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  5. 2023Active Retrieval Augmented Generationcited by 516
  6. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 346
  7. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 316
  8. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  9. 2023Augmenting Language Models with Long-Term Memorycited by 189
  10. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  11. 2025Overcoming catastrophic forgetting in neural networkscited by 55
  12. 2026EverMemOS: A Self-Organizing Memory Operating System for Structured Long-Horizon Reasoningcited by 54
  13. 2022An Analysis of Fusion Functions for Hybrid Retrievalcited by 51
  14. 2025What Deserves Memory: Adaptive Memory Distillation for LLM Agentscited by 48
  15. 2026M2A: Multimodal Memory Agent with Dual-Layer Hybrid Memory for Long-Term Personalized Interactionscited by 25
  16. 2026Learning Query-Aware Budget-Tier Routing for Runtime Agent Memorycited by 13
  17. 2026Memora: A Harmonic Memory Representation Balancing Abstraction and Specificitycited by 11
  18. 2026Locas: Your Models are Principled Initializers of Locally-Supported Parametric Memoriescited by 8
  19. 2026Query-focused and Memory-aware Reranker for Long Context Processingcited by 6
  20. 2026TraceMem: Weaving Narrative Memory Schemata from User Conversational Tracescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026ByteRover: Agent-Native Memory Through LLM-Curated Hierarchical Contextcited by 3
  2. 2026Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankingscited by 3
  3. 2026Mnemon: Raw Records, Fast Judgments, Slow Thoughtscited by 2
  4. 2026When Does Selection Replace Extraction? A Pre-Registered Test of Agent Memory with a Typed Decision Modelcited by 1
  5. 2026What Eviction Destroys: A Restore-Counterfactual Audit of Forgetting in Agent Memorycited by 0