mlsanityData to 7 Oct 2026

Structured Belief State and the First Precision-Aware Benchmark for LLM Memory Retrieval

arXiv 2605.11325 ↗First posted 11 May 2026cs.IR cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Current LLM memory benchmarks evaluate answer quality rather than retrieval accuracy. Consequently, a system that dumps its entire belief store can achieve perfect recall and mask severe precision failures. We show this evaluation gap persists across multiple embedding models where similarity-based retrieval over domain-specific corpora inherently struggles to isolate target beliefs from semantically proximate ones. Furthermore, multi-turn topic drift compounds this retrieval noise while driving up latency and operational costs. To decouple retrieval quality from generative performance, we introduce PrecisionMemBench, an 89-case benchmark measuring precision, noise isolation, session latency, and belief mutability. We also present Tenure, a structured belief-store proxy that resolves scope and retrieval before inference and injects typed belief state as ambient instruction before the model sees the prompt, removing model-side discretion over whether memory is consulted. Evaluated across 13 configurations, Tenure achieves perfect retrieval passes across all active, non-session, and session test cases. In contrast, the baseline configurations fail to reach even half of the active passes, with precision scores clustering at 0.22 and below. Our results demonstrate that while current memory systems successfully store information, they fail to retrieve it cleanly; a structural vulnerability that traditional answer-quality benchmarks conceal.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

13 of 29 references.

References · 13

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
  4. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  5. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  6. 2025Zep: A Temporal Knowledge Graph Architecture for Agent Memorycited by 372
  7. 2024Nomic Embed: Training a Reproducible Long Context Text Embeddercited by 255
  8. 2023AnglE-optimized Text Embeddingscited by 98
  9. 2025Hindsight is 20/20: Building Agent Memory that Retains, Recalls, and Reflectscited by 40
  10. 2025Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoningcited by 17
  11. 2026Evaluating Memory Structure in LLM Agentscited by 12
  12. 2026Memori: A Persistent Memory Layer for Efficient, Context-Aware LLM Agentscited by 12
1 more
  1. 2026Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memorycited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.