mlsanityData to 14 Oct 2026

Total Recall at What Cost? Benchmarking the Serving Cost of Agentic Memory Systems

arXiv 2608.11879 ↗First posted 12 Aug 2026cs.CL cs.IRcited by 1

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Long-running conversational agents increasingly rely on a memory system to avoid resending the whole conversation each turn, yet how much that costs to serve has received little systematic benchmarking. We compare three memory systems (Mem0, Hindsight, and Mastra Observational Memory) against two reference strategies -- a fixed-size rolling window and resubmitting the full transcript -- across two backbones and conversations of up to 400 turns, pairing every cost measurement with answer accuracy on 665 LoCoMo questions. First, a memory system's serving cost cannot be predicted from conversation length and message size alone: a regression that tracks the two reference strategies closely misses the memory systems by 18-69%, their cost driven instead by internal memory behavior. Second, a break-even analysis shows that whether -- and when -- a memory system becomes cheaper to serve than the full transcript is highly sensitive to the system and the backbone, from the first tens of turns for the cheapest to never within 400 turns for the most expensive. Third, no system wins on both axes: accuracy spans 21-54%, and the backbone choice drives cost as much as the memory system does.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 13 of its 27 references that are in the map. First posted on arXiv 12 Aug 2026.

Builds on

13 of 27 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents12 Jan
  2. 2025PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory7 Dec
  3. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory28 Apr
  4. 2025A-MEM: Agentic Memory for LLM Agents17 Feb
  5. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 8

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  5. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  6. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  7. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
  8. 2026Diffusion-Pretrained Dense and Contextual Embeddingscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents
  2. 2026MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
  3. 2026Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents
  4. 2026Reproducing LightMem: Naive RAG Is Just as Good for Memory Management ↗
  5. 2026Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents
  6. 2026MINDSET: Energy-based Schema Evolution for Long Conversational Agent Memory
  7. 2026Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents
  8. 2026BudgetBench: A Budget-Tiered Protocol and Pilot Harness for Memory Strategy Evaluation in Local Large Language Model Agents