mlsanityData to 14 Oct 2026

From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents

arXiv 2604.20006 ↗First posted 21 Apr 2026cs.CLcited by 36

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Personalized agents that interact with users over long periods must maintain persistent memory across sessions and update it as circumstances change. However, existing benchmarks predominantly frame long-term memory evaluation as fact retrieval from past conversations, providing limited insight into agents' ability to consolidate memory over time or handle frequent knowledge updates. We introduce Memora, a long-term memory benchmark spanning weeks to months long user conversations. The benchmark evaluates three memory-grounded tasks: remembering, reasoning, and recommending. To ensure data quality, we employ automated memory-grounding checks and human evaluation. We further introduce Forgetting-Aware Memory Accuracy (FAMA), a metric that penalizes reliance on obsolete or invalidated memory when evaluating long-term memory. Evaluations of four LLMs and six memory agents reveal frequent reuse of invalid memories and failures to reconcile evolving memories. Memory agents offer marginal improvements, exposing shortcomings in long-term memory for personalized agents.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 22 of its 52 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

22 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions7 Jul
  2. 2025A-MEM: Agentic Memory for LLM Agents17 Feb
  3. 2024MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants30 Sep
  4. 2024Evaluating Very Long-Term Conversational Memory of LLM Agents27 Feb

Other references · 18

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  7. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  8. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  9. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  10. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
  11. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 152
  12. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 127
6 more
  1. 2025From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMscited by 124
  2. 2022Long Time No See! Open-Domain Conversation with Long-Term Persona Memorycited by 108
  3. 2024Controllable Text Generation for Large Language Models: A Surveycited by 57
  4. 2023Personalized Large Language Model Assistant with Evolving Conditional Memorycited by 25
  5. 2024DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agentscited by 20
  6. 2024GPTutor: Great Personalized Tutor with Large Language Models for Personalized Learning Content Generationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 36.

36 papers.

Sort:

By area:

  1. 2026Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agentscited by 20
  2. 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 13
  3. 2026TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memorycited by 8
  4. 2026When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responsescited by 7
  5. 2026MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recoverycited by 6
  6. 2026Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systemscited by 6
  7. 2026Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgentscited by 5
  8. 2026FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environmentscited by 4
  9. 2026MemTrace: Probing What Final Accuracy Misses in Long-Term Memorycited by 4
  10. 2026Mi-Memory: A Lifecycle Memory Framework for Personal AIcited by 3
  11. 2026When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memorycited by 3
  12. 2026MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcarecited by 2
24 more
  1. 2026AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agentscited by 2
  2. 2026Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agentscited by 2
  3. 2026VoiceMem: Streaming Dual-Brain Memory for Real-Time Interactioncited by 2
  4. 2026ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Modelscited by 1
  5. 2026When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflictcited by 1
  6. 2026MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Usecited by 1
  7. 2026CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agentscited by 1
  8. 2026Memory Consolidation Flattens the Temporal Shape of User Factscited by 1
  9. 2026ConvMemory v3: A Validity Context Layer for Conversational Memory via Target-Conditioned Relation Verificationcited by 0
  10. 2026Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agentscited by 0
  11. 2026Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistantscited by 0
  12. 2026MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scalecited by 0
  13. 2026FinLifeBench: Exhaustive Life-Event History and Financial-State Reconstruction from Longitudinal Banking Dialoguecited by 0
  14. 2026TWIST: A Proposed Benchmark for Intervention Quality in Conversational Memory, with a Human-Validated Draft-Alignmentcited by 0
  15. 2026When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agentscited by 0
  16. 2026MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimizationcited by 0
  17. 2026Authority Before Utility: Non-Compensatory Control for Persistent LLM Memorycited by 0
  18. 2026Personalized State-Transition-Aware Memory for Clinical Agentscited by 0
  19. 2026Evaluating Persistent Calibration under Evolving Model Knowledgecited by 0
  20. 2026When Context Changes: Understanding Update Failures in LLMscited by 0
  21. 2026Knowing the Store: What a Memory Backend Must Write Down Before an Agent Can Read Itcited by 0
  22. 2026StateWise: Diagnosing and Repairing Persistent Operational State Before Agent Actionscited by 0
  23. 2026Vela: User Behavior as Supervision Sparks LLM Intelligencecited by 0
  24. 2026Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations
  2. 2026AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents
  3. 2026Evaluating Memory Structure in LLM Agents
  4. 2026DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users
  5. 2026When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents
  6. 2026LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
  7. 2026From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space
  8. 2026RaMem: Contextual Reinstatement for Long-term Agentic Memory