# From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents

arXiv: https://arxiv.org/abs/2604.20006 · HTML: https://mlsanity.com/paper/2604.20006/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 52 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 22 of 22)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [∞Bench: Extending Long Context Evaluation Beyond 100K Tokens](https://mlsanity.com/paper/2402.13718/) (2024)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale](https://mlsanity.com/paper/2504.14225/) (2025)
- [Benchmarking Complex Instruction-Following with Multiple Constraints Composition](https://mlsanity.com/paper/2407.03978/) (2024)
- [Context Length Alone Hurts LLM Performance Despite Perfect Retrieval](https://mlsanity.com/paper/2510.05381/) (2025)
- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [Long Time No See! Open-Domain Conversation with Long-Term Persona Memory](https://mlsanity.com/paper/2203.05797/) (2022)
- [Controllable Text Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2408.12599/) (2024)
- [Personalized Large Language Model Assistant with Evolving Conditional Memory](https://mlsanity.com/paper/2312.17257/) (2023)
- [MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistants](https://mlsanity.com/paper/2409.20163/) (2024)
- [DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agents](https://mlsanity.com/paper/2406.13144/) (2024)
- [GPTutor: Great Personalized Tutor with Large Language Models for Personalized Learning Content Generation](https://mlsanity.com/paper/2407.09484/) (2024)

## Built on by (most cited first, 30 of 36)

- [Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents](https://mlsanity.com/paper/2606.27472/) (2026)
- [GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents](https://mlsanity.com/paper/2606.18829/) (2026)
- [TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memory](https://mlsanity.com/paper/2606.06240/) (2026)
- [When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responses](https://mlsanity.com/paper/2608.01619/) (2026)
- [MemAudit: Auditing Long-Term Agent Memory via Hidden User-State Recovery](https://mlsanity.com/paper/2606.24595/) (2026)
- [Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems](https://mlsanity.com/paper/2607.14611/) (2026)
- [Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgents](https://mlsanity.com/paper/2606.30306/) (2026)
- [FAMA: Failure-Aware Meta-Agentic Framework for Open-Source LLMs in Interactive Tool Use Environments](https://mlsanity.com/paper/2604.25135/) (2026)
- [MemTrace: Probing What Final Accuracy Misses in Long-Term Memory](https://mlsanity.com/paper/2606.17328/) (2026)
- [Mi-Memory: A Lifecycle Memory Framework for Personal AI](https://mlsanity.com/paper/2607.18975/) (2026)
- [When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory](https://mlsanity.com/paper/2608.25553/) (2026)
- [MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare](https://mlsanity.com/paper/2605.11814/) (2026)
- [AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents](https://mlsanity.com/paper/2606.21144/) (2026)
- [Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents](https://mlsanity.com/paper/2608.19564/) (2026)
- [VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction](https://mlsanity.com/paper/2608.26005/) (2026)
- [ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models](https://mlsanity.com/paper/2607.26455/) (2026)
- [When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict](https://mlsanity.com/paper/2608.13921/) (2026)
- [MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use](https://mlsanity.com/paper/2608.20202/) (2026)
- [CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents](https://mlsanity.com/paper/2608.30147/) (2026)
- [Memory Consolidation Flattens the Temporal Shape of User Facts](https://mlsanity.com/paper/2609.36457/) (2026)
- [ConvMemory v3: A Validity Context Layer for Conversational Memory via Target-Conditioned Relation Verification](https://mlsanity.com/paper/2606.26753/) (2026)
- [Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Self-Improving Personal Agents](https://mlsanity.com/paper/2607.10526/) (2026)
- [Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants](https://mlsanity.com/paper/2607.26611/) (2026)
- [MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale](https://mlsanity.com/paper/2608.02613/) (2026)
- [FinLifeBench: Exhaustive Life-Event History and Financial-State Reconstruction from Longitudinal Banking Dialogue](https://mlsanity.com/paper/2609.01198/) (2026)
- [TWIST: A Proposed Benchmark for Intervention Quality in Conversational Memory, with a Human-Validated Draft-Alignment](https://mlsanity.com/paper/2609.28575/) (2026)
- [When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agents](https://mlsanity.com/paper/2609.32520/) (2026)
- [MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization](https://mlsanity.com/paper/2609.36435/) (2026)
- [Authority Before Utility: Non-Compensatory Control for Persistent LLM Memory](https://mlsanity.com/paper/2609.37474/) (2026)
- [Personalized State-Transition-Aware Memory for Clinical Agents](https://mlsanity.com/paper/2609.38490/) (2026)

## Abstract (arXiv)

Personalized agents that interact with users over long periods must maintain persistent memory across sessions and update it as circumstances change. However, existing benchmarks predominantly frame long-term memory evaluation as fact retrieval from past conversations, providing limited insight into agents' ability to consolidate memory over time or handle frequent knowledge updates. We introduce Memora, a long-term memory benchmark spanning weeks to months long user conversations. The benchmark evaluates three memory-grounded tasks: remembering, reasoning, and recommending. To ensure data quality, we employ automated memory-grounding checks and human evaluation. We further introduce Forgetting-Aware Memory Accuracy (FAMA), a metric that penalizes reliance on obsolete or invalidated memory when evaluating long-term memory. Evaluations of four LLMs and six memory agents reveal frequent reuse of invalid memories and failures to reconcile evolving memories. Memory agents offer marginal improvements, exposing shortcomings in long-term memory for personalized agents.