# Useful Memories Become Faulty When Continuously Updated by LLMs

arXiv: https://arxiv.org/abs/2605.12978 · HTML: https://mlsanity.com/paper/2605.12978/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 13 May 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 9 of 9)

- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents](https://mlsanity.com/paper/2407.18901/) (2024)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain](https://mlsanity.com/paper/2305.07141/) (2023)
- [CLIN: A Continually Learning Language Agent for Rapid Task Adaptation and Generalization](https://mlsanity.com/paper/2310.10134/) (2023)
- [ARC-GEN: A Mimetic Procedural Benchmark Generator for the Abstraction and Reasoning Corpus](https://mlsanity.com/paper/2511.00162/) (2025)
- [PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning](https://mlsanity.com/paper/2601.11957/) (2026)

## Built on by (most cited first, 20 of 20)

- [TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memory](https://mlsanity.com/paper/2606.06240/) (2026)
- [From Faulty Memories to Corrected Actions: Dependency-Guided Rollback Repair for Memory-Augmented Agents](https://mlsanity.com/paper/2608.10502/) (2026)
- [Agent Memory Is a Surface for Endogenous Authorization Laundering](https://mlsanity.com/paper/2609.01836/) (2026)
- [Test-Time Learning with an Evolving Library](https://mlsanity.com/paper/2605.14477/) (2026)
- [Filesystem-Based Memory for LLM Agents: Organization, Evolution, and Sustainability](https://mlsanity.com/paper/2607.26637/) (2026)
- [Memory Reward Inflation in Self-Improving LLM Agents](https://mlsanity.com/paper/2608.00017/) (2026)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [Memory Contagion: Cross-Temporal Propagation of Evaluator Bias via Agent Memory](https://mlsanity.com/paper/2606.23195/) (2026)
- [CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning](https://mlsanity.com/paper/2605.28742/) (2026)
- [MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use](https://mlsanity.com/paper/2608.20202/) (2026)
- [When Errors Become Memories: Causal Pathway Tracing in Multi-Turn Memory-Augmented LLMs](https://mlsanity.com/paper/2608.30198/) (2026)
- [MemTools: A Unified Research Framework for Interoperable Agent Memory](https://mlsanity.com/paper/2607.21404/) (2026)
- [DreamBench-SWE: A Multi-Session Memory-Hygiene Benchmark for Software Agents](https://mlsanity.com/paper/2608.20664/) (2026)
- [Self-Evolving Multimedia Verification through Memory Consolidation of Contestation Experiences](https://mlsanity.com/paper/2609.27175/) (2026)
- [Emergi-PersonaOS: A Persona Agent Operating System for Situational Adaptation and Controllable Evolution](https://mlsanity.com/paper/2609.27417/) (2026)
- [RoutePrism: Tracing Construction Order Effects in Agent Memory](https://mlsanity.com/paper/2609.34160/) (2026)
- [The Delegation Danger Band: Why Mid-Capability Sub-Agents Over-Trust Inherited Stale State](https://mlsanity.com/paper/2610.00041/) (2026)
- [AECG: Asymmetric Experience Consolidation and Governance In Multi-Agent Systems](https://mlsanity.com/paper/2610.05176/) (2026)
- [Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory](https://mlsanity.com/paper/2610.07311/) (2026)
- [VitaBench: Memory Lifecycles Beyond Retrieval](https://mlsanity.com/paper/2610.14731/) (2026)

## Abstract (arXiv)

Learning from past experience benefits from two complementary forms of memory: episodic traces -- raw trajectories of what happened -- and consolidated abstractions distilled across many episodes into reusable, schema-like lessons. Recent agentic-memory systems pursue the consolidated form: an LLM rewrites past trajectories into a textual memory bank that it continuously updates with new interactions, promising self-improving agents without parameter updates. Yet we find that such consolidated memories produced by today's LLMs are often faulty even when derived from useful experiences. As consolidation proceeds, memory utility first rises, then degrades, and can fall below the no-memory baseline. More surprisingly, even when consolidating from ground-truth solutions, GPT-5.4 fails on 54% of a set of ARC-AGI problems it had previously solved without memory. We trace the regression to the consolidation step rather than the underlying experience: the same trajectories yield qualitatively different memories under different update schedules, and an episodic-only control that simply retains those trajectories remains competitive with the consolidators we test. In a controlled ARC-AGI Stream environment that exposes Retain, Delete, and Consolidate actions, agents preserve raw episodes by default and double the accuracy of their forced-consolidation counterparts; disabling consolidation entirely (episodic management only) matches this auto regime. Practically, robust agent memory should treat raw episodes as first-class evidence and gate consolidation explicitly rather than firing it after every interaction. Looking forward, reliable agentic memory will require LLMs that can consolidate without overwriting the evidence they depend on.