# Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations

arXiv: https://arxiv.org/abs/2602.19320 · HTML: https://mlsanity.com/paper/2602.19320/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 58 of its 84 references that are in the map.
- First posted on arXiv 22 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 30 of 58)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [LightMem: Lightweight and Efficient Memory-Augmented Generation](https://mlsanity.com/paper/2510.18866/) (2025)
- [MIRIX: Multi-Agent Memory System for LLM-Based Agents](https://mlsanity.com/paper/2507.07957/) (2025)
- [SimpleMem: Efficient Lifelong Memory for LLM Agents](https://mlsanity.com/paper/2601.02553/) (2026)
- [Scaling Long-Horizon LLM Agent via Context-Folding](https://mlsanity.com/paper/2510.11967/) (2025)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026)
- [ACON: Optimizing Context Compression for Long-horizon LLM Agents](https://mlsanity.com/paper/2510.00615/) (2025)
- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [MemGen: Weaving Generative Latent Memory for Self-Evolving Agents](https://mlsanity.com/paper/2509.24704/) (2025)
- [Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution](https://mlsanity.com/paper/2512.10696/) (2025)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks](https://mlsanity.com/paper/2510.12635/) (2025)
- [MAGMA: A Multi-Graph based Agentic Memory Architecture for AI Agents](https://mlsanity.com/paper/2601.03236/) (2026)
- [LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation](https://mlsanity.com/paper/2510.04851/) (2025)
- [KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systems](https://mlsanity.com/paper/2409.14908/) (2024)
- [LiCoMemory: Lightweight and Cognitive Agentic Memory for Efficient Long-Term Reasoning](https://mlsanity.com/paper/2511.01448/) (2025)

## Built on by (most cited first, 21 of 21)

- [Jev-Mem: System-One-Controlled Agentic Memory for Efficient AI Agents](https://mlsanity.com/paper/2609.23986/) (2026)
- [Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance](https://mlsanity.com/paper/2606.00305/) (2026)
- [Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation](https://mlsanity.com/paper/2605.09253/) (2026)
- [The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management](https://mlsanity.com/paper/2605.23071/) (2026)
- [Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS](https://mlsanity.com/paper/2604.10734/) (2026)
- [Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents](https://mlsanity.com/paper/2606.10616/) (2026)
- [V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memory](https://mlsanity.com/paper/2608.01543/) (2026)
- [ByteRover: Agent-Native Memory Through LLM-Curated Hierarchical Context](https://mlsanity.com/paper/2604.01599/) (2026)
- [RCBSF: A Multi-Agent Framework for Automated Contract Revision via Stackelberg Game](https://mlsanity.com/paper/2604.10740/) (2026)
- [FAST: A Synergistic Framework of Attention and State-space Models for Spatiotemporal Traffic Prediction](https://mlsanity.com/paper/2604.13453/) (2026)
- [Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection](https://mlsanity.com/paper/2604.15065/) (2026)
- [When Stored Evidence Stops Being Usable: Scale-Conditioned Evaluation of Agent Memory](https://mlsanity.com/paper/2605.07313/) (2026)
- [Ground Truth First: A Longitudinal Evaluation Instrument for Agent Memory, and the Tenure Crossover in Memory-Architecture Rankings](https://mlsanity.com/paper/2607.21962/) (2026)
- [The Missing Knowledge Layer in Cognitive Architectures for AI Agents](https://mlsanity.com/paper/2604.11364/) (2026)
- [Biological Motifs for Agentic Control](https://mlsanity.com/paper/2607.04240/) (2026)
- [Memory-Orchestrated Semantic System (MOSS): An Auditable Agentic Memory Architecture](https://mlsanity.com/paper/2607.04391/) (2026)
- [Beyond Memory Leaderboards: Evaluating Scientific Memory as Budgeted Context Restoration](https://mlsanity.com/paper/2607.16848/) (2026)
- [Shared Organizational Memory for Enterprise Coding Agents: System Design and Deployment Snapshot](https://mlsanity.com/paper/2608.00122/) (2026)
- [Anatomy of a Quantized Agent: VRAM Stability and Forecasting in Code-Synthesis Agentic Workloads](https://mlsanity.com/paper/2608.15117/) (2026)
- [AkasicMEM: Governed Enterprise Memory for Agents](https://mlsanity.com/paper/2609.25563/) (2026)
- [Bookkeeping, Composition, or Unreachable Gold? Reading MemoryAgentBench's Conflict-Resolution Scores Against a Frozen Last-Write Resolver](https://mlsanity.com/paper/2610.09193/) (2026)

## Abstract (arXiv)

Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows. Despite rapid architectural development, the empirical foundations of these systems remain fragile: existing benchmarks are often underscaled, evaluation metrics are misaligned with semantic utility, performance varies significantly across backbone models, and system-level costs are frequently overlooked. This survey presents a structured analysis of agentic memory from both architectural and system perspectives. We first introduce a concise taxonomy of MAG systems based on four memory structures. Then, we analyze key pain points limiting current systems, including benchmark saturation effects, metric validity and judge sensitivity, backbone-dependent accuracy, and the latency and throughput overhead introduced by memory maintenance. By connecting the memory structure to empirical limitations, this survey clarifies why current agentic memory systems often underperform their theoretical promise and outlines directions for more reliable evaluation and scalable system design.