# On the Structural Memory of LLM Agents

arXiv: https://arxiv.org/abs/2412.15266 · HTML: https://mlsanity.com/paper/2412.15266/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 28 of 28)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [Precise Zero-Shot Dense Retrieval without Relevance Labels](https://mlsanity.com/paper/2212.10496/) (2022)
- [From Persona to Personalization: A Survey on Role-Playing Language Agents](https://mlsanity.com/paper/2404.18231/) (2024)
- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviews](https://mlsanity.com/paper/2310.17976/) (2023)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization](https://mlsanity.com/paper/2402.17574/) (2024)
- [MetaAgents: Large Language Model Based Agents for Decision-Making on Teaming](https://mlsanity.com/paper/2310.06500/) (2023)
- [Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2406.14891/) (2024)
- [Don't Forget to Connect! Improving RAG with Graph-based Reranking](https://mlsanity.com/paper/2405.18414/) (2024)
- [CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks](https://mlsanity.com/paper/2402.01176/) (2024)
- [TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.11460/) (2024)

## Built on by (most cited first, 30 of 37)

- [Memory Injection Attacks on LLM Agents via Query-Only Interaction](https://mlsanity.com/paper/2503.03704/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior](https://mlsanity.com/paper/2505.16067/) (2025)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving](https://mlsanity.com/paper/2507.06229/) (2025)
- [EvoAgentX: An Automated Framework for Evolving Agentic Workflows](https://mlsanity.com/paper/2507.03616/) (2025)
- [MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learning](https://mlsanity.com/paper/2511.02805/) (2025)
- [Graph-based Agent Memory: Taxonomy, Techniques, and Applications](https://mlsanity.com/paper/2602.05665/) (2026)
- [Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities](https://mlsanity.com/paper/2506.18019/) (2025)
- [Self-Improvements in Modern Agentic Systems: A Survey](https://mlsanity.com/paper/2607.13104/) (2026)
- [The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems](https://mlsanity.com/paper/2507.02097/) (2025)
- [MEMTRACK: Evaluating Long-Term Memory and State Tracking in Multi-Platform Dynamic Agent Environments](https://mlsanity.com/paper/2510.01353/) (2025)
- [On the Failure of Latent State Persistence in Large Language Models](https://mlsanity.com/paper/2505.10571/) (2025)
- [Gated Differentiable Working Memory for Long-Context Language Modeling](https://mlsanity.com/paper/2601.12906/) (2026)
- [A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysis](https://mlsanity.com/paper/2506.12263/) (2025)
- [Aligning Agentic World Models via Knowledgeable Experience Learning](https://mlsanity.com/paper/2601.13247/) (2026)
- [MA-EgoQA: Question Answering over Egocentric Videos from Multiple Embodied Agents](https://mlsanity.com/paper/2603.09827/) (2026)
- [Learning to Retrieve: Dual-Level Long-Term Memory for Text-to-SQL Agents](https://mlsanity.com/paper/2606.00547/) (2026)
- [MARK: Memory Augmented Refinement of Knowledge](https://mlsanity.com/paper/2505.05177/) (2025)
- [Can Past Experience Accelerate LLM Reasoning?](https://mlsanity.com/paper/2505.20643/) (2025)
- [A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness](https://mlsanity.com/paper/2512.08936/) (2025)
- [Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agents](https://mlsanity.com/paper/2606.11680/) (2026)
- [Training-Free Multimodal Large Language Model Orchestration](https://mlsanity.com/paper/2508.10016/) (2025)
- [Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation](https://mlsanity.com/paper/2602.09319/) (2026)
- [From Agent Loops to Deterministic Graphs: Execution Lineage for Reproducible AI-Native Work](https://mlsanity.com/paper/2605.06365/) (2026)
- [Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory](https://mlsanity.com/paper/2601.00821/) (2025)
- [MetaEvo: A Meta-Optimization Framework for Experience-Driven Agent Evolution](https://mlsanity.com/paper/2606.07603/) (2026)
- [A Technical Taxonomy of LLM Agent Communication Protocols](https://mlsanity.com/paper/2606.19135/) (2026)
- [Dual-Layer Agentic Memory with Fast Write Routing and Slow Consolidation](https://mlsanity.com/paper/2608.22215/) (2026)
- [When Does Selection Replace Extraction? A Pre-Registered Test of Agent Memory with a Typed Decision Model](https://mlsanity.com/paper/2609.34227/) (2026)

## Abstract (arXiv)

Memory plays a pivotal role in enabling large language model~(LLM)-based agents to engage in complex and long-term interactions, such as question answering (QA) and dialogue systems. While various memory modules have been proposed for these tasks, the impact of different memory structures across tasks remains insufficiently explored. This paper investigates how memory structures and memory retrieval methods affect the performance of LLM-based agents. Specifically, we evaluate four types of memory structures, including chunks, knowledge triples, atomic facts, and summaries, along with mixed memory that combines these components. In addition, we evaluate three widely used memory retrieval methods: single-step retrieval, reranking, and iterative retrieval. Extensive experiments conducted across four tasks and six datasets yield the following key insights: (1) Different memory structures offer distinct advantages, enabling them to be tailored to specific tasks; (2) Mixed memory structures demonstrate remarkable resilience in noisy environments; (3) Iterative retrieval consistently outperforms other methods across various scenarios. Our investigation aims to inspire further research into the design of memory systems for LLM-based agents.