# Self-evolving Agents with reflective and memory-augmented abilities

arXiv: https://arxiv.org/abs/2409.00872 · HTML: https://mlsanity.com/paper/2409.00872/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 1 Sep 2024.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)
- Synthetic data and self-improvement for LLMs / [Self-evolving LLM agents](https://mlsanity.com/synth/self-evolving-llm-agents/)

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://mlsanity.com/paper/2207.01206/) (2022)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems](https://mlsanity.com/paper/2306.03091/) (2023)
- [Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions](https://mlsanity.com/paper/2306.02224/) (2023)
- [TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods](https://mlsanity.com/paper/2403.20150/) (2024)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing](https://mlsanity.com/paper/2404.12253/) (2024)
- [MuTual: A Dataset for Multi-Turn Dialogue Reasoning](https://mlsanity.com/paper/2004.04494/) (2020)
- [LongCoder: A Long-Range Pre-trained Language Model for Code Completion](https://mlsanity.com/paper/2306.14893/) (2023)
- [CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models](https://mlsanity.com/paper/2404.01663/) (2024)
- [Dense Passage Retrieval: Is it Retrieving?](https://mlsanity.com/paper/2402.11035/) (2024)
- [Long-form Question Answering: An Iterative Planning-Retrieval-Generation Approach](https://mlsanity.com/paper/2311.09383/) (2023)

## Built on by (most cited first, 30 of 52)

- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems](https://mlsanity.com/paper/2506.07398/) (2025)
- [How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior](https://mlsanity.com/paper/2505.16067/) (2025)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections](https://mlsanity.com/paper/2602.15654/) (2026)
- [A Survey on the Optimization of Large Language Model-based Agents](https://mlsanity.com/paper/2503.12434/) (2025)
- [SCORE: Story Coherence and Retrieval Enhancement for AI Narratives](https://mlsanity.com/paper/2503.23512/) (2025)
- [AI PERSONA: Towards Life-long Personalization of LLMs](https://mlsanity.com/paper/2412.13103/) (2024)
- [ReCreate: Reasoning and Creating Domain Agents Driven by Experience](https://mlsanity.com/paper/2601.11100/) (2026)
- [Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks](https://mlsanity.com/paper/2510.08002/) (2025)
- [Coarse-to-Fine Grounded Memory for LLM Agent Planning](https://mlsanity.com/paper/2508.15305/) (2025)
- [A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compression](https://mlsanity.com/paper/2604.19572/) (2026)
- [ForgetMe: Evaluating Selective Forgetting in Generative Models](https://mlsanity.com/paper/2504.12574/) (2025)
- [Alita-G: Self-Evolving Generative Agent for Agent Generation](https://mlsanity.com/paper/2510.23601/) (2025)
- [Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architectures](https://mlsanity.com/paper/2508.10824/) (2025)
- [Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems](https://mlsanity.com/paper/2605.26302/) (2026)
- [Meta-Policy Reflexion: Reusable Reflective Memory and Rule Admissibility for Resource-Efficient LLM Agent](https://mlsanity.com/paper/2509.03990/) (2025)
- [ID-RAG: Identity Retrieval-Augmented Generation for Long-Horizon Persona Coherence in Generative Agents](https://mlsanity.com/paper/2509.25299/) (2025)
- [PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMs](https://mlsanity.com/paper/2601.20539/) (2026)
- [SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents](https://mlsanity.com/paper/2604.07791/) (2026)
- [ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Querying](https://mlsanity.com/paper/2604.09747/) (2026)
- [From History to State: Constant-Context Skill Learning for LLM Agents](https://mlsanity.com/paper/2605.05413/) (2026)
- [AutoSci: A Memory-Centric Agentic System for the Full Scientific Research Lifecycle](https://mlsanity.com/paper/2605.31468/) (2026)
- [PreAct: Computer-Using Agents that Get Faster on Repeated Tasks](https://mlsanity.com/paper/2606.17929/) (2026)
- [Beyond Parameters: Exploring Virtual Logic Depth for Scaling Laws](https://mlsanity.com/paper/2506.18233/) (2025)
- [Your Code Agent Can Grow Alongside You with Structured Memory](https://mlsanity.com/paper/2603.13258/) (2026)
- [Time is Not a Label: Continuous Phase Rotation for Temporal Knowledge Graphs and Agentic Memory](https://mlsanity.com/paper/2604.11544/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have made significant advances in the field of natural language processing, but they still face challenges such as continuous decision-making. In this research, we propose a novel framework by integrating iterative feedback, reflective mechanisms, and a memory optimization mechanism based on the Ebbinghaus forgetting curve, it significantly enhances the agents' capabilities in handling multi-tasking and long-span information.