# Learning to Continually Learn via Meta-learning Agentic Memory Designs

arXiv: https://arxiv.org/abs/2602.07755 · HTML: https://mlsanity.com/paper/2602.07755/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 33 of its 88 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 30 of 33)

- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [Agent Workflow Memory](https://mlsanity.com/paper/2409.07429/) (2024)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents](https://mlsanity.com/paper/2505.22954/) (2025)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory](https://mlsanity.com/paper/2509.25140/) (2025)
- [Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning](https://mlsanity.com/paper/2508.19828/) (2025)
- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [Memento: Fine-tuning LLM Agents without Fine-tuning LLMs](https://mlsanity.com/paper/2508.16153/) (2025)
- [G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems](https://mlsanity.com/paper/2506.07398/) (2025)
- [MemGen: Weaving Generative Latent Memory for Self-Evolving Agents](https://mlsanity.com/paper/2509.24704/) (2025)
- [Symbolic Learning Enables Self-Evolving Agents](https://mlsanity.com/paper/2406.18532/) (2024)
- [Memp: Exploring Agent Procedural Memory](https://mlsanity.com/paper/2508.06433/) (2025)
- [MemEvolve: Meta-Evolution of Agent Memory Systems](https://mlsanity.com/paper/2512.18746/) (2025)
- [AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence](https://mlsanity.com/paper/1905.10985/) (2019)
- [Agent Learning via Early Experience](https://mlsanity.com/paper/2510.08558/) (2025)
- [Latent Collaboration in Multi-Agent Systems](https://mlsanity.com/paper/2511.20639/) (2025)
- [Mem-α: Learning Memory Construction via Reinforcement Learning](https://mlsanity.com/paper/2509.25911/) (2025)
- [What Deserves Memory: Adaptive Memory Distillation for LLM Agents](https://mlsanity.com/paper/2508.03341/) (2025)
- [GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators](https://mlsanity.com/paper/2512.19682/) (2025)
- [SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence](https://mlsanity.com/paper/2506.15672/) (2025)

## Built on by (most cited first, 18 of 18)

- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents](https://mlsanity.com/paper/2605.22166/) (2026)
- [Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents](https://mlsanity.com/paper/2604.15877/) (2026)
- [SPADE: Self-Play in Adaptive Synthetic Executable Environments](https://mlsanity.com/paper/2608.19197/) (2026)
- [M^star: Every Task Deserves Its Own Memory Harness](https://mlsanity.com/paper/2604.11811/) (2026)
- [Harnessing Agentic Evolution](https://mlsanity.com/paper/2605.13821/) (2026)
- [DemoEvolve: Overcoming Sparse Feedback in Agentic Harness Evolution with Demonstrations](https://mlsanity.com/paper/2605.24539/) (2026)
- [AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?](https://mlsanity.com/paper/2608.00155/) (2026)
- [Self-Evolving LLM Memory Extraction Across Heterogeneous Tasks](https://mlsanity.com/paper/2604.11610/) (2026)
- [APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay](https://mlsanity.com/paper/2603.29093/) (2026)
- [Rethinking Experience Utilization in Self-Evolving Language Model Agents](https://mlsanity.com/paper/2605.07164/) (2026)
- [Learning to Learn from Multimodal Experience](https://mlsanity.com/paper/2605.16857/) (2026)
- [Heuristic Learning for Active Flow Control Using Coding Agents](https://mlsanity.com/paper/2607.11565/) (2026)
- [Eta Given Delta: Defining LLM Tool Efficiency With Marginal Tool Utility](https://mlsanity.com/paper/2607.14108/) (2026)
- [MetaBench-Harness: Unlocking End-to-End Optimization of Benchmark Harnesses](https://mlsanity.com/paper/2609.33411/) (2026)
- [Vestrum: Improving Agent Harnesses by Adapting Their Verification, Structure and Memory](https://mlsanity.com/paper/2609.33822/) (2026)
- [MemCodex: Self-Programming Hierarchical Memory for Language Agents](https://mlsanity.com/paper/2609.39765/) (2026)
- [Test-Time Adaptation of Reasoning Strategies with Bayesian Nonparametric Memory](https://mlsanity.com/paper/2610.06516/) (2026)

## Abstract (arXiv)

The statelessness of foundation models bottlenecks agentic systems' ability to continually learn, a core capability for long-horizon reasoning and adaptation. To address this limitation, agentic systems commonly incorporate memory modules to retain and reuse past experience, aiming for continual learning during test time. However, most existing memory designs are human-crafted and fixed, which limits their ability to adapt to the diversity and non-stationarity of real-world tasks. In this paper, we introduce ALMA (Automated meta-Learning of Memory designs for Agentic systems), a framework that meta-learns memory designs to replace hand-engineered memory designs, therefore minimizing human effort and enabling agentic systems to be continual learners across diverse domains. Our approach employs a Meta Agent that searches over memory designs expressed as executable code in an open-ended manner, theoretically allowing the discovery of arbitrary memory designs, including database schemas as well as their retrieval and update mechanisms. Extensive experiments across four sequential decision-making domains demonstrate that the learned memory designs enable more effective and efficient learning from experience than state-of-the-art human-crafted memory designs on all benchmarks. When developed and deployed safely, ALMA represents a step toward self-improving AI systems that learn to be adaptive, continual learners.