# AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agents

arXiv: https://arxiv.org/abs/2606.02461 · HTML: https://mlsanity.com/paper/2606.02461/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents](https://mlsanity.com/paper/2401.13178/) (2024)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory](https://mlsanity.com/paper/2511.20857/) (2025)
- [LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners](https://mlsanity.com/paper/2505.11942/) (2025)
- [MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems](https://mlsanity.com/paper/2510.17281/) (2025)
- [StreamBench: Towards Benchmarking Continuous Improvement of Language Agents](https://mlsanity.com/paper/2406.08747/) (2024)
- [SWE-Bench-CL: Continual Learning for Coding Agents](https://mlsanity.com/paper/2507.00014/) (2025)
- [Agent-Dice: Disentangling Knowledge Updates via Geometric Consensus for Agent Continual Learning](https://mlsanity.com/paper/2601.03641/) (2026)
- [CGL: Advancing Continual GUI Learning via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2603.02951/) (2026)

## Built on by (most cited first, 9 of 9)

- [EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?](https://mlsanity.com/paper/2609.04280/) (2026)
- [ERRAND: Budgeted Maintenance of Agent Memory](https://mlsanity.com/paper/2609.29545/) (2026)
- [EngramBench: A Capability-Grounded Benchmark for Skill-Evolution Harnesses](https://mlsanity.com/paper/2609.39284/) (2026)
- [Connect the Dots: Training LLMs for Long-Lifecycle Agents with Cross-Domain Generalization Via Reinforcement Learning](https://mlsanity.com/paper/2606.20002/) (2026)
- [AhaBench: Do Agents Turn Experience into Reusable Insights? A Long-Horizon Benchmark for Continual Learning](https://mlsanity.com/paper/2609.05435/) (2026)
- [Measure Learning at Steady State: A BIRD-SQL Formula 1 Case Study](https://mlsanity.com/paper/2609.31640/) (2026)
- [ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?](https://mlsanity.com/paper/2610.07792/) (2026)
- [Agent Plasticity: Measuring Self-Improvement Through Experience](https://mlsanity.com/paper/2610.08902/) (2026)
- [EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents](https://mlsanity.com/paper/2610.13021/) (2026)

## Abstract (arXiv)

Language agents spend substantial inference time solving individual tasks, yet the experience acquired in one episode is often underutilized in future episodes. Continual learning expects an agent to accumulate experience across a stream of tasks, improve over time, and avoid interference from irrelevant experiences. Unfortunately, existing benchmarks struggle to evaluate this problem rigorously. Most efforts focus on retrieval and reasoning over long-context conversations or documents, while recent lifelong-adaptation benchmarks often stream existing datasets whose cross-task dependencies are unknown, making it difficult to understand what an agent learns and reuses over time. This paper presents an evaluation framework AgentCL for continual learning in agents, centered on distinguishable task streams and metrics for CL properties. AgentCL distinguishes three cross-task relations by the availability of reusable knowledge. In dependent streams, later tasks can reuse knowledge from earlier ones. In conventional streams, whether such reuse exists is unknown. Besides, independent tasks are held out with little transferable knowledge. We use the benchmark to evaluate non-parametric memory designs for continual learning. To diagnose how memory design choices affect continual learning, we develop MemProbe, a probing method that stores interactions, insights, and skills, while filtering unreliable experiences during consolidation. Empirical analysis across coding, deep research, and language understanding/reasoning tasks shows that conventional streams offer limited ability to distinguish memory designs, whereas dependent streams more clearly distinguish their plasticity. Meanwhile, conventional streams and independent tasks often yield limited gains and can expose memory-induced degradation. These results highlight the need for stronger memory designs that balance plasticity and stable reuse.