mlsanityData to 7 Oct 2026

MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory

arXiv 2601.03192 ↗First posted 6 Jan 2026cs.CLcited by 113

Not on the map. Why?

Cited by 78 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The hallmark of human intelligence is the self-evolving ability to master new skills by learning from past experiences. However, current AI agents struggle to emulate this self-evolution: fine-tuning is computationally expensive and prone to catastrophic forgetting, while existing memory-based methods rely on passive semantic matching that often retrieves noise. To address these challenges, we propose MemRL, a non-parametric approach that evolves via reinforcement learning on episodic memory. By decoupling stable reasoning from plastic memory, MemRL employs a Two-Phase Retrieval mechanism to filter noise and identify high-utility strategies through environmental feedback. Extensive experiments on HLE, BigCodeBench, ALFWorld, and Lifelong Agent Bench demonstrate that MemRL significantly outperforms state-of-the-art baselines, confirming that MemRL effectively reconciles the stability-plasticity dilemma, enabling continuous runtime improvement without weight updates. Code is available at https://github.com/MemTensor/MemRL.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 42 of its 76 references that are in the map. First posted on arXiv 6 Jan 2026.

Builds on

42 of 76 references.

References · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  10. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  12. 2020Learning to summarize from human feedbackcited by 2,562
30 more
  1. 2016Learning without Forgettingcited by 2,465
  2. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  8. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  9. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  10. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  11. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  12. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  13. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  14. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  15. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  16. 2017Neural Episodic Controlcited by 148
  17. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  18. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  19. 2025Agentic Reasoning and Tool Integration for LLMs via Reinforcement Learningcited by 107
  20. 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 93
  21. 2025Memp: Exploring Agent Procedural Memorycited by 88
  22. 2025MemInsight: Autonomous Memory Augmentation for LLM Agentscited by 84
  23. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 68
  24. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 65
  25. 2025LiCoMemory: Lightweight and Cognitive Agentic Memory for Efficient Long-Term Reasoningcited by 37
  26. 2025Learn to Memorize: Optimizing LLM-based Agents with Adaptive Memory Frameworkcited by 25
  27. 2024TRAD: Enhancing LLM Agents with Step-Wise Thought Retrieval and Aligned Decisioncited by 23
  28. 2024Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughtscited by 16
  29. 2025Task Memory Engine: Spatial Memory for Robust Multi-Step LLM Agentscited by 5
  30. 2025Retrieval Feedback Memory Enhancement Large Model Retrieval Generation Methodcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 113.

113 papers.

Sort:

By area:

  1. 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 252
  2. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  3. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 76
  4. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  5. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  6. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 45
  7. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 44
  8. 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 35
  9. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 33
  10. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  11. 2026SkillGraph: Skill-Augmented Reinforcement Learning for Agents via Evolving Skill Graphscited by 15
  12. 2026Mem-T: Densifying Rewards for Long-Horizon Memory Agentscited by 14
38 more
  1. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 13
  2. 2026ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RLcited by 13
  3. 2026MemMA: Coordinating the Memory Cycle through Multi-Agent Reasoning and In-Situ Self-Evolutioncited by 11
  4. 2026EvolveMem:Self-Evolving Memory Architecture via AutoResearch for LLM Agentscited by 11
  5. 2026Skills on the Fly: Test-Time Adaptive Skill Synthesis for LLM Agentscited by 10
  6. 2026Belief Memory: Agent Memory Under Partial Observabilitycited by 9
  7. 2026Auto-Dreamer: Learning Offline Memory Consolidation for Language Agentscited by 9
  8. 2026SkillMaster: Toward Autonomous Skill Mastery in LLM Agentscited by 8
  9. 2026TAME: A Trustworthy Test-Time Evolution of Agent Memory with Systematic Benchmarkingcited by 7
  10. 2026UMEM: Unified Memory Extraction and Management Framework for Generalizable Memorycited by 7
  11. 2026Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systemscited by 6
  12. 2026MemReranker: Reasoning-Aware Reranking for Agent Memory Retrievalcited by 6
  13. 2026CODESKILL: Learning Self-Evolving Skills for Coding Agentscited by 6
  14. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  15. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  16. 2026SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignmentcited by 6
  17. 2026MonoScale: Scaling Multi-Agent System with Monotonic Improvementcited by 5
  18. 2026Complementary RL: Towards Efficient Experience-Driven Agent Learningcited by 5
  19. 2026GSEM: Graph-based Self-Evolving Memory for Experience Augmented Clinical Reasoningcited by 5
  20. 2026Contextual Agentic Memory is a Memo, Not True Memorycited by 5
  21. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  22. 2026Skill0.5: Joint Skill Internalization and Utilization for Out-of-Distribution Generalization in Agentic Reinforcement Learningcited by 5
  23. 2026SIRI: Self-Internalizing Reinforcement Learning with Intrinsic Skills for LLM Agent Trainingcited by 5
  24. 2026VideoWeaver: Evaluating and Evolving Skills for Agentic Long Video Generationcited by 5
  25. 2026Co-Evolving Skill Generation and Policy Optimizationcited by 5
  26. 2026Metis: Memory Foundation Modelcited by 5
  27. 2026PAST-Bench: Benchmarking the Foundations of Recursive Self-Improvement in Personal Agentscited by 5
  28. 2026Holos: A Web-Scale LLM-Based Multi-Agent System for the Agentic Webcited by 4
  29. 2026MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGscited by 4
  30. 2026AdaMEM: Test-Time Adaptive Memory for Language Agentscited by 4
  31. 2026Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agentscited by 4
  32. 2026AutoMem: Automated Learning of Memory as a Cognitive Skillcited by 4
  33. 2026Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspectivecited by 4
  34. 2026Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesiscited by 3
  35. 2026SkillMAS: Skill Co-Evolution with LLM-based Multi-Agent Systemcited by 3
  36. 2026Mem-π: Adaptive Memory through Learning When and What to Generatecited by 3
  37. 2026FinEvolveBench: A Benchmark for Self-Evolving Agents on Low-Repetition Tasks with Implicit Rewardscited by 3
  38. 2026Organize then Retrieve: Hierarchical Memory Navigation for Efficient Agentscited by 3

50 most cited of 113 All 113 (JSON)