mlsanityData to 7 Oct 2026

MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents

arXiv 2506.15841 ↗First posted 18 Jun 2025cs.CL cs.AI cs.IRcited by 226

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Modern language agents must operate over long-horizon, multi-turn interactions, where they retrieve external information, adapt to observations, and answer interdependent queries. Yet, most LLM systems rely on full-context prompting, appending all past turns regardless of their relevance. This leads to unbounded memory growth, increased computational costs, and degraded reasoning performance on out-of-distribution input lengths. We introduce MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks. At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning. This state integrates prior memory with new observations from the environment while strategically discarding irrelevant or redundant information. To support training in more realistic and compositional settings, we propose a simple yet effective and scalable approach to constructing multi-turn environments by composing existing datasets into arbitrarily complex task sequences. Experiments across three domains, including internal retrieval QA, open-domain web QA, and multi-turn web shopping, show that MEM1-7B improves performance by 3.5x while reducing memory usage by 3.7x compared to Qwen2.5-14B-Instruct on a 16-objective multi-hop QA task, and generalizes beyond the training horizon. Our results demonstrate the promise of reasoning-driven memory consolidation as a scalable alternative to existing solutions for training long-horizon interactive agents, where both efficiency and performance are optimized.

Key facts. Cited by 226 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 56 of its 71 references that are in the map. First posted on arXiv 18 Jun 2025.

Builds on

56 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training20 Jan

Other references · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2024Qwen2.5 Technical Reportcited by 4,713
  10. 2024GPT-4o System Cardcited by 4,467
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
38 more
  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  11. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  12. 2025s1: Simple test-time scalingcited by 1,311
  13. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  14. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  15. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  16. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  17. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  18. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  19. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
  20. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
  21. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
  22. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
  23. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  24. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  25. 2024The Faiss librarycited by 370
  26. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  27. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  28. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  29. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 268
  30. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  31. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  32. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  33. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  34. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
  35. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  36. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
  37. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  38. 2025LightThinker: Thinking Step-by-Step Compressioncited by 104

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 50; 2026: 176.

226 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 285
  2. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 253
  3. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 236
  4. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  5. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  6. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  7. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 136
  8. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 132
  9. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  10. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 126
  11. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  12. 2025MemGen: Weaving Generative Latent Memory for Self-Evolving Agentscited by 99
38 more
  1. 2025AgentFold: Long-Horizon Web Agents with Proactive Context Managementcited by 95
  2. 2025Memp: Exploring Agent Procedural Memorycited by 88
  3. 2025PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memorycited by 82
  4. 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 62
  5. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memorycited by 61
  6. 2025WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learningcited by 59
  7. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
  8. 2025Mem-α: Learning Memory Construction via Reinforcement Learningcited by 49
  9. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  10. 2026SkillOS: Learning Skill Curation for Self-Evolving Agentscited by 45
  11. 2025Reducing Cost of LLM Agents with Trajectory Reductioncited by 44
  12. 2026Skill-Pro: Learning Reusable Skills from Experience via Non-Parametric PPO for LLM Agentscited by 44
  13. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  14. 2025LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automationcited by 39
  15. 2026CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discoverycited by 38
  16. 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Taskscited by 37
  17. 2025AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learningcited by 32
  18. 2025O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agentscited by 30
  19. 2025CriticSearch: Fine-Grained Credit Assignment for Search Agents via a Retrospective Criticcited by 29
  20. 2025MemSearcher: Training LLMs to Reason, Search and Manage Memory via End-to-End Reinforcement Learningcited by 26
  21. 2026The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Breakcited by 25
  22. 2025IterResearch: Rethinking Long-Horizon Agents with Interaction Scalingcited by 22
  23. 2026AgentOCR: Reimagining Agent History via Optical Self-Compressioncited by 22
  24. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 22
  25. 2025SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agentscited by 21
  26. 2026SkillRevise: Improving LLM-Authored Agent Skills via Trace-Conditioned Skill Revisioncited by 21
  27. 2026Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitationscited by 20
  28. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  29. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  30. 2025Context Engineering 2.0: The Context of Context Engineeringcited by 17
  31. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memorycited by 16
  32. 2026PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Recordscited by 16
  33. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 16
  34. 2025WebCoach: Self-Evolving Web Agents with Cross-Session Memory Guidancecited by 15
  35. 2026Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agentscited by 14
  36. 2026Mem-T: Densifying Rewards for Long-Horizon Memory Agentscited by 14
  37. 2026HyperMem: Hypergraph Memory for Long-Term Conversationscited by 14
  38. 2025Reinforcement Learning Foundations for Deep Research Systems: A Surveycited by 13

50 most cited of 226 All 226 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
  2. 2026MemWeaver: Weaving Hybrid Memories for Traceable Long-Horizon Agentic Reasoning
  3. 2026MemGym: a Long-Horizon Memory Environment for LLM Agents
  4. 2026Mem-π: Adaptive Memory through Learning When and What to Generate
  5. 2026MemoryCPT: An End-to-End Agent Memory Framework for Cost-Performance Trade-off
  6. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents
  7. 2026MemReread: Enhancing Agentic Long-Context Reasoning via Memory-Guided Rereading
  8. 2026Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents