# MemPO: Self-Memory Policy Optimization for Long-Horizon Agents

arXiv: https://arxiv.org/abs/2603.00680 · HTML: https://mlsanity.com/paper/2603.00680/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges](https://mlsanity.com/paper/2401.07339/) (2024)
- [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025)
- [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024)
- [Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control](https://mlsanity.com/paper/2306.07863/) (2023)
- [MapCoder: Multi-Agent Code Generation for Competitive Problem Solving](https://mlsanity.com/paper/2405.11403/) (2024)
- [WebWalker: Benchmarking LLMs in Web Traversal](https://mlsanity.com/paper/2501.07572/) (2025)
- [Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.12941/) (2024)
- [VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool](https://mlsanity.com/paper/2408.08927/) (2024)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [ExpSeek: Self-Triggered Experience Seeking for Web Agents](https://mlsanity.com/paper/2601.08605/) (2026)
- [Beyond Turn Limits: Training Deep Search Agents with Dynamic Context Window](https://mlsanity.com/paper/2510.08276/) (2025)
- [RE-RAG: Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.05794/) (2024)
- [Exploring Forgetting in Large Language Model Pre-Training](https://mlsanity.com/paper/2410.17018/) (2024)
- [ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning](https://mlsanity.com/paper/2602.09953/) (2026)
- [MOA: Multi-Objective Alignment for Role-Playing Agents](https://mlsanity.com/paper/2512.09756/) (2025)
- [Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints](https://mlsanity.com/paper/2510.08549/) (2025)

## Built on by (most cited first, 12 of 12)

- [Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents](https://mlsanity.com/paper/2604.15877/) (2026)
- [HiMPO: Hindsight-Informed Memory Policy Optimization for Less-Entangled Credit in Long-Horizon Agents](https://mlsanity.com/paper/2606.16285/) (2026)
- [ECHO: Prune To Act, Trace To Learn With Selective Turn Memory In Agentic RL](https://mlsanity.com/paper/2606.31650/) (2026)
- [MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents](https://mlsanity.com/paper/2608.07068/) (2026)
- [Break It Down, Pass It On: Cross-Task Skill Transfer in LLM Agents](https://mlsanity.com/paper/2608.20274/) (2026)
- [The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents](https://mlsanity.com/paper/2608.06663/) (2026)
- [FORGE: Form-Optimal Routing of Grounded Evidence for Frozen LLM Agents](https://mlsanity.com/paper/2609.34358/) (2026)
- [Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning](https://mlsanity.com/paper/2609.34422/) (2026)
- [Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses](https://mlsanity.com/paper/2609.34649/) (2026)
- [Learning What to Remember: Long-horizon Counterfactual Memory Optimization](https://mlsanity.com/paper/2609.37930/) (2026)
- [Causal Memory Policy: Making Memory Utility Identifiable by Intervening on Retrieval](https://mlsanity.com/paper/2610.02070/) (2026)
- [TreeFlex: Agents Navigating Their Own Search Trees](https://mlsanity.com/paper/2610.20334/) (2026)

## Abstract (arXiv)

Long-horizon agents face the challenge of growing context size during interaction with environment, which degrades the performance and stability. Existing methods typically introduce the external memory module and look up the relevant information from the stored memory, which prevents the model itself from proactively managing its memory content and aligning with the agent's overarching task objectives. To address these limitations, we propose the self-memory policy optimization algorithm (MemPO), which enables the agent (policy model) to autonomously summarize and manage their memory during interaction with environment. By improving the credit assignment mechanism based on memory effectiveness, the policy model can selectively retain crucial information, significantly reducing token consumption while preserving task performance. Extensive experiments and analyses confirm that MemPO achieves absolute F1 score gains of 25.98 over the base model and 7.1 over the previous SOTA baseline, while reducing token usage by 67.58% and 73.12%. The code is released at https://github.com/TheNewBeeKing/MemPO.