ECHO: Prune To Act, Trace To Learn With Selective Turn Memory In Agentic RL
On the map: LLM agents and tool use → Agent memory systems
Abstract (arXiv)
Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows. Context-management methods make such rollouts feasible by simplifying past interactions through deletion, folding, or memory editing. However, when useful history is collapsed into compressed states, the reconstructed context may no longer reveal which earlier observations support a successful final answer. This creates a mismatch between bounded-context acting and outcome-based reinforcement learning: the policy acts on reconstructed context, while the learner lacks source-level provenance for assigning credit to the evidence that mattered. We propose ECHO, a selective turn-memory framework for traceable context reconstruction in Agentic RL. ECHO compresses each completed environment turn into a compact source-indexed memory record, reconstructs bounded policy contexts by selecting useful records, and reuses the selected source indices to route positive outcome credit to the final trajectory segment, reused evidence turns, memory findings, and memory-selection actions. On BrowseComp-Plus, ECHO reaches 43.4% held-out accuracy, outperforming GRPO at 28.9% and the rolling-summary baseline SUPO at 36.1%, while using fewer turns and lower trajectory volume than SUPO. The trained policy also improves zero-shot generalization across multi-objective QA, code generation, and deep information-seeking benchmarks on both dense and MoE backbones.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 51 of its 57 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
51 of 57 references.
Closest ancestors · 12
Same direction, newest first
- 2026SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent23 May
- 2026ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents24 Apr
- 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks14 Oct
- 2025Scaling Long-Horizon LLM Agent via Context-Folding13 Oct
- 2025ACON: Optimizing Context Compression for Long-horizon LLM Agents1 Oct
- 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization16 Sep
- 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning27 Aug
- 2025MIRIX: Multi-Agent Memory System for LLM-Based Agents10 Jul
- 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents18 Jun
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory28 Apr
- 2025A-MEM: Agentic Memory for LLM Agents17 Feb
- 2024HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model18 Aug
Other references · 39
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
27 more
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2023MemoryBank: Enhancing Large Language Models with Long-Term Memorycited by 556
- 2025ToolRL: Reward is All Tool Learning Needscited by 362
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
- 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
- 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 189
- 2025ToRL: Scaling Tool-Integrated RLcited by 169
- 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
- 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agentscited by 68
- 2024Prompt Compression for Large Language Models: A Surveycited by 67
- 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 65
- 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
- 2025PENCIL: Long Thoughts with Short Memorycited by 33
- 2025Cognitive Memory in Large Language Modelscited by 29
- 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 23
- 2026Milestone-Guided Policy Learning for Long-Horizon Language Agentscited by 19
- 2026HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agentscited by 18
- 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 17
- 2025LoCoBench-Agent: An Interactive Benchmark for LLM Agents in Long-Context Software Engineeringcited by 16
- 2024In-Context Former: Lightning-fast Compressing Context for Large Language Modelcited by 12
- 2024Concise and Precise Context Compression for Tool-Using Language Modelscited by 12
- 2026MemPO: Self-Memory Policy Optimization for Long-Horizon Agentscited by 12
- 2026Agent-Omit: Adaptive Context Omission for Efficient LLM Agentscited by 5
- 2026AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learningcited by 5
- 2025QwenLong-CPRS: Towards ∞-LLMs with Dynamic Context Optimizationcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Group-Reflective Self-Distillation for Agentic Reinforcement Learningcited by 6
- 2026BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RLcited by 0
- 2026Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learningcited by 0
- 2026Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning28 Sep 2026
- 2026BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RL9 Sep 2026
- 2026Group-Reflective Self-Distillation for Agentic Reinforcement Learning30 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents
- 2026ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL
- 2026Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
- 2026Memory Control Signals Emerge Before Action in Long Horizon Agents
- 2026ECHO: Learning Epistemically Adaptive Language Agents with Turn-Level Credit ↗
- 2026MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents
- 2025Memento 2: Learning by Stateful Reflective Memory
- 2026Skill Reuse as Compression in Agentic RL