# Scaling Long-Horizon LLM Agent via Context-Folding

arXiv: https://arxiv.org/abs/2510.11967 · HTML: https://mlsanity.com/paper/2510.11967/

## Key facts

- Cited by 148 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 33 of its 42 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 30 of 33)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [WebThinker: Empowering Large Reasoning Models with Deep Research Capability](https://mlsanity.com/paper/2504.21776/) (2025)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents](https://mlsanity.com/paper/2506.15841/) (2025)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [Measuring AI Ability to Complete Long Software Tasks](https://mlsanity.com/paper/2503.14499/) (2025)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization](https://mlsanity.com/paper/2509.13313/) (2025)
- [Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving](https://mlsanity.com/paper/2507.06229/) (2025)
- [WideSearch: Benchmarking Agentic Broad Info-Seeking](https://mlsanity.com/paper/2508.07999/) (2025)
- [WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents](https://mlsanity.com/paper/2509.13309/) (2025)
- [WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning](https://mlsanity.com/paper/2509.13305/) (2025)
- [Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management](https://mlsanity.com/paper/2510.06727/) (2025)
- [LongAgent: Scaling Language Models to 128k Context through Multi-Agent Collaboration](https://mlsanity.com/paper/2402.11550/) (2024)

## Built on by (most cited first, 30 of 148)

- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [Recursive Language Models](https://mlsanity.com/paper/2512.24601/) (2025)
- [DeepAgent: A General Reasoning Agent with Scalable Toolsets](https://mlsanity.com/paper/2510.21618/) (2025)
- [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026)
- [Meta Context Engineering via Agentic Skill Evolution](https://mlsanity.com/paper/2601.21557/) (2026)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [Training Proactive and Personalized LLM Agents](https://mlsanity.com/paper/2511.02208/) (2025)
- [SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents](https://mlsanity.com/paper/2601.16746/) (2026)
- [A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents](https://mlsanity.com/paper/2602.06052/) (2026)
- [Context as a Tool: Context Management for Long-Horizon SWE-Agents](https://mlsanity.com/paper/2512.22087/) (2025)
- [AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration](https://mlsanity.com/paper/2602.03786/) (2026)
- [SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale](https://mlsanity.com/paper/2602.23866/) (2026)
- [SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training](https://mlsanity.com/paper/2602.03411/) (2026)
- [Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent](https://mlsanity.com/paper/2606.30616/) (2026)
- [OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent](https://mlsanity.com/paper/2601.07779/) (2026)
- [Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations](https://mlsanity.com/paper/2602.19320/) (2026)
- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding](https://mlsanity.com/paper/2511.22850/) (2025)
- [SCOPE: Prompt Evolution for Enhancing Agent Effectiveness](https://mlsanity.com/paper/2512.15374/) (2025)
- [Mem-T: Densifying Rewards for Long-Horizon Memory Agents](https://mlsanity.com/paper/2601.23014/) (2026)
- [Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned](https://mlsanity.com/paper/2603.05344/) (2026)
- [Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks](https://mlsanity.com/paper/2604.11753/) (2026)
- [MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization](https://mlsanity.com/paper/2603.25973/) (2026)
- [Evo-Harness: Context-to-Harness Skill Compilation for Self-Evolving Agents](https://mlsanity.com/paper/2608.15071/) (2026)
- [MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning](https://mlsanity.com/paper/2601.21468/) (2026)
- [ClawGym: A Scalable Framework for Building Effective Claw Agents](https://mlsanity.com/paper/2604.26904/) (2026)
- [Learning Agent-Compatible Context Management for Long-Horizon Tasks](https://mlsanity.com/paper/2605.30785/) (2026)
- [SWE-MeM: Learning Adaptive Memory Management for Long-Horizon Coding Agents](https://mlsanity.com/paper/2606.28434/) (2026)
- [ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents](https://mlsanity.com/paper/2604.01664/) (2026)
- [CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents](https://mlsanity.com/paper/2607.05378/) (2026)

## Abstract (arXiv)

Large language model (LLM) agents are fundamentally constrained by context length on long-horizon tasks. We introduce Context-Folding, a framework that empowers agents to actively manage their working context. An agent can procedurally branch into a sub-trajectory to handle a subtask and then fold it upon completion, collapsing the intermediate steps while retaining a concise summary of the outcome. To make this behavior learnable, we develop an end-to-end reinforcement learning framework FoldGRPO with specific process rewards to encourage effective task decomposition and context management. On complex long-horizon tasks (Deep Research and SWE), our folding agent matches or outperforms the ReAct baselines while using an active context 10× smaller and significantly outperforms models that rely on summarization-based context management.