mlsanityData to 7 Oct 2026

CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

arXiv 2607.05378 ↗First posted 6 Jul 2026cs.LGcited by 8

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Long-horizon agentic LLMs are increasingly limited by finite context windows, as extended interaction trajectories can exceed the maximum context length before a task is completed. Context compaction offers a natural solution by summarizing previous interaction states and continuing the rollout under a compressed context, but incorporating compaction into reinforcement learning remains underexplored. We propose CompactionRL, a reinforcement learning strategy to train long-horizon agentic LLMs with context compaction. Our approach jointly optimizes task execution and summary generation with token-level loss normalization and cross-segment generalized advantage estimation. This design enables the LLM agents to learn from compacted long-horizon trajectories. We train CompactionRL on top of open models and observe consistent performance gains on agentic coding tasks. CompactionRL enables the open GLM-4.5-Air model (106B-A12B) to achieve Pass@1 scores of 66.4% on SWE-bench Verified and 26.2% on Terminal-Bench 2.0, exceeding the base model under inference-time compaction by 6.6 and 4.9 points, respectively. Built upon GLM-4.7-Flash (30B-A3B), CompactionRL improves Pass@1 by 5.5 and 6.7 points against the base model, reaching 56.0% on SWE-bench Verified and 20.2% on Terminal-Bench 2.0. CompactionRL is thus deployed in the RL pipeline for training the open GLM-5.2 model (750B-A40B).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 41 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

33 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar

Other references · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  12. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
20 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  3. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  4. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  5. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  6. 2025Group Sequence Policy Optimizationcited by 788
  7. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  8. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 443
  9. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 281
  10. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  11. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  12. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 136
  13. 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 132
  14. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secretcited by 95
  15. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77
  16. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 75
  17. 2025Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Managementcited by 51
  18. 2025MobileRL: Online Agentic Reinforcement Learning for Mobile GUI Agentscited by 43
  19. 2025SWE-Dev: Building Software Engineering Agents with Training and Inference Scalingcited by 24
  20. 2026Learning Agent-Compatible Context Management for Long-Horizon Taskscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Occamy-1.0: Open Pareto-frontier 35B Intelligence for Co-workcited by 3
  2. 2026Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flatteningcited by 1
  3. 2026KV-streams for Efficient Compaction in Agentic Reinforcement Learningcited by 1
  4. 2026AgentOmnia: Scaling Agentic Models for Full-Scenario Applicationscited by 0
  5. 2026Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learningcited by 0
  6. 2026Continuous Context Managementcited by 0
  7. 2026Context Language Modelscited by 0
  8. 2026AutoCompact: Learning When to Compact Context in Long-Horizon Coding Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
  2. 2025FoldAct: Efficient and Stable Context Folding for Long-Horizon Search Agents
  3. 2026Act More, Decide Less: Skill-Guided Adaptive Action Chunking for Long-Horizon LLM Agents
  4. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization
  5. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  6. 2025Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
  7. 2026Agentic ESOpt: Fine-Tuning Long-Horizon LLM Agents with Minimal GPU Requirements
  8. 2026Parallel Context Compaction for Long-Horizon LLM Agent Serving