# QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents

arXiv: https://arxiv.org/abs/2609.33848 · HTML: https://mlsanity.com/paper/2609.33848/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 43 of its 57 references that are in the map.
- First posted on arXiv 27 Sep 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)
- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 30 of 43)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism](https://mlsanity.com/paper/1909.08053/) (2019)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2504.20073/) (2025)
- [SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?](https://mlsanity.com/paper/2509.16941/) (2025)
- [AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning](https://mlsanity.com/paper/2505.24298/) (2025)
- [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://mlsanity.com/paper/2508.03680/) (2025)
- [Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library](https://mlsanity.com/paper/2506.06122/) (2025)
- [VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use](https://mlsanity.com/paper/2509.01055/) (2025)
- [NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents](https://mlsanity.com/paper/2512.12730/) (2025)
- [AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework](https://mlsanity.com/paper/2510.04206/) (2025)
- [StreamRL: Scalable, Heterogeneous, and Elastic RL for LLMs with Disaggregated Stream Generation](https://mlsanity.com/paper/2504.15930/) (2025)
- [DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks](https://mlsanity.com/paper/2607.07946/) (2026)
- [SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent](https://mlsanity.com/paper/2511.16108/) (2025)
- [AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training](https://mlsanity.com/paper/2507.01663/) (2025)
- [ReaL: Efficient RLHF Training of Large Language Models with Parameter Reallocation](https://mlsanity.com/paper/2406.14088/) (2024)
- [Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning](https://mlsanity.com/paper/2607.07508/) (2026)
- [SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training](https://mlsanity.com/paper/2602.03411/) (2026)
- [DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training](https://mlsanity.com/paper/2604.26256/) (2026)
- [Polar: Agentic RL on Any Harness at Scale](https://mlsanity.com/paper/2605.24220/) (2026)
- [Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learning](https://mlsanity.com/paper/2607.14171/) (2026)
- [StaleFlow: Staleness-Aware Data Management for Mitigating Data Skewness in Fully Disaggregated RL Post-Training](https://mlsanity.com/paper/2601.12784/) (2026)
- [MegaFlow: Large-Scale Distributed Orchestration System for the Agentic Era](https://mlsanity.com/paper/2601.07526/) (2026)
- [Heddle: A Distributed Orchestration System for Agentic RL Rollout](https://mlsanity.com/paper/2603.28101/) (2026)
- [ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RL](https://mlsanity.com/paper/2605.06534/) (2026)
- [Agent Lightning v1.0: Towards Harnessed Agentic RL](https://mlsanity.com/paper/2608.17528/) (2026)
- [Tree Training: Accelerating Agentic LLMs Training via Shared Prefix Reuse](https://mlsanity.com/paper/2511.00413/) (2025)

## Abstract (arXiv)

Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model--environment interactions, and nearly 1M tokens per rollout. Applying online reinforcement learning (RL) to such executions poses two fundamental challenges: (1) severe execution variance and prolonged rollout delays cause massive GPU idling; and (2) complex non-linear branching generates massive trajectory redundancy, crippling training efficiency. To address these, we presents QwenGyre, an end-to-end framework for xlong-horizon online RL. QwenGyre elastically reallocates GPUs between rollout and training without interrupting live executions, while its trajectory processor reconstructs branching histories, scores partial progress, and deduplicates redundant paths to bound training costs. Scaled to our flagship model, Qwen~3.8 2.4T, with 700K tokens per rollout, QwenGyre yields a 6.0% absolute gain on NL2RepoBench (52.5% → 58.5%) in 48 steps. Across our evaluations on diverse domains of training datasets, QwenGyre delivers up to 1.85× and 1.78× speedups over Colocate and Async, respectively.