# QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning

arXiv: https://arxiv.org/abs/2505.17667 · HTML: https://mlsanity.com/paper/2505.17667/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 24 of its 59 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OpenAI o1 System Card](https://mlsanity.com/paper/2412.16720/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://mlsanity.com/paper/2503.18892/) (2025)
- [Process Reinforcement through Implicit Rewards](https://mlsanity.com/paper/2502.01456/) (2025)
- [LIMO: Less is More for Reasoning](https://mlsanity.com/paper/2502.03387/) (2025)
- [Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models](https://mlsanity.com/paper/2501.09686/) (2025)
- [Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning](https://mlsanity.com/paper/2502.14768/) (2025)
- [Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond](https://mlsanity.com/paper/2503.10460/) (2025)
- [Equivalence Between Policy Gradients and Soft Q-Learning](https://mlsanity.com/paper/1704.06440/) (2017)
- [Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.12941/) (2024)
- [How to Train Long-Context Language Models (Effectively)](https://mlsanity.com/paper/2410.02660/) (2024)
- [Competitive Programming with Large Reasoning Models](https://mlsanity.com/paper/2502.06807/) (2025)
- [A Comprehensive Survey on Long Context Language Modeling](https://mlsanity.com/paper/2503.17407/) (2025)
- [What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret](https://mlsanity.com/paper/2503.01491/) (2025)
- [A Comparative Study on Reasoning Patterns of OpenAI's o1 Model](https://mlsanity.com/paper/2410.13639/) (2024)

## Built on by (most cited first, 30 of 50)

- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents](https://mlsanity.com/paper/2509.23040/) (2025)
- [OPSDL: On-Policy Self-Distillation for Long-Context Language Models](https://mlsanity.com/paper/2604.17535/) (2026)
- [SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models](https://mlsanity.com/paper/2509.23863/) (2025)
- [LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts](https://mlsanity.com/paper/2510.19363/) (2025)
- [LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards](https://mlsanity.com/paper/2603.02146/) (2026)
- [InfMem: Learning System-2 Memory Control for Long-Context Agent](https://mlsanity.com/paper/2602.02704/) (2026)
- [When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning](https://mlsanity.com/paper/2602.10560/) (2026)
- [Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learning](https://mlsanity.com/paper/2602.08382/) (2026)
- [Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoning](https://mlsanity.com/paper/2601.10306/) (2026)
- [CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoning](https://mlsanity.com/paper/2601.14952/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [LongRM: Revealing and Unlocking the Context Boundary of Reward Modeling](https://mlsanity.com/paper/2510.06915/) (2025)
- [Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning](https://mlsanity.com/paper/2608.19181/) (2026)
- [RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learning](https://mlsanity.com/paper/2510.07721/) (2025)
- [Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping](https://mlsanity.com/paper/2601.12465/) (2026)
- [Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition](https://mlsanity.com/paper/2604.05523/) (2026)
- [LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning](https://mlsanity.com/paper/2604.14922/) (2026)
- [A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation](https://mlsanity.com/paper/2605.12227/) (2026)
- [LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents](https://mlsanity.com/paper/2606.18388/) (2026)
- [EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systems](https://mlsanity.com/paper/2609.01360/) (2026)
- [LensVLM: Selective Context Expansion for Compressed Visual Representation of Text](https://mlsanity.com/paper/2605.07019/) (2026)
- [LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewards](https://mlsanity.com/paper/2605.31584/) (2026)
- [OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling](https://mlsanity.com/paper/2608.05141/) (2026)
- [ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety](https://mlsanity.com/paper/2508.20468/) (2025)
- [Probing How Scalable Table Data Enhances General Long-Context Reasoning](https://mlsanity.com/paper/2603.21719/) (2026)
- [A Decomposition Perspective to Long-context Reasoning for LLMs](https://mlsanity.com/paper/2604.07981/) (2026)
- [RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrieval](https://mlsanity.com/paper/2604.09494/) (2026)
- [StoryAlign: Evaluating and Training Reward Models for Story Generation](https://mlsanity.com/paper/2605.04831/) (2026)

## Abstract (arXiv)

Recent large reasoning models (LRMs) have demonstrated strong reasoning capabilities through reinforcement learning (RL). These improvements have primarily been observed within the short-context reasoning tasks. In contrast, extending LRMs to effectively process and reason on long-context inputs via RL remains a critical unsolved challenge. To bridge this gap, we first formalize the paradigm of long-context reasoning RL, and identify key challenges in suboptimal training efficiency and unstable optimization process. To address these issues, we propose QwenLong-L1, a framework that adapts short-context LRMs to long-context scenarios via progressive context scaling. Specifically, we utilize a warm-up supervised fine-tuning (SFT) stage to establish a robust initial policy, followed by a curriculum-guided phased RL technique to stabilize the policy evolution, and enhanced with a difficulty-aware retrospective sampling strategy to incentivize the policy exploration. Experiments on seven long-context document question-answering benchmarks demonstrate that QwenLong-L1-32B outperforms flagship LRMs like OpenAI-o3-mini and Qwen3-235B-A22B, achieving performance on par with Claude-3.7-Sonnet-Thinking, demonstrating leading performance among state-of-the-art LRMs. This work advances the development of practical long-context LRMs capable of robust reasoning across information-intensive environments.