mlsanityData to 14 Oct 2026

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning

arXiv 2505.17667 ↗First posted 23 May 2025cs.CLcited by 50

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent large reasoning models (LRMs) have demonstrated strong reasoning capabilities through reinforcement learning (RL). These improvements have primarily been observed within the short-context reasoning tasks. In contrast, extending LRMs to effectively process and reason on long-context inputs via RL remains a critical unsolved challenge. To bridge this gap, we first formalize the paradigm of long-context reasoning RL, and identify key challenges in suboptimal training efficiency and unstable optimization process. To address these issues, we propose QwenLong-L1, a framework that adapts short-context LRMs to long-context scenarios via progressive context scaling. Specifically, we utilize a warm-up supervised fine-tuning (SFT) stage to establish a robust initial policy, followed by a curriculum-guided phased RL technique to stabilize the policy evolution, and enhanced with a difficulty-aware retrospective sampling strategy to incentivize the policy exploration. Experiments on seven long-context document question-answering benchmarks demonstrate that QwenLong-L1-32B outperforms flagship LRMs like OpenAI-o3-mini and Qwen3-235B-A22B, achieving performance on par with Claude-3.7-Sonnet-Thinking, demonstrating leading performance among state-of-the-art LRMs. This work advances the development of practical long-context LRMs capable of robust reasoning across information-intensive environments.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 24 of its 59 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

24 of 59 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Competitive Programming with Large Reasoning Models3 Feb
  2. 2025s1: Simple test-time scaling31 Jan
  3. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model17 Oct

Other references · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  8. 2024OpenAI o1 System Cardcited by 1,244
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2024DeepSeek-V3 Technical Reportcited by 811
  11. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 511
  12. 2025Process Reinforcement through Implicit Rewardscited by 504
9 more
  1. 2025LIMO: Less is More for Reasoningcited by 498
  2. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  3. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 262
  4. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  5. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 173
  6. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 162
  7. 2024How to Train Long-Context Language Models (Effectively)cited by 146
  8. 2025A Comprehensive Survey on Long Context Language Modelingcited by 117
  9. 2025What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secretcited by 98

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 41.

50 papers.

Sort:

By area:

  1. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  2. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 246
  3. 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 38
  4. 2026OPSDL: On-Policy Self-Distillation for Long-Context Language Modelscited by 30
  5. 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 29
  6. 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contextscited by 29
  7. 2026LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewardscited by 15
  8. 2026InfMem: Learning System-2 Memory Control for Long-Context Agentcited by 9
  9. 2026When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoningcited by 9
  10. 2026Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learningcited by 8
  11. 2026Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoningcited by 7
  12. 2026CorpusQA: A 10 Million Token Benchmark for Corpus-Level Analysis and Reasoningcited by 7
38 more
  1. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  2. 2025LongRM: Revealing and Unlocking the Context Boundary of Reward Modelingcited by 5
  3. 2026Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoningcited by 4
  4. 2025RePainter: Empowering E-commerce Object Removal via Spatial-matting Reinforcement Learningcited by 3
  5. 2026Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shapingcited by 3
  6. 2026Market-Bench: Benchmarking Large Language Models on Economic and Trade Competitioncited by 3
  7. 2026LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learningcited by 3
  8. 2026A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillationcited by 3
  9. 2026LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agentscited by 3
  10. 2026EDGE: Error Dependency Graph-Guided Multi-Error Attribution in Multi-Agent LLM Systemscited by 3
  11. 2026LensVLM: Selective Context Expansion for Compressed Visual Representation of Textcited by 2
  12. 2026LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewardscited by 2
  13. 2026OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modelingcited by 2
  14. 2025ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safetycited by 1
  15. 2026Probing How Scalable Table Data Enhances General Long-Context Reasoningcited by 1
  16. 2026A Decomposition Perspective to Long-context Reasoning for LLMscited by 1
  17. 2026RecaLLM: Addressing the Lost-in-Thought Phenomenon with Explicit In-Context Retrievalcited by 1
  18. 2026StoryAlign: Evaluating and Training Reward Models for Story Generationcited by 1
  19. 2026GenesisFunc: Multi-Agent Data Generation for Accurate and Generalizable Function-Callingcited by 1
  20. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learningcited by 1
  21. 2026Evidence-State Rewards for Long-Context Reasoningcited by 1
  22. 2026Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learningcited by 1
  23. 2026REFACT: Adaptive Fact Restatement for Compact and Faithful Chain-of-Thought Reasoningcited by 1
  24. 2026Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Driftcited by 1
  25. 2025ControlMed: Adding Reasoning Control to Medical Language Modelcited by 0
  26. 2026Document Reconstruction Unlocks Scalable Long-Context RLVRcited by 0
  27. 2026Balancing the Reasoning Load: Difficulty-Differentiated Policy Optimization with Length Redistribution for Efficient and Robust Reinforcement Learningcited by 0
  28. 2026Internalized Reasoning for Long-Context Visual Document Understandingcited by 0
  29. 2026Metacognition as Reward: Reinforcing LLM Reasoning via Knowledge and Regulation Signalscited by 0
  30. 2026MMLDSum-LLM: Multimodal Long-Document Summarization with Visual-Alignment and Keyword-Awarecited by 0
  31. 2026PI-Mem: Pushing Long-Context Reasoning to 3.6M Tokens with Parallel-Iterative Memorycited by 0
  32. 2026Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Papercited by 0
  33. 2026Highlight-Then-Summarize: Learning to Compress Evidence for Long-Context Understandingcited by 0
  34. 2026ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agentscited by 0
  35. 2026StateTree: Enhancing Long-Term Dialogue Reasoning via Reinforcement Learningcited by 0
  36. 2026LatentHarness: Learning Latent Actions for Memory and Reasoning via Counterfactual Policy Distillationcited by 0
  37. 2026LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Managementcited by 0
  38. 2026MemDuet: Learning Dual-Process Memory for Long-Context Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
  2. 2026LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards
  3. 2025Training Language Models to Reason Efficiently
  4. 2026Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  6. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failures
  7. 2025R-Horizon: How Far Can Your Large Reasoning Model Really Go in Breadth and Depth?
  8. 2025FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models