mlsanityData to 14 Oct 2026

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?

arXiv 2504.13837 ↗First posted 18 Apr 2025cs.AI cs.CL cs.CVcited by 982

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has recently demonstrated notable success in enhancing the reasoning performance of large language models (LLMs), particularly on mathematics and programming tasks. Similar to how traditional RL helps agents explore and learn new strategies, RLVR is believed to enable LLMs to continuously self-improve, thus acquiring novel reasoning abilities beyond those of the corresponding base models. In this study we critically examine the current state of RLVR by systematically probing the reasoning capability boundaries of RLVR-trained LLMs across various model families, RL algorithms, and math, coding, and visual reasoning benchmarks, using pass@k at large k values as the evaluation metric. Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model. Treating the base model as an upper bound, our quantitative analysis shows that six popular RLVR algorithms perform similarly and remain far from optimal in leveraging the potential of the base model. By contrast, we find that distillation can introduce new reasoning patterns from the teacher and genuinely expand the model's reasoning capabilities. Overall, our findings suggest that current RLVR methods have not yet realized the potential of RL to elicit truly novel reasoning abilities in LLMs. This highlights the need for improved RL paradigms, such as continual scaling and multi-turn agent-environment interaction, to unlock this potential.

Key facts. Cited by 982 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 66 references that are in the map. First posted on arXiv 18 Apr 2025.

Builds on

36 of 66 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining10 Apr
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  8. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 28

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  11. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  12. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
16 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  2. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  3. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  4. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  5. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  6. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  8. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  9. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  10. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  11. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 383
  12. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  13. 2023ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Modelscited by 187
  14. 2025Qwen2.5-1M Technical Reportcited by 164
  15. 2025Rethinking Reflection in Pre-Trainingcited by 43
  16. 2022Value-Consistent Representation Learning for Data-Efficient Reinforcement Learningcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 402; 2026: 580.

982 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  2. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  3. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  4. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  5. 2026Reinforcement Learning via Self-Distillationcited by 332
  6. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  7. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  8. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  9. 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
  10. 2025Learning to Reason under Off-Policy Guidancecited by 281
  11. 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
  12. 2026Self-Distillation Enables Continual Learningcited by 247
38 more
  1. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  2. 2025Learning to Reason without External Rewardscited by 210
  3. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
  4. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  5. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  6. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  7. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  8. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  9. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  10. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 154
  11. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 138
  12. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
  13. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 132
  14. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 130
  15. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 129
  16. 2025GRIT: Teaching MLLMs to Think with Imagescited by 125
  17. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 121
  18. 2026TIP: Token Importance in On-Policy Distillationcited by 104
  19. 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 97
  20. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97
  21. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 92
  22. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
  23. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 90
  24. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedbackcited by 90
  25. 2026Privileged Information Distillation for Language Modelscited by 90
  26. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 89
  27. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learningcited by 87
  28. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoningcited by 84
  29. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
  30. 2025Outcome-based Exploration for LLM Reasoningcited by 83
  31. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 82
  32. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpeningcited by 80
  33. 2025Kevin: Multi-Turn RL for Generating CUDA Kernelscited by 78
  34. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspectivecited by 76
  35. 2026SkillX: Automatically Constructing Skill Knowledge Bases for Agentscited by 76
  36. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  37. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  38. 2026Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervisioncited by 68

50 most cited of 982 All 982 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  2. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  3. 2026Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
  4. 2026Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains
  5. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  6. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement
  7. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  8. 2025Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries