mlsanityData to 14 Oct 2026

Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs

arXiv 2506.14245 ↗First posted 17 Jun 2025cs.AI cs.CLcited by 289

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Recent advancements in long chain-of-thought (CoT) reasoning, particularly through the Group Relative Policy Optimization algorithm used by DeepSeek-R1, have led to significant interest in the potential of Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Models (LLMs). While RLVR promises to improve reasoning by allowing models to learn from free exploration, there remains debate over whether it truly enhances reasoning abilities or simply boosts sampling efficiency. This paper systematically investigates the impact of RLVR on LLM reasoning. We revisit Pass@K experiments and demonstrate that RLVR can extend the reasoning boundary for both mathematical and coding tasks. This is supported by our introduction of a novel evaluation metric, CoT-Pass@K, which captures reasoning success by accounting for both the final answer and intermediate reasoning steps. Furthermore, we present a theoretical framework explaining RLVR's incentive mechanism, demonstrating how it can encourage correct reasoning even when rewards are based solely on answer correctness. Our analysis of RLVR's training dynamics reveals that it incentivizes correct reasoning early in the process, with substantial improvements in reasoning quality confirmed through extensive evaluations. These findings provide strong evidence of RLVR's potential to enhance LLM reasoning, offering valuable insights into its mechanisms and performance improvements.

Key facts. Cited by 289 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 50 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

37 of 50 references.

Closest ancestors · 21

Same direction, newest first

  1. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models30 May
  2. 2025NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning23 May
  3. 2025AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning22 May
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  5. 2025Learning to Reason under Off-Policy Guidance21 Apr
  6. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  7. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization8 Apr
  8. 2025Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning6 Apr
  9. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  11. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  12. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
9 more
  1. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  2. 2025Process Reinforcement through Implicit Rewards3 Feb
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  4. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov
  5. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  6. 2023Let's Verify Step by Step31 May
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  8. 2022Solving math word problems with process- and outcome-based feedback25 Nov
  9. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 16

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  8. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  9. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 302
  10. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  11. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  12. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
4 more
  1. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  2. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 90
  3. 2025Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoningcited by 11
  4. 2025Large Language Models' Reasoning Stalls: An Investigation into the Capabilities of Frontier Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 61; 2026: 228.

289 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025The Invisible Leash: Why RLVR May or May Not Escape Its Origincited by 97
  3. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  4. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Frameworkcited by 62
  5. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 62
  6. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 54
  7. 2025ExGRPO: Learning to Reason from Experiencecited by 53
  8. 2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agentcited by 48
  9. 2025Online Rubrics Elicitation from Pairwise Comparisonscited by 46
  10. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 45
  11. 2025QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentationcited by 44
  12. 2025From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Onescited by 42
38 more
  1. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 41
  2. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVRcited by 39
  3. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMscited by 34
  4. 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 33
  5. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 31
  6. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 31
  7. 2025Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspectivecited by 28
  8. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Modelscited by 27
  9. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 26
  10. 2025RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agentscited by 25
  11. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 25
  12. 2026Reasoning Models Struggle to Control their Chains of Thoughtcited by 25
  13. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 22
  14. 2026Composer 2 Technical Reportcited by 22
  15. 2025Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learningcited by 21
  16. 2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datacited by 21
  17. 2025HunyuanOCR Technical Reportcited by 20
  18. 2026Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMscited by 20
  19. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  20. 2026Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulationcited by 19
  21. 2026Does Your Reasoning Model Implicitly Know When to Stop Thinking?cited by 19
  22. 2026LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewardscited by 15
  23. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  24. 2025Reinforcement Learning with Inverse Rewards for World Model Post-trainingcited by 13
  25. 2025CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoningcited by 13
  26. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic Viewcited by 13
  27. 2025Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learningcited by 13
  28. 2025Propose, Solve, Verify: Self-Play Through Formal Verificationcited by 13
  29. 2026Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoningcited by 13
  30. 2026Shaping capabilities with token-level data filteringcited by 13
  31. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 12
  32. 2025Clip-Low Increases Entropy and Clip-High Decreases Entropy in Reinforcement Learning of Large Language Modelscited by 12
  33. 2026Group Distributionally Robust Optimization-Driven Reinforcement Learning for LLM Reasoningcited by 12
  34. 2026Learning to Hint for Reinforcement Learningcited by 12
  35. 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11
  36. 2026Future-as-Label: Scalable Supervision from Real-World Outcomescited by 11
  37. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Modelscited by 11
  38. 2026Safe and Scalable Web Agent Learning via Recreated Websitescited by 11

50 most cited of 289 All 289 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  2. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  3. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  4. 2025From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
  5. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  6. 2026On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
  7. 2025Language Models that Think, Chat Better
  8. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains