# Reinforcement learning for reasoning

Area: [LLM reasoning](https://mlsanity.com/lr/index.md). HTML: https://mlsanity.com/lr/reinforcement-learning-for-reasoning/

## Key facts

- Reinforcement learning for reasoning: 1,653 papers on arXiv since 2017 (data through 14 Oct 2026).
- 570 of its 1,653 mapped papers (34%) cite “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning” (2025, arXiv 2501.12948).
- 542 of its 1,653 mapped papers (33%) cite “Proximal Policy Optimization Algorithms” (2017, arXiv 1707.06347).
- 130 new papers 17 Sep–14 Oct.

## Most cited here

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025), cited by 570 papers here
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017), cited by 542 papers here
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025), cited by 450 papers here
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023), cited by 309 papers here
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023), cited by 292 papers here
- [Understanding R1-Zero-Like Training: A Critical Perspective](https://mlsanity.com/paper/2503.20783/) (2025), cited by 265 papers here
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://mlsanity.com/paper/2312.08935/) (2023), cited by 190 papers here
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025), cited by 188 papers here
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025), cited by 175 papers here
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://mlsanity.com/paper/2411.15124/) (2024), cited by 158 papers here

## New (17 Sep–14 Oct)

- [VASPO: Self-Distillation with Privileged Value Estimation](https://mlsanity.com/paper/2610.20516/) (2026-10-14)
- [Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling](https://mlsanity.com/paper/2610.20497/) (2026-10-14)
- [When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO](https://mlsanity.com/paper/2610.20089/) (2026-10-14)
- [Learning Perception to Action Rules through Reinforcement Learning for Theory-of-Mind Reasoning](https://mlsanity.com/paper/2610.19796/) (2026-10-14)
- [C3: Rethinking Process Supervision via Confidence-Guided Credit Correction in Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2610.19695/) (2026-10-14)
- [Function-Space Temporal Self-Distillation for Policy Optimization](https://mlsanity.com/paper/2610.19687/) (2026-10-14)
- [Where to Start, How to Move: Analyzing Curricula under GRPO for Long-Horizon LLM Agents](https://mlsanity.com/paper/2610.19349/) (2026-10-14)
- [ProAS: A Progress-Aware Advantage Shaping Framework for Multi-Turn Agentic Reinforcement Learning](https://mlsanity.com/paper/2610.18683/) (2026-10-13)
- [Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR](https://mlsanity.com/paper/2610.17824/) (2026-10-13)
- [Explore with Memory, Learn without It: In-Context Exploration and Self-Distillation for LLM Reasoning](https://mlsanity.com/paper/2610.17734/) (2026-10-13)
- [RELAY: Selective Self-Distillation with Adaptive Hand-off for Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2610.17623/) (2026-10-13)
- [TPPO: Tracking-Value PPO For Language Model Reasoning](https://mlsanity.com/paper/2610.16794/) (2026-10-12)
- [Double-Rollout Reinforcement Learning with Sampling-based Reward Estimation](https://mlsanity.com/paper/2610.16593/) (2026-10-12)
- [Spectral-Driven Adaptive-Rank Optimizer for Large Language Model Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2610.16533/) (2026-10-12)
- [Growing and Transferring Pathways for LLM Reinforcement Learning](https://mlsanity.com/paper/2610.16489/) (2026-10-12)
- [Variance-Optimal State Resampling for Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2610.15888/) (2026-10-11)
- [Verify, Act, Distill: Turning Test-Time Compute into Test-Time Adaptation for Vision-Language-Action Models](https://mlsanity.com/paper/2610.14885/) (2026-10-10)
- [Reward-Aligned Distillation](https://mlsanity.com/paper/2610.14572/) (2026-10-10)
- [SAT-RL: Training Language Models for Logical Reasoning with Symbolic Solvers](https://mlsanity.com/paper/2610.14402/) (2026-10-09)
- [A Critical Look at Power Sampling when Test Time Sharpening LLMs](https://mlsanity.com/paper/2610.13653/) (2026-10-09)
- [MENTOR: Learning from Failure with On-Policy Pedagogical Intervention](https://mlsanity.com/paper/2610.13594/) (2026-10-09)
- [Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits](https://mlsanity.com/paper/2610.13438/) (2026-10-09)
- [AdaC-GRPO: Towards Mastering Long-Horizon Visual Sudoku Puzzles via Adaptive Curriculum Learning](https://mlsanity.com/paper/2610.13184/) (2026-10-09)
- [Trajectory-Aware Policy Optimization: Closing the State Distribution Gap in Long-Horizon LLM RL](https://mlsanity.com/paper/2610.13020/) (2026-10-09)
- [Generalized Proximal Policy Optimization: Unifying LLM RL Algorithms from the Perspective of Divergence](https://mlsanity.com/paper/2610.13015/) (2026-10-09)
- [TreeHCA: Hindsight-Guided Exploration and Credit Assignment for Multi-Turn LLM Agents](https://mlsanity.com/paper/2610.12828/) (2026-10-09)
- [GRPODropout: Less is More for Online Reinforcement Learning Rollouts](https://mlsanity.com/paper/2610.11854/) (2026-10-08)
- [A 3D Characterization Framework for Intelligent Sequential Decision Making](https://mlsanity.com/paper/2610.11696/) (2026-10-08)
- [Residual Advantage: Student-Relative Teacher Guidance for RL with Verifiable Rewards](https://mlsanity.com/paper/2610.11519/) (2026-10-08)
- [Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimization](https://mlsanity.com/paper/2610.11502/) (2026-10-08)