mlsanityData to 7 Oct 2026

Reinforcement learning for reasoning

1,653 papers10 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: reinforcement, reinforcement learning, optimization, policy, policy optimization

Key facts. Reinforcement learning for reasoning: 1,653 papers on arXiv since 2017 (data through 7 Oct 2026). 570 of its 1,653 mapped papers (34%) cite “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning” (2025, arXiv 2501.12948). 542 of its 1,653 mapped papers (33%) cite “Proximal Policy Optimization Algorithms” (2017, arXiv 1707.06347). 93 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

93 new papers, 17 Sep – 7 Oct; 10 this week.

  1. This week · 10
  2. Stochastic Teacher Intervention for Agentic On-Policy Distillation
  3. RollVerify: Bridging Efficiency and Accuracy in Long-Tail Rollout Reinforcement Learning
  4. BoT-GRPO: Efficient Process-Reward RL for Reasoning via Bag-of-Token Aggregation
  5. Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
  6. What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training
  7. RELACE: retrospective likelihood-based action credit estimation for long-horizon language agents
  8. Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding
  9. Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
  10. Evolving in Thought Space: Training a Small Model at Test Time Unlocks Better Discoveries also fits: Reasoning for agents, planning and code
  11. Transfer-Stratified On-Policy Distillation for RL-Improved Reasoning Teachers
  12. Earlier · 83
All 93
  1. How Should Teachers Be Prepared? RL on Student-Induced States for On-Policy Distillation also fits: Prompting and chain-of-thought reasoning
  2. DiVeR: Decision-Critical Verifier Learning for VLA Test-Time Scaling also fits: Spatial and embodied reasoning
  3. Prompt Dominance and Asymmetric Verifier Costs: Empirical Ablations of GRPO at 1B Scale on GSM8K
  4. Residual Visual Credit Optimization: Conserved Evidence Routing for Multimodal Reinforcement Learning
  5. Rewrite What Matters: Adaptive Multilingual Query Rewriting for Reasoning via Agentic Reinforcement Learning also fits: Reasoning for agents, planning and code
  6. ResOPD: Tail Residualization for Sparse On-Policy Distillation
  7. Learning to Decide, Not to Reason: Parameter-Efficient Decision Operators via Low-Rank Activation Steering
  8. DiffGate: Difficulty-Gated Teacher Guidance for On-Policy Distillation
  9. Hierarchical Credit Assignment for RLVR on Fused Gromov-Wasserstein Geometry
  10. How RL Reshapes LLM Reasoning: Transferability, Coverage, and Scaling Laws
  11. PB-GRPO: Learning Socially Adaptive LLM Agents from Persona-Driven Simulation with Preference-Batched GRPO
  12. Representation-Aligned Auxiliary Supervision for Language Model Adaptation also fits: Test-time compute and efficient thinking
  13. Beyond Entropy: Self-Diagnostic Multi-Role Token Optimization for Video Reasoning also fits: Multimodal reasoning
  14. Follow the Winners: Conservative Policy Improvement with the Cross-Entropy Method for Critic-Free RFT
  15. All Work And No Play Makes Jack a Dull Boy: Understanding and Preventing Catastrophic Strategy Collapse in RLVR
  16. Learning from Evolving Errors: Adaptive Iterative Repair for On-Policy Distillation
  17. CARM: Cancellation-Aware Response Masking for LLM Reinforcement Learning
  18. On Language Drift during RLVR Post-Training
  19. Asynchronous LLM Post-Training: Group-Mass Capping and Convergence Analysis
  20. Function-Structured Reinforcement Learning with Executable Verifiers for Mathematical Reasoning
  21. Range-GRPO: Policy Optimization via Pairwise Relations among Reward Intervals
  22. Rethinking Probability-Based Reinforcement Learning From Posterior Concentration
  23. Improving Math Reasoning through Value-guided Informative Search
  24. Adapter Thickets: Splitting an RLVR Budget Beats Concentrating It
  25. What Do Verifiable Rewards Teach Video-Language Models About Time? A Controlled Multi-Model Study
  26. Semifactual Credit-Augmented Policy Optimization
  27. Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation also fits: Test-time compute and efficient thinking
  28. Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimization
  29. GRPO Training Dynamics for Small Language Models
  30. Learning Process Rewards via Reasoning State Propagation
  31. Frame Differential On-Policy Self-Distillation for Video Reasoning also fits: Multimodal reasoning
  32. Mitigating the Length-Scaling Tax with Online Distillation also fits: Test-time compute and efficient thinking
  33. OpenJev-RLCD: A Working RLCD Implementation
  34. The Weakest Link: Distilling LLM Reasoning with Worst-Case Constrained Reinforcement Learning also fits: Prompting and chain-of-thought reasoning
  35. Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO
  36. Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR
  37. Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR
  38. Looped Actor: Depth-Recurrent Reasoning Models for Reinforcement Learning
  39. STAR-GRPO: Canonical Anchoring and Reliability-First Advantages against Representation-Dependent Reward Hacking
  40. CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning
  41. Towards Better Training Signal: Advantage Clipped Policy Optimization
  42. VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction
  43. Inducing Process Supervision from Outcome-Only Reinforcement Learning
  44. Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models also fits: Reasoning for agents, planning and code
  45. Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
  46. ABC: Advantage-Based Control Variates for Reinforcement Learning with Verifiable Rewards
  47. Distillation Defenses Easily Break After Reinforcement Learning
  48. An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
  49. Frontier Learning: Training LLM Reasoners at the Edge of Capability
  50. PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents
  51. Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
  52. MaPP: A Unified Marginalized Posterior-Predictive Framework for Data-Efficient RLVR
  53. Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
  54. Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts
  55. Learning to Steer, Steering to See: Unveiling the Geometry of RLVR in Large Language Models via Trainable Vectors
  56. Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoning
  57. Selecting Diverse SFT Traces Improves Post-RL Generalization
  58. Allspark: Weak to Strong Transfer via Alternating Chain of Thought also fits: Test-time compute and efficient thinking
  59. T5: Twin-Critic Training for Token-Level Thoughts in Reinforcement Mid-Training
  60. Scaling Properties of Same-Family On-Policy Distillation
  61. Expected Reasoning-Step Return Unifies On-Policy Learning from Rewards and Teachers also fits: Prompting and chain-of-thought reasoning
  62. From Outcomes to Strategies: Learning Strategy Utility for Mathematical Reasoning
  63. On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models also fits: Test-time compute and efficient thinking
  64. Rethinking Training-Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It
  65. RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning
  66. Strategically Diverse Sampling for Self-Training also fits: Test-time compute and efficient thinking
  67. IterSynth: Rethinking Deep Search Agents via Role-Decoupled Iterative Synthesis also fits: Reasoning for agents, planning and code
  68. CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment
  69. SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL
  70. Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning
  71. RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory
  72. When and Where to Trust the Teacher: Unifying On-Policy Distillation and GRPO through Entropy-Calibrated Credit Assignment
  73. DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment
  74. Reinforcement Learning with Decomposed Subtasks
  75. Transcribe, Translate, and Optimize: Joint Reward Learning for Speech Translation
  76. Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA
  77. Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
  78. Information-Time Proximal Policy Optimization
  79. Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcement
  80. Calibrating Teacher--Student Discrepancy for On-Policy Distillation also fits: Prompting and chain-of-thought reasoning
  81. GVPO++: Group Variance Policy Optimization for LLM Post-Training and On-Policy Distillation
  82. UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learning also fits: Reasoning for agents, planning and code
  83. EPIG-Tree: Compute-Optimal Branching for Gradient-Efficient Reinforcement Learning

Foundations

Most cited by papers here

  1. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025cited by 570 here
  2. Proximal Policy Optimization Algorithms2017cited by 542 here
  3. DAPO: An Open-Source LLM Reinforcement Learning System at Scale2025cited by 450 here
  4. Direct Preference Optimization: Your Language Model is Secretly a Reward Model2023cited by 309 here
  5. Let's Verify Step by Step2023cited by 292 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 315 here
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 169 here
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 221 here
  4. 2022Solving math word problems with process- and outcome-based feedbackcited by 139 here
  5. 2023Let's Verify Step by Stepcited by 292 here
  6. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 190 here
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 773 here
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 570 here
16 more it builds on, not in this chain
  1. 2017Proximal Policy Optimization Algorithmscited by 542 here
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 127 here
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 309 here
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 123 here
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 114 here
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 158 here
  7. 2024OpenAI o1 System Cardcited by 222 here
  8. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 110 here
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 175 here
  10. 2025Process Reinforcement through Implicit Rewardscited by 123 here
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 450 here
  12. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 130 here
  13. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 265 here
  14. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 147 here
  15. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 188 here
  16. 2025Qwen3 Technical Reportcited by 319 here

The 24 most cited of 2,675.

Browse

30 of 1,653, ranked 6–35

  1. 6Understanding R1-Zero-Like Training: A Critical Perspective2025265
  2. 7Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations2023190
  3. 8Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?2025188
  4. 9Kimi k1.5: Scaling Reinforcement Learning with LLMs2025175
  5. 10Tulu 3: Pushing Frontiers in Open Language Model Post-Training2024158
  6. 11Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model2025147
  7. 12Solving math word problems with process- and outcome-based feedback2022139
  8. 13SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild2025130
  9. 14Process Reinforcement through Implicit Rewards2025123
  10. 15The Lessons of Developing Process Reward Models in Mathematical Reasoning2025110
All 30
  1. 16The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models2025109
  2. 17Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning2025103
  3. 18Improve Mathematical Reasoning in Language Models by Automated Process Supervision202489
  4. 19Learning to Reason under Off-Policy Guidance202586
  5. 20VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks202585
  6. 21Constitutional AI: Harmlessness from AI Feedback202281
  7. 22REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization202579
  8. 23Reinforcement Learning for Reasoning in Large Language Models with One Training Example202577
  9. 24Reasoning with Exploration: An Entropy Perspective202576
  10. 25Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning202473
  11. 26Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs202572
  12. 27Generative Verifiers: Reward Modeling as Next-Token Prediction202470
  13. 28Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning202567
  14. 29ProcessBench: Identifying Process Errors in Mathematical Reasoning202462
  15. 30ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models202561
  16. 31Spurious Rewards: Rethinking Training Signals in RLVR202553
  17. 32Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs202553
  18. 33ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search202451
  19. 34Free Process Rewards without Process Labels202451
  20. 35The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning202548