mlsanityData to 14 Oct 2026

A Survey of Reinforcement Learning for Large Reasoning Models

arXiv 2509.08827 ↗First posted 10 Sep 2025cs.CL cs.AI cs.LGcited by 180

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

In this paper, we survey recent advances in Reinforcement Learning (RL) for reasoning with Large Language Models (LLMs). RL has achieved remarkable success in advancing the frontier of LLM capabilities, particularly in addressing complex logical tasks such as mathematics and coding. As a result, RL has emerged as a foundational methodology for transforming LLMs into LRMs. With the rapid progress of the field, further scaling of RL for LRMs now faces foundational challenges not only in computational resources but also in algorithm design, training data, and infrastructure. To this end, it is timely to revisit the development of this domain, reassess its trajectory, and explore strategies to enhance the scalability of RL toward Artificial SuperIntelligence (ASI). In particular, we examine research applying RL to LLMs and LRMs for reasoning abilities, especially since the release of DeepSeek-R1, including foundational components, core problems, training resources, and downstream applications, to identify future opportunities and directions for this rapidly evolving area. We hope this review will promote future research on RL for broader reasoning models. Github: https://github.com/TsinghuaC3I/Awesome-RL-for-LRMs

Key facts. Cited by 180 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 557 of its 751 references that are in the map. First posted on arXiv 10 Sep 2025.

Builds on

557 of 751 references.

Closest ancestors · 147

Same direction, newest first

  1. 2025Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning8 Sep
  2. 2025Advancing SLM Tool-Use Capability using Reinforcement Learning3 Sep
  3. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learning3 Sep
  4. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning2 Sep
  5. 2025AR2: Adversarial Reinforcement Learning for Abstract Reasoning in Large Language Models27 Aug
  6. 2025TreePO: Bridging the Gap of Policy Optimization and Efficacy and Inference Efficiency with Heuristic Tree-based Modeling24 Aug
  7. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning23 Aug
  8. 2025Recall-Extend Dynamics: Enhancing Small Language Models through Controlled Exploration and Refined Offline Integration21 Aug
  9. 2025AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning19 Aug
  10. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR19 Aug
  11. 2025Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration19 Aug
  12. 2025G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance18 Aug
38 more
  1. 2025SSPO: Self-traced Step-wise Preference Optimization for Process Supervision and Reasoning Compression18 Aug
  2. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models14 Aug
  3. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning13 Aug
  4. 2025InternBootcamp: Boosting LLM Reasoning with Verifiable Task Scaling12 Aug
  5. 2025Part I: Tricks or Traps? A Deep Dive into RL for LLM Reasoning11 Aug
  6. 2025From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR11 Aug
  7. 2025Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models7 Aug
  8. 2025SPaCe: Unlocking Sample-Efficient Large Language Models Training With Self-Pace Curriculum Learning7 Aug
  9. 2025GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy6 Aug
  10. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Models6 Aug
  11. 2025VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training5 Aug
  12. 2025CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment4 Aug
  13. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models1 Aug
  14. 2025Good Learners Think Their Thinking: Generative PRM Makes Large Reasoning Model More Efficient Math Learner31 Jul
  15. 2025Libra: Assessing and Improving Reward Model by Learning to Think29 Jul
  16. 2025Geometric-Mean Policy Optimization28 Jul
  17. 2025Agentic Reinforced Policy Optimization26 Jul
  18. 2025Predictive Scaling Laws for Efficient GRPO Training of Large Reasoning Models24 Jul
  19. 2025Dynamic and Generalizable Process Reward Modeling23 Jul
  20. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learning23 Jul
  21. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR21 Jul
  22. 2025Omni-Thinker: Scaling Multi-Task RL in LLMs with Hybrid Reward and Task Scheduling20 Jul
  23. 2025RLEP: Reinforcement Learning with Experience Replay for LLM Reasoning10 Jul
  24. 2025First Return, Entropy-Eliciting Explore9 Jul
  25. 2025Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model9 Jul
  26. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models5 Jul
  27. 2025StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reason3 Jul
  28. 2025Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling2 Jul
  29. 2025Self-Guided Process Reward Optimization with Redefined Step-wise Advantage for Process Reinforcement Learning2 Jul
  30. 2025EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework27 Jun
  31. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling25 Jun
  32. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning24 Jun
  33. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs23 Jun
  34. 2025Confucius3-Math: A Lightweight High-Performance Reasoning LLM for Chinese K-12 Mathematics Learning23 Jun
  35. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers23 Jun
  36. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoning20 Jun
  37. 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning20 Jun
  38. 2025Truncated Proximal Policy Optimization18 Jun

50 newest of 147 All 147 (JSON)

Other references · 410

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  8. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  10. 2023A Survey of Large Language Modelscited by 2,576
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  12. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
38 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  3. 2024Mixtral of Expertscited by 1,823
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  9. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  10. 2023Mastering Diverse Domains through World Modelscited by 1,253
  11. 2024OpenAI o1 System Cardcited by 1,244
  12. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  13. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  14. 2025Qwen-Image Technical Reportcited by 1,044
  15. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  16. 2025Large Language Diffusion Modelscited by 912
  17. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  18. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  19. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  20. 2025Group Sequence Policy Optimizationcited by 835
  21. 2024DeepSeek-V3 Technical Reportcited by 811
  22. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  23. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  24. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779
  25. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  26. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  27. 2024Self-Rewarding Language Modelscited by 681
  28. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  29. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  30. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  31. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  32. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  33. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  34. 2025Kimi K2: Open Agentic Intelligencecited by 567
  35. 2025Dream 7B: Diffusion Large Language Modelscited by 544
  36. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  37. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  38. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513

50 most cited of 410 All 410 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 62; 2026: 118.

180 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  3. 2025ExGRPO: Learning to Reason from Experiencecited by 53
  4. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspectivecited by 52
  5. 2025FLEX: Continuous Agent Evolution via Forward Learning from Experiencecited by 52
  6. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  7. 2025BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clippingcited by 47
  8. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Livecited by 44
  9. 2025FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlightingcited by 42
  10. 2025FlowRL: Matching Reward Distributions for LLM Reasoningcited by 40
  11. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 39
  12. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
38 more
  1. 2026Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpeningcited by 31
  2. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 31
  3. 2026Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Modelscited by 30
  4. 2025Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoningcited by 29
  5. 2025Rethinking Entropy Regularization in Large Reasoning Modelscited by 26
  6. 2025Nudging the Boundaries of LLM Reasoningcited by 26
  7. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubricscited by 26
  8. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
  9. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Modelscited by 23
  10. 2025A Survey on Agentic Multimodal Large Language Modelscited by 23
  11. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Modelscited by 23
  12. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 22
  13. 2025RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learningcited by 21
  14. 2025When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RLcited by 21
  15. 2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Densecited by 20
  16. 2025DiFFPO: Training Diffusion LLMs to Reason Fast and Furious via Reinforcement Learningcited by 19
  17. 2025Rethinking Sample Polarity in Reinforcement Learning with Verifiable Rewardscited by 19
  18. 2025DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Modelscited by 19
  19. 2026Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillationcited by 19
  20. 2025Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learningcited by 17
  21. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  22. 2025Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friendscited by 15
  23. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 15
  24. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  25. 2026LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewardscited by 15
  26. 2025Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Modelscited by 13
  27. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  28. 2026Rethinking Reward Supervision: Rubric-Conditioned Self-Distillationcited by 13
  29. 2025Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learningcited by 12
  30. 2025Advantage Shaping as Surrogate Reward Maximization: Unifying Pass@K Policy Gradientscited by 11
  31. 2026Reward-free Alignment for Conflicting Objectivescited by 11
  32. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 11
  33. 2026To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Modelscited by 11
  34. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 11
  35. 2025Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoningcited by 10
  36. 2026Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoningcited by 10
  37. 2025Reinforcement Learning Fine-Tuning Enhances Activation Intensity and Diversity in the Internal Circuitry of LLMscited by 9
  38. 2025h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learningcited by 9

50 most cited of 180 All 180 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  2. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key
  3. 2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
  4. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoning
  5. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs
  6. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learning
  7. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study
  8. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models