Data to 7 Oct 2026

An Empirical Study on Eliciting and Improving R1-like Reasoning Models

arXiv 2503.04548 ↗First posted 6 Mar 2025cs.CLcited by 78

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

In this report, we present the third technical report on the development of slow-thinking models as part of the STILL project. As the technical pathway becomes clearer, scaling RL training has become a central technique for implementing such reasoning models. We systematically experiment with and document the effects of various factors influencing RL training, conducting experiments on both base models and fine-tuned models. Specifically, we demonstrate that our RL training approach consistently improves the Qwen2.5-32B base models, enhancing both response length and test accuracy. Furthermore, we show that even when a model like DeepSeek-R1-Distill-Qwen-1.5B has already achieved a high performance level, it can be further refined through RL training, reaching an accuracy of 39.33% on AIME 2024. Beyond RL training, we also explore the use of tool manipulation, finding that it significantly boosts the reasoning performance of large reasoning models. This approach achieves a remarkable accuracy of 86.67% with greedy search on AIME 2024, underscoring its effectiveness in enhancing model capabilities. We release our resources at the STILL project website: https://github.com/RUCAIBox/Slow_Thinking_with_LLMs.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 15 of its 22 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

15 of 22 references.

References · 15

2022Training language models to follow instructions with human feedbackcited by 15,712
2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
2024Qwen2.5 Technical Reportcited by 4,713
2020Learning to summarize from human feedbackcited by 2,562
2023A Survey of Large Language Modelscited by 2,537
2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
2024Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Modelscited by 277
2025Training Language Models to Reason Efficientlycited by 242
2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 162
2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77

3 more

2024Enhancing LLM Reasoning with Reward-guided Tree Searchcited by 58
2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluationcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 61; 2026: 17.

78 papers.

Sort:

By area:

2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 322
2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
2025Reasoning with Exploration: An Entropy Perspectivecited by 277
2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 134
2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121

38 more

2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 92
2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
2025Not All Rollouts are Useful: Down-Sampling Rollouts in LLM Reinforcement Learningcited by 84
2025Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Modelscited by 75
2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLMcited by 67
2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Modelscited by 65
2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 43
2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Modelscited by 37
2025Accelerating RL for LLM Reasoning with Optimal Advantage Regressioncited by 23
2025Towards Effective Code-Integrated Reasoningcited by 23
2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?cited by 21
2025Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learningcited by 21
2025DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Modelscited by 19
2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 18
2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
2025AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agentcited by 17
2025Learning from Peers in Reasoning Modelscited by 16
2025DisCO: Reinforcing Large Reasoning Models with Discriminative Constrained Optimizationcited by 16
2025PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Processcited by 16
2025CoRT: Code-integrated Reasoning within Thinkingcited by 15
2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
2025EEPO: Exploration-Enhanced Policy Optimization via Sample-Then-Forgetcited by 14
2025How Much Backtracking is Enough? Exploring the Interplay of SFT and RL in Enhancing LLM Reasoningcited by 13
2025ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contestscited by 12
2025Slow Thinking for Sequential Recommendationcited by 11
2025Surrogate Signals from Format and Length: Reinforcement Learning for Solving Mathematical Problems without Ground Truth Answerscited by 11
2025Mol-R1: Towards Explicit Long-CoT Reasoning in Molecule Discoverycited by 11
2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 11
2025Through the Valley: Path to Effective Long CoT Training for Small Language Modelscited by 10
2025Revisiting Entropy Regularization: Adaptive Coefficient Unlocks Its Potential for LLM Reinforcement Learningcited by 10
2025Shape of Thought: When Distribution Matters More than Correctness in Reasoning Taskscited by 8
2025A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Modelscited by 7
2025Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Modelscited by 5
2025Teaching Language Models to Reason with Toolscited by 5
2025Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoningcited by 5

50 most cited of 78 All 78 (JSON)

Similar, not cited

Close in wording, no citation either way