mlsanityData to 7 Oct 2026

A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility

arXiv 2504.07086 ↗First posted 9 Apr 2025cs.LG cs.CLcited by 89

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Reasoning has emerged as the next major frontier for language models (LMs), with rapid advances from both academic and industrial labs. However, this progress often outpaces methodological rigor, with many evaluations relying on benchmarking practices that lack transparency, robustness, or statistical grounding. In this work, we conduct a comprehensive empirical study and find that current mathematical reasoning benchmarks are highly sensitive to subtle implementation choices--including decoding parameters, random seeds, prompt formatting, and even hardware and software configurations. Performance gains reported in recent studies frequently hinge on unclear comparisons or unreported sources of variance. To address these issues, we propose a standardized evaluation framework with clearly defined best practices and reporting standards. Using this framework, we reassess recent methods and find that most reinforcement learning (RL) approaches yield only modest improvements--far below prior claims--and are prone to overfitting, especially on small-scale benchmarks like AIME'24. In contrast, supervised finetuning (SFT) methods show consistently stronger generalization in the settings we study. To foster reproducibility, we release all code, prompts, and model outputs, for reasoning benchmarks, establishing more rigorous foundations for future work.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 86 of its 133 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

86 of 133 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyond13 Mar
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Models24 Feb
  3. 2024Are Your LLMs Capable of Stable Reasoning?17 Dec

Other references · 83

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2025s1: Simple test-time scalingcited by 1,311
  10. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
38 more
  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  4. 2017Deep Reinforcement Learning that Matterscited by 727
  5. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  6. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  7. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  8. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  9. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  10. 2025LIMO: Less is More for Reasoningcited by 488
  11. 2025Process Reinforcement through Implicit Rewardscited by 478
  12. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  13. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  14. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  15. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  16. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  17. 2020A Metric Learning Reality Checkcited by 317
  18. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 281
  19. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
  20. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learningcited by 260
  21. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RLcited by 244
  22. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 175
  23. 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
  24. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  25. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systemscited by 162
  26. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 162
  27. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  28. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 150
  29. 2020With Little Power Comes Great Responsibilitycited by 138
  30. 2025LIMR: Less is More for RL Scalingcited by 136
  31. 2021The Benchmark Lotterycited by 133
  32. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
  33. 2018Troubling Trends in Machine Learning Scholarshipcited by 107
  34. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Modelscited by 93
  35. 2024On Designing Effective RL Reward at Training Time for LLM Reasoningcited by 87
  36. 2024Non-Determinism of "Deterministic" LLM Settingscited by 84
  37. 2025NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questionscited by 81
  38. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79

50 most cited of 83 All 83 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 63; 2026: 26.

89 papers.

Sort:

By area:

  1. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  2. 2025Generalizing Verifiable Instruction Followingcited by 178
  3. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  4. 2025Phi-4-reasoning Technical Reportcited by 146
  5. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 83
  6. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
  7. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  8. 2025Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inferencecited by 47
  9. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 39
  10. 2025Learning to Reason Across Parallel Samples for LLM Reasoningcited by 32
  11. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 30
  12. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
38 more
  1. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 19
  2. 2025How to Train Your LLM Web Agent: A Statistical Diagnosiscited by 18
  3. 2025Language Models Can Learn from Verbal Feedback Without Scalar Rewardscited by 17
  4. 2025Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoningcited by 15
  5. 2025Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scalingcited by 15
  6. 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
  7. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  8. 2025FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inferencecited by 14
  9. 2025Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoningcited by 14
  10. 2025Scalpel vs. Hammer: GRPO Amplifies Existing Capabilities, SFT Replaces Themcited by 14
  11. 2025The Debate on RLVR Reasoning Capability Boundary: Shrinkage, Expansion, or Both? A Two-Stage Dynamic Viewcited by 13
  12. 2025REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Oncecited by 12
  13. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMscited by 12
  14. 2025Context Is What You Need: The Maximum Effective Context Window for Real World Limits of LLMscited by 11
  15. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 11
  16. 2025DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Searchcited by 11
  17. 2025Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Modelscited by 10
  18. 2025From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimizationcited by 10
  19. 2025Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thoughtcited by 10
  20. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspectivecited by 9
  21. 2025Let it Calm: Exploratory Annealed Decoding for Verifiable Reinforcement Learningcited by 9
  22. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  23. 2025Variational Reasoning for Language Modelscited by 8
  24. 2025Mapping Post-Training Forgetting in Language Models at Scalecited by 8
  25. 2026Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVRcited by 8
  26. 2026DatBench: Discriminative, Faithful, and Efficient VLM Evaluationscited by 7
  27. 2026Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptationcited by 7
  28. 2025ScaleDiff: Scaling Difficult Problems for Advanced Mathematical Reasoningcited by 6
  29. 2025ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoningcited by 6
  30. 2026Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoningcited by 6
  31. 2025SHARP: Synthesizing High-quality Aligned Reasoning Problems for Large Reasoning Models Reinforcement Learningcited by 5
  32. 2025Dissecting Long-Chain-of-Thought Reasoning Models: An Empirical Studycited by 5
  33. 2025RAST: Reasoning Activation in LLMs via Small-model Transfercited by 5
  34. 2025Uncertainty Under the Curve: A Sequence-Level Entropy Area Metric for Reasoning LLMcited by 5
  35. 2025The Hitchhiker's Guide to Program Analysis, Part II: Deep Thoughts by LLMscited by 4
  36. 2025Discovering Hierarchical Latent Capabilities of Language Models via Causal Representation Learningcited by 4
  37. 2025ICPO: Intrinsic Confidence-Driven Group Relative Preference Optimization for Efficient Reinforcement Learningcited by 4
  38. 2025Closing the Data Loop: Using OpenDataArena to Engineer Superior Training Datasetscited by 4

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models
  2. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  3. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
  4. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains
  5. 2025Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
  6. 2025Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection
  7. 2025Code-enabled language models can outperform reasoning models on diverse tasks
  8. 2025Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning