mlsanityData to 7 Oct 2026

Demystifying Long Chain-of-Thought Reasoning in LLMs

arXiv 2502.03373 ↗First posted 5 Feb 2025cs.CL cs.LGcited by 323

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Scaling inference compute enhances reasoning in large language models (LLMs), with long chains-of-thought (CoTs) enabling strategies like backtracking and error correction. Reinforcement learning (RL) has emerged as a crucial method for developing these capabilities, yet the conditions under which long CoTs emerge remain unclear, and RL training requires careful design choices. In this study, we systematically investigate the mechanics of long CoT reasoning, identifying the key factors that enable models to generate long CoT trajectories. Through extensive supervised fine-tuning (SFT) and RL experiments, we present four main findings: (1) While SFT is not strictly necessary, it simplifies training and improves efficiency; (2) Reasoning capabilities tend to emerge with increased training compute, but their development is not guaranteed, making reward shaping crucial for stabilizing CoT length growth; (3) Scaling verifiable reward signals is critical for RL. We find that leveraging noisy, web-extracted solutions with filtering mechanisms shows strong potential, particularly for out-of-distribution (OOD) tasks such as STEM reasoning; and (4) Core abilities like error correction are inherently present in base models, but incentivizing these skills effectively for complex tasks via RL demands significant compute, and measuring their emergence requires a nuanced approach. These insights provide practical guidance for optimizing training strategies to enhance long CoT reasoning in LLMs. Our code is available at: https://github.com/eddycmu/demystify-long-cot.

Key facts. Cited by 323 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 6 of its 54 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

6 of 54 references.

References · 6

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  4. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  5. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  6. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 233

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 233; 2026: 90.

323 papers.

Sort:

By area:

  1. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  2. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  3. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  4. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  5. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  6. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  7. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  8. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
  9. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  10. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  11. 2025Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and Beyondcited by 175
  12. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
38 more
  1. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 154
  2. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
  3. 2025Phi-4-reasoning Technical Reportcited by 146
  4. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 132
  5. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  6. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  7. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 121
  8. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 115
  9. 2025Small Models Struggle to Learn from Strong Reasonerscited by 109
  10. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  11. 2025Self-Evolving Curriculum for LLM Reasoningcited by 107
  12. 2025Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Modelscited by 104
  13. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoningcited by 101
  14. 2025Self-Training Elicits Concise Reasoning in Large Language Modelscited by 97
  15. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 91
  16. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  17. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  18. 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Modelscited by 81
  19. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
  20. 2025Concise Reasoning via Reinforcement Learningcited by 78
  21. 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectoriescited by 75
  22. 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shapingcited by 74
  23. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn'tcited by 72
  24. 2025SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLMcited by 67
  25. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
  26. 2025Atom of Thoughts for Markov LLM Test-Time Scalingcited by 54
  27. 2025Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVRcited by 53
  28. 2025Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replaycited by 52
  29. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  30. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  31. 2025Rethinking Entropy Interventions in RLVR: An Entropy Change Perspectivecited by 50
  32. 2025Inverse Scaling in Test-Time Computecited by 49
  33. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  34. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  35. 2025GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoningcited by 46
  36. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  37. 2025Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Modelscited by 45
  38. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 44

50 most cited of 323 All 323 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs
  2. 2025Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought
  3. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
  4. 2025Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
  5. 2025Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL
  6. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  7. 2025Rectifying LLM Thought from Lens of Optimization
  8. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation