mlsanityData to 7 Oct 2026

Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

arXiv 2503.09567 ↗First posted 12 Mar 2025cs.AI cs.CLcited by 381

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Recent advancements in reasoning with large language models (RLLMs), such as OpenAI-O1 and DeepSeek-R1, have demonstrated their impressive capabilities in complex domains like mathematics and coding. A central factor in their success lies in the application of long chain-of-thought (Long CoT) characteristics, which enhance reasoning abilities and enable the solution of intricate problems. However, despite these developments, a comprehensive survey on Long CoT is still lacking, limiting our understanding of its distinctions from traditional short chain-of-thought (Short CoT) and complicating ongoing debates on issues like "overthinking" and "inference-time scaling." This survey seeks to fill this gap by offering a unified perspective on Long CoT. (1) We first distinguish Long CoT from Short CoT and introduce a novel taxonomy to categorize current reasoning paradigms. (2) Next, we explore the key characteristics of Long CoT: deep reasoning, extensive exploration, and feasible reflection, which enable models to handle more complex tasks and produce more efficient, coherent outcomes compared to the shallower Short CoT. (3) We then investigate key phenomena such as the emergence of Long CoT with these characteristics, including overthinking, and inference-time scaling, offering insights into how these processes manifest in practice. (4) Finally, we identify significant research gaps and highlight promising future directions, including the integration of multi-modal reasoning, efficiency improvements, and enhanced knowledge frameworks. By providing a structured overview, this survey aims to inspire future research and further the development of logical reasoning in artificial intelligence.

Key facts. Cited by 381 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 583 of its 1,150 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

583 of 1,150 references.

Closest ancestors · 35

Same direction, newest first

  1. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervision28 Feb
  2. 2025Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking27 Feb
  3. 2025Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?26 Feb
  4. 2025Chain of Draft: Thinking Faster by Writing Less25 Feb
  5. 2025Unveiling and Causalizing CoT: A Causal Pespective25 Feb
  6. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning25 Feb
  7. 2025CoT2Align: Cross-Chain of Thought Distillation via Optimal Transport Alignment for Language Models with Different Tokenizers24 Feb
  8. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps20 Feb
  9. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models18 Feb
  10. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs17 Feb
  11. 2025Is Depth All You Need? An Exploration of Iterative Reasoning in LLMs15 Feb
  12. 2025Distillation Scaling Laws12 Feb
23 more
  1. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!11 Feb
  2. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb
  3. 2025Emergent Response Planning in LLMs10 Feb
  4. 2025Dynamic Chain-of-Thought: Towards Adaptive Deep Reasoning7 Feb
  5. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning7 Feb
  6. 2025BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillation6 Feb
  7. 2025Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers4 Feb
  8. 2025Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillation2 Feb
  9. 2025Zero-Shot Verification-guided Chain of Thoughts21 Jan
  10. 2025RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?20 Jan
  11. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought8 Jan
  12. 2025CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis3 Jan
  13. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  14. 2024Thinking LLMs: General Instruction Following with Thought Generation14 Oct
  15. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning18 Sep
  16. 2024Distilling System 2 into System 18 Jul
  17. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs3 Jul
  18. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  19. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
  20. 2024Can We Verify Step by Step for Incorrect Answer Detection?16 Feb
  21. 2024Demystifying Chains, Trees, and Graphs of Thoughts25 Jan
  22. 2023Guiding Language Model Reasoning with Planning Tokens9 Oct
  23. 2023GRACE: Discriminator-Guided Chain-of-Thought Reasoning24 May

Other references · 548

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2024Qwen2.5 Technical Reportcited by 4,713
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,105
38 more
  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  4. 2024Mixtral of Expertscited by 1,785
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2025s1: Simple test-time scalingcited by 1,311
  9. 2024OpenAI o1 System Cardcited by 1,220
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  11. 2023PaLM 2 Technical Reportcited by 1,154
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  13. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  14. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  15. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  16. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  17. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  18. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  19. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  20. 2024DeepSeek-V3 Technical Reportcited by 724
  21. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  22. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  23. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  24. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  25. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  26. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  27. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  28. 2025LIMO: Less is More for Reasoningcited by 488
  29. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  30. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  31. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  32. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  33. 2025Process Reinforcement through Implicit Rewardscited by 478
  34. 2025Humanity's Last Examcited by 467
  35. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  36. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  37. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  38. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429

50 most cited of 548 All 548 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 236; 2026: 145.

381 papers.

Sort:

By area:

  1. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  2. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  4. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  5. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
  6. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  7. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 187
  8. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  9. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  10. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  11. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  12. 2025Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMscited by 115
38 more
  1. 2025Efficient Reasoning Models: A Surveycited by 93
  2. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
  3. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  4. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  5. 2025UFT: Unifying Supervised and Reinforcement Fine-Tuningcited by 58
  6. 2025PolyMath: Evaluating Mathematical Reasoning in Multilingual Contextscited by 52
  7. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  8. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
  9. 2025Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendationcited by 50
  10. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 50
  11. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agentscited by 49
  12. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  13. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  14. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  15. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
  16. 2025Crosslingual Reasoning through Test-Time Scalingcited by 46
  17. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lenscited by 42
  18. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 42
  19. 2025Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigmcited by 41
  20. 2025ARM: Adaptive Reasoning Modelcited by 38
  21. 2025Optimizing Length Compression in Large Reasoning Modelscited by 37
  22. 2025Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agentscited by 37
  23. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  24. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  25. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoningcited by 34
  26. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Modelscited by 33
  27. 2025SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learningcited by 31
  28. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
  29. 2025Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMscited by 30
  30. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  31. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  32. 2025Synergizing RAG and Reasoning: A Systematic Reviewcited by 27
  33. 2025Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Informationcited by 27
  34. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  35. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 26
  36. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 25
  37. 2025System-1.5 Reasoning: Traversal in Language and Latent Spaces with Dynamic Shortcutscited by 23
  38. 2025Route to Reason: Adaptive Routing for LLM and Reasoning Strategy Selectioncited by 23

50 most cited of 381 All 381 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Long Chain-of-Thought Reasoning Across Languages
  2. 2025Deconstructing Long Chain-of-Thought: A Structured Reasoning Optimization Framework for Long CoT Distillation
  3. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
  4. 2025Efficient Long CoT Reasoning in Small Language Models
  5. 2025Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering
  6. 2026LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning
  7. 2025One Missing Piece for Open-Source Reasoning Models: A Dataset to Mitigate Cold-Starting Short CoT LLMs in RL
  8. 2025Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations