mlsanityData to 7 Oct 2026

Chain-of-Thought Reasoning Without Prompting

arXiv 2402.10200 ↗First posted 15 Feb 2024cs.CLcited by 208

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

In enhancing the reasoning capabilities of large language models (LLMs), prior research primarily focuses on specific prompting techniques such as few-shot or zero-shot chain-of-thought (CoT) prompting. These methods, while effective, often involve manually intensive prompt engineering. Our study takes a novel approach by asking: Can LLMs reason effectively without prompting? Our findings reveal that, intriguingly, CoT reasoning paths can be elicited from pre-trained LLMs by simply altering the textitdecoding process. Rather than conventional greedy decoding, we investigate the top-k alternative tokens, uncovering that CoT paths are frequently inherent in these sequences. This approach not only bypasses the confounders of prompting but also allows us to assess the LLMs' textitintrinsic reasoning abilities. Moreover, we observe that the presence of a CoT in the decoding path correlates with a higher confidence in the model's decoded answer. This confidence metric effectively differentiates between CoT and non-CoT paths. Extensive empirical studies on various reasoning benchmarks show that the proposed CoT-decoding effectively elicits reasoning capabilities from language models, which were previously obscured by standard greedy decoding.

Key facts. Cited by 208 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 58 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

46 of 58 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Large Language Models as Analogical Reasoners3 Oct
  2. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  3. 2023Why think step by step? Reasoning emerges from the locality of experience7 Apr
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks22 Nov
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  6. 2022Large Language Models are Zero-Shot Reasoners24 May
  7. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  9. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2018Hierarchical Neural Story Generationcited by 1,182
  10. 2023PaLM 2 Technical Reportcited by 1,154
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  12. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 975
25 more
  1. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2023Large Language Models as Optimizerscited by 830
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  5. 2022Large Language Models Can Self-Improvecited by 656
  6. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  7. 2022PAL: Program-aided Language Modelscited by 635
  8. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  9. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  10. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  11. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  12. 2016Solving General Arithmetic Word Problemscited by 353
  13. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  14. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  15. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
  16. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  17. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  18. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 160
  19. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
  20. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  21. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  22. 2018Learning to Write with Cooperative Discriminatorscited by 119
  23. 2023Contrastive Decoding Improves Reasoning in Large Language Modelscited by 71
  24. 2023PathFinder: Guided Search over Multi-Step Reasoning Pathscited by 12
  25. 2019Minimum-Margin Active Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 58; 2025: 101; 2026: 49.

208 papers.

Sort:

By area:

  1. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  2. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  3. 2025ThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planningcited by 152
  4. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 148
  5. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  6. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  7. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  8. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  9. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
  10. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 87
  11. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  12. 2024Learning to Route LLMs with Confidence Tokenscited by 66
38 more
  1. 2025Chain-of-Visual-Thought: Teaching VLMs to See and Think Better with Continuous Visual Tokenscited by 65
  2. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewardingcited by 53
  3. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  4. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  5. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  6. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  7. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  8. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 40
  9. 2024FIRE: Fact-checking with Iterative Retrieval and Verificationcited by 39
  10. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  11. 2025Improving LLM-as-a-Judge Inference with the Judgment Distributioncited by 36
  12. 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Modelscited by 33
  13. 2024CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Followingcited by 32
  14. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 32
  15. 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMscited by 30
  16. 2024From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiencycited by 28
  17. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  18. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  19. 2024Calibrating Reasoning in Language Models with Internal Consistencycited by 27
  20. 2024On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoningcited by 26
  21. 2024Non-myopic Generation of Language Models for Reasoning and Planningcited by 25
  22. 2024Deliberation in Latent Space via Differentiable Cache Augmentationcited by 25
  23. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardcited by 25
  24. 2025Symbiotic Cooperation for Web Agents: Harnessing Complementary Strengths of Large and Small LLMscited by 24
  25. 2025BOLT: Bootstrap Long Chain-of-Thought in Language Models without Distillationcited by 23
  26. 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
  27. 2025Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimizationcited by 23
  28. 2024Can LLMs Solve longer Math Word Problems Better?cited by 22
  29. 2025Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematicscited by 22
  30. 2024Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Modelscited by 21
  31. 2024Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparatorscited by 19
  32. 2024Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitationcited by 19
  33. 2025Automatic Evaluation of Healthcare LLMs Beyond Question-Answeringcited by 19
  34. 2025Empirical Analysis of Decoding Biases in Masked Diffusion Modelscited by 19
  35. 2024Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generationcited by 18
  36. 2025On the Effect of Sampling Diversity in Scaling LLM Inferencecited by 18
  37. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  38. 2025CER: Confidence Enhanced Reasoning in LLMscited by 16

50 most cited of 208 All 208 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  2. 2023An automatically discovered chain-of-thought prompt generalizes to novel models and datasets
  3. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
  4. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  5. 2024ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting
  6. 2025Innate Reasoning is Not Enough: In-Context Learning Enhances Reasoning Large Language Models with Less Overthinking
  7. 2025"Well, Keep Thinking": Enhancing LLM Reasoning with Adaptive Injection Decoding
  8. 2024Break the Chain: Large Language Models Can be Shortcut Reasoners