mlsanityData to 7 Oct 2026

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

arXiv 2201.11903 ↗First posted 28 Jan 2022cs.CL cs.AIcited by 12,302

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking. For instance, prompting a 540B-parameter language model with just eight chain of thought exemplars achieves state of the art accuracy on the GSM8K benchmark of math word problems, surpassing even finetuned GPT-3 with a verifier.

Key facts. Cited by 12,302 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 64 of its 119 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

64 of 119 references.

References · 64

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2021Program Synthesis with Large Language Modelscited by 3,509
  12. 2018Deep contextualized word representationscited by 2,285
38 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  10. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  12. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  13. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  14. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  15. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  16. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  17. 2016Solving General Arithmetic Word Problemscited by 353
  18. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
  19. 2020Transformers as Soft Reasoners over Languagecited by 265
  20. 2021AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Promptscited by 265
  21. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  22. 2021How Many Data Points is a Prompt Worth?cited by 197
  23. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  24. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  25. 2019Neural Logic Machinescited by 144
  26. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  27. 2021Reframing Human-AI Collaboration for Generating Free-Text Explanationscited by 125
  28. 2022CommonsenseQA 2.0: Exposing the Limits of AI through Gamificationcited by 108
  29. 2021A Recipe For Arbitrary Text Style Transfer with Large Language Modelscited by 107
  30. 2014Learning to Executecited by 105
  31. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  32. 2021Few-Shot Self-Rationalization with Natural Language Promptscited by 83
  33. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  34. 2017Learning with Latent Languagecited by 75
  35. 2021When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Datacited by 75
  36. 2019NumNet: Machine Reading Comprehension with Numerical Reasoningcited by 59
  37. 2018Training Classifiers with Natural Language Explanationscited by 58
  38. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 281; 2023: 1828; 2024: 2802; 2025: 4138; 2026: 3253.

12,302 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  10. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  11. 2023Let's Verify Step by Stepcited by 3,598
  12. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2023A Survey of Large Language Modelscited by 2,537
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  13. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  14. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  15. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  16. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  17. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  18. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  19. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  20. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  21. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  22. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  23. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  24. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  25. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  26. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  27. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  28. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  29. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  30. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  31. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  32. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  33. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  34. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  35. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  36. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  37. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  38. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348

50 most cited of 12,302 All 12,302 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  2. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?
  3. 2023Contrastive Chain-of-Thought Prompting
  4. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  5. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
  6. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods
  7. 2024Break the Chain: Large Language Models Can be Shortcut Reasoners
  8. 2024CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning