mlsanityData to 7 Oct 2026

Large Language Models are Zero-Shot Reasoners

arXiv 2205.11916 ↗First posted 24 May 2022cs.CL cs.AI cs.LGcited by 4,323

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars. Notably, chain of thought (CoT) prompting, a recent technique for eliciting complex multi-step reasoning through step-by-step answer examples, achieved the state-of-the-art performances in arithmetics and symbolic reasoning, difficult system-2 tasks that do not follow the standard scaling laws for LLMs. While these successes are often attributed to LLMs' ability for few-shot learning, we show that LLMs are decent zero-shot reasoners by simply adding "Let's think step by step" before each answer. Experimental results demonstrate that our Zero-shot-CoT, using the same single prompt template, significantly outperforms zero-shot LLM performances on diverse benchmark reasoning tasks including arithmetics (MultiArith, GSM8K, AQUA-RAT, SVAMP), symbolic reasoning (Last Letter, Coin Flip), and other logical reasoning tasks (Date Understanding, Tracking Shuffled Objects), without any hand-crafted few-shot examples, e.g. increasing the accuracy on MultiArith from 17.7% to 78.7% and GSM8K from 10.4% to 40.7% with large InstructGPT model (text-davinci-002), as well as similar magnitudes of improvements with another off-the-shelf large model, 540B parameter PaLM. The versatility of this single prompt across very diverse reasoning tasks hints at untapped and understudied fundamental zero-shot capabilities of LLMs, suggesting high-level, multi-task broad cognitive capabilities may be extracted by simple prompting. We hope our work not only serves as the minimal strongest zero-shot baseline for the challenging reasoning benchmarks, but also highlights the importance of carefully exploring and analyzing the enormous zero-shot knowledge hidden inside LLMs before crafting finetuning datasets or few-shot exemplars.

Key facts. Cited by 4,323 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 35 of its 61 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

35 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  10. 2016Pointer Sentinel Mixture Modelscited by 2,992
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  12. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
21 more
  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  5. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  10. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  11. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  13. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  14. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  15. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  16. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  17. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  18. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  19. 2016Solving General Arithmetic Word Problemscited by 353
  20. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  21. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 118; 2023: 906; 2024: 1273; 2025: 1248; 2026: 778.

4,323 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  10. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
38 more
  1. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  5. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  7. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  8. 2023LIMA: Less Is More for Alignmentcited by 1,080
  9. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  10. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  13. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  14. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  15. 2023Teaching Large Language Models to Self-Debugcited by 963
  16. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  17. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  18. 2023A Survey on Multimodal Large Language Modelscited by 898
  19. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  20. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  21. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  22. 2023Large Language Models as Optimizerscited by 830
  23. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  24. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
  25. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  26. 2023StarCoder: may the source be with you!cited by 790
  27. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  28. 2022Galactica: A Large Language Model for Sciencecited by 736
  29. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  30. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  31. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  32. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  33. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  34. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  35. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  36. 2022Large Language Models Can Self-Improvecited by 656
  37. 2022PAL: Program-aided Language Modelscited by 635
  38. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630

50 most cited of 4,323 All 4,323 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Zero-Shot Verification-guided Chain of Thoughts
  2. 2024Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Models
  3. 2024Break the Chain: Large Language Models Can be Shortcut Reasoners
  4. 2025Revisiting Chain-of-Thought Prompting: Zero-shot Can Be Stronger than Few-shot
  5. 2024A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration
  6. 2025Reasoning Capabilities and Invariability of Large Language Models
  7. 2024Instance-adaptive Zero-shot Chain-of-Thought Prompting
  8. 2026Diverge to Induce Prompting: Multi-Rationale Induction for Zero-Shot Reasoning