mlsanityData to 14 Oct 2026

Self-Consistency Improves Chain of Thought Reasoning in Language Models

arXiv 2203.11171 ↗First posted 21 Mar 2022cs.CL cs.AIcited by 5,708

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%), StrategyQA (+6.4%) and ARC-challenge (+3.9%).

Key facts. Cited by 5,708 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 81 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

48 of 81 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 47

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  7. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  10. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  11. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
35 more
  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  2. 2018Hierarchical Neural Story Generationcited by 1,193
  3. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  4. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  6. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  7. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  8. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  11. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  12. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  13. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  14. 2016Solving General Arithmetic Word Problemscited by 360
  15. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  16. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 149
  17. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  18. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 138
  19. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  20. 2018Learning to Write with Cooperative Discriminatorscited by 119
  21. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  22. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  23. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  24. 2019Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanationscited by 71
  25. 2019NumNet: Machine Reading Comprehension with Numerical Reasoningcited by 59
  26. 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
  27. 2020Consistency of a Recurrent Language Model With Respect to Incomplete Decodingcited by 54
  28. 2019Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehensioncited by 51
  29. 2019Multi-hop Question Answering via Reasoning Chainscited by 42
  30. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  31. 2021MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solverscited by 39
  32. 2016Mutual Information and Diverse Decoding Improve Neural Machine Translationcited by 35
  33. 2021Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoningcited by 23
  34. 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Expertscited by 19
  35. 2021Exploiting Reasoning Chains for Multi-hop Science Question Answeringcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 100; 2023: 716; 2024: 1206; 2025: 1866; 2026: 1820.

5,708 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  12. 2023A Survey of Large Language Modelscited by 2,576
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,512
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  12. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  13. 2023PaLM 2 Technical Reportcited by 1,162
  14. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  15. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  16. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  17. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  18. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  19. 2023Teaching Large Language Models to Self-Debugcited by 988
  20. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  21. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  22. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  23. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  24. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  25. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  26. 2023Large Language Models as Optimizerscited by 864
  27. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  28. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  29. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  30. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  31. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  32. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  33. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  34. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  35. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  36. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  37. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  38. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668

50 most cited of 5,708 All 5,708 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning
  2. 2023SCOTT: Self-Consistent Chain-of-Thought Distillation
  3. 2026Enhancing reasoning accuracy in large language models during inference time
  4. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
  5. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  6. 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought
  7. 2024General Purpose Verification for Chain of Thought Prompting
  8. 2026Enhancing Self-Correction in Large Language Models through Multi-Perspective Reflection