mlsanityData to 7 Oct 2026

Large Language Models are Better Reasoners with Self-Verification

arXiv 2212.09561 ↗First posted 19 Dec 2022cs.AI cs.CLcited by 304

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Recently, with the chain of thought (CoT) prompting, large language models (LLMs), e.g., GPT-3, have shown strong reasoning ability in several natural language processing tasks such as arithmetic, commonsense, and logical reasoning. However, LLMs with CoT require multi-step prompting and multi-token prediction, which is highly sensitive to individual mistakes and vulnerable to error accumulation. The above issues make the LLMs need the ability to verify the answers. In fact, after inferring conclusions in some thinking decision tasks, people often check them by re-verifying steps to avoid some mistakes. In this paper, we propose and prove that LLMs also have similar self-verification abilities. We take the conclusion obtained by CoT as one of the conditions for solving the original problem. By performing a backward verification of the answers that LLM deduced for itself, we can obtain interpretable answer validation scores to select the candidate answer with the highest score. Experimental results demonstrate that the proposed method can improve the reasoning performance on various arithmetic, commonsense, and logical reasoning datasets. Our code is publicly available at: https://github.com/WENGSYX/Self-Verification.

Key facts. Cited by 304 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 37 of its 55 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

37 of 55 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  2. 2022Complexity-Based Prompting for Multi-Step Reasoning3 Oct
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  9. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  10. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  11. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  12. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
20 more
  1. 2022PAL: Program-aided Language Modelscited by 635
  2. 2016Solving General Arithmetic Word Problemscited by 353
  3. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  4. 2019Abductive Commonsense Reasoningcited by 300
  5. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  6. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  7. 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
  8. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  9. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  10. 2020Logic-Guided Data Augmentation and Regularization for Consistent Question Answeringcited by 77
  11. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  12. 2018Translating a Math Word Problem to an Expression Treecited by 57
  13. 2021Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Textcited by 46
  14. 2022Reasoning Like Program Executorscited by 45
  15. 2019A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoningcited by 41
  16. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  17. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  18. 2022What Makes Pre-trained Language Models Better Zero-shot Learners?cited by 23
  19. 2021ReasonBERT: Pre-trained to Reason with Distant Supervisioncited by 22
  20. 2022QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 43; 2024: 86; 2025: 105; 2026: 70.

304 papers.

Sort:

By area:

  1. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  2. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  3. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  4. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  5. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  6. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  7. 2024LoRA Learns Less and Forgets Lesscited by 313
  8. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  9. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 251
  10. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  11. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
38 more
  1. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  2. 2023G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Modelcited by 208
  3. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  4. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 182
  5. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  6. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  7. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  8. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  9. 2024Distilling System 2 into System 1cited by 152
  10. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 145
  11. 2024CycleResearcher: Improving Automated Research via Automated Reviewcited by 142
  12. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  13. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 137
  14. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  15. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  16. 2025Confidence Improves Self-Consistency in LLMscited by 111
  17. 2025DeepReview: Improving LLM-based Paper Review with Human-like Deep Thinking Processcited by 97
  18. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 96
  19. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  20. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
  21. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  22. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  23. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  24. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 84
  25. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 82
  26. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipelinecited by 81
  27. 2024Formal Mathematical Reasoning: A New Frontier in AIcited by 80
  28. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  29. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 68
  30. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Modelscited by 65
  31. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  32. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Samplingcited by 63
  33. 2023Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observationscited by 58
  34. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 58
  35. 2025DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressivelycited by 56
  36. 2024Large Language Models Can Self-Improve At Web Agent Taskscited by 54
  37. 2025SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scalingcited by 54
  38. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52

50 most cited of 304 All 304 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024General Purpose Verification for Chain of Thought Prompting
  2. 2025Asking LLMs to Verify First is Almost Free Lunch
  3. 2023It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning
  4. 2024Chain-of-Thought Reasoning Without Prompting
  5. 2025Incentivizing LLMs to Self-Verify Their Answers
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  7. 2025Large Language Models' Reasoning Stalls: An Investigation into the Capabilities of Frontier Models
  8. 2023Deductive Verification of Chain-of-Thought Reasoning