mlsanityData to 7 Oct 2026

Small Language Models Need Strong Verifiers to Self-Correct Reasoning

arXiv 2404.17140 ↗First posted 26 Apr 2024cs.CLcited by 85

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-correction has emerged as a promising solution to boost the reasoning performance of large language models (LLMs), where LLMs refine their solutions using self-generated critiques that pinpoint the errors. This work explores whether small (<= 13B) language models (LMs) have the ability of self-correction on reasoning tasks with minimal inputs from stronger LMs. We propose a novel pipeline that prompts smaller LMs to collect self-correction data that supports the training of self-refinement abilities. First, we leverage correct solutions to guide the model in critiquing their incorrect responses. Second, the generated critiques, after filtering, are used for supervised fine-tuning of the self-correcting reasoner through solution refinement. Our experimental results show improved self-correction abilities of two models on five datasets spanning math and commonsense reasoning, with notable performance gains when paired with a strong GPT-4-based verifier, though limitations are identified when using a weak self-verifier for determining when to correct.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 43 of its 54 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

43 of 54 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  3. 2023Making Large Language Models Better Reasoners with Alignment5 Sep
  4. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies6 Aug
  5. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  6. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  7. 2022Large Language Models Can Self-Improve20 Oct

Other references · 36

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2023Teaching Large Language Models to Self-Debugcited by 963
  10. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  11. 2024Self-Rewarding Language Modelscited by 654
  12. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
24 more
  1. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  2. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 399
  3. 2022Self-critiquing models for assisting human evaluatorscited by 372
  4. 2023Language Models can Solve Computer Taskscited by 371
  5. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  6. 2022Generating Sequences by Learning to Self-Correctcited by 271
  7. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  8. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 231
  9. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  10. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  11. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  12. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
  13. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  14. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 63
  15. 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 60
  16. 2024CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilitiescited by 43
  17. 2023GRACE: Discriminator-Guided Chain-of-Thought Reasoningcited by 35
  18. 2024Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphscited by 32
  19. 2023Teaching Language Models to Self-Improve through Interactive Demonstrationscited by 26
  20. 2024Small Language Model Can Self-correctcited by 23
  21. 2024The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?cited by 20
  22. 2024The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluatecited by 16
  23. 2024Can We Verify Step by Step for Incorrect Answer Detection?cited by 15
  24. 2024ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Self-Refinementcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 56; 2026: 18.

85 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  4. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  5. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  6. 2025Process Reward Models That Thinkcited by 118
  7. 2025Confidence Improves Self-Consistency in LLMscited by 111
  8. 2025Efficient Reasoning Models: A Surveycited by 93
  9. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 90
  10. 2025Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?cited by 69
  11. 2024Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problemscited by 49
  12. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
38 more
  1. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  2. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
  3. 2025ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoningcited by 41
  4. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 41
  5. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  6. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 37
  7. 2025Towards Reasoning Ability of Small Language Modelscited by 29
  8. 2024Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMscited by 28
  9. 2025Adversarial Reasoning at Jailbreaking Timecited by 27
  10. 2025To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoningcited by 24
  11. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  12. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 20
  13. 2024Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter?cited by 18
  14. 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 18
  15. 2024S^3cMath: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasonerscited by 17
  16. 2025HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Taskscited by 16
  17. 2025Incentivizing LLMs to Self-Verify Their Answerscited by 16
  18. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 14
  19. 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoningcited by 13
  20. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  21. 2025Sample Complexity and Representation Ability of Test-time Scaling Paradigmscited by 12
  22. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMscited by 12
  23. 2025ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflectioncited by 11
  24. 2025Sherlock: Self-Correcting Reasoning in Vision-Language Modelscited by 11
  25. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 11
  26. 2025Step-level Verifier-guided Hybrid Test-Time Scaling for Large Language Modelscited by 10
  27. 2025Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoningcited by 9
  28. 2025Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMscited by 9
  29. 2025A Probabilistic Inference Scaling Theory for LLM Self-Correctioncited by 9
  30. 2025Stackelberg Learning from Human Feedback: Preference Optimization as a Sequential Gamecited by 9
  31. 2024Rationale-Aware Answer Verification by Pairwise Self-Evaluationcited by 8
  32. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 8
  33. 2025Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesiscited by 8
  34. 2025Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Modelscited by 7
  35. 2025Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaborationcited by 7
  36. 2025On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inferencecited by 6
  37. 2026ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinementcited by 5
  38. 2024Towards Intrinsic Self-Correction Enhancement in Monte Carlo Tree Search Boosted Reasoning via Iterative Preference Learningcited by 4

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning
  2. 2024Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models
  3. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
  4. 2026More Yap Less Meaning: Uncovering Self-Improvement Behavior in SLMs
  5. 2024Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoning
  6. 2025Superficial Self-Improved Reasoners Benefit from Model Merging
  7. 2025Self-rewarding correction for mathematical reasoning
  8. 2024Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models