mlsanityData to 7 Oct 2026

Large Language Models have Intrinsic Self-Correction Ability

arXiv 2406.15673 ↗First posted 21 Jun 2024cs.CL cs.AIcited by 45

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models (LLMs) have attracted significant attention for their exceptional abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-correction, intrinsic self-correction is considered a promising direction because it does not utilize external knowledge. However, recent works doubt the validity of LLM's ability to conduct intrinsic self-correction. In this paper, we present a novel perspective on the intrinsic self-correction capabilities of LLMs through theoretical analyses and empirical experiments. In addition, we identify two critical factors for successful self-correction: zero temperature and fair prompts. Leveraging these factors, we demonstrate that intrinsic self-correction ability is exhibited across multiple existing LLMs. Our findings offer insights into the fundamental theories underlying the self-correction behavior of LLMs and remark on the importance of unbiased prompts and zero temperature settings in harnessing their full potential.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 29 of its 32 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

29 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models19 Oct
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  3. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May
  4. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2023Teaching Large Language Models to Self-Debugcited by 963
  11. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  12. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
13 more
  1. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  2. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  3. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  4. 2023Language Models can Solve Computer Taskscited by 371
  5. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  6. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 293
  7. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  8. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
  9. 2023Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarizationcited by 70
  10. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
  11. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  12. 2020Contextual Temperature for Language Modelingcited by 25
  13. 2024Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architecturescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 22; 2026: 17.

45 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025Confidence Improves Self-Consistency in LLMscited by 111
  3. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planningcited by 58
  4. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  5. 2025Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verificationcited by 49
  6. 2024Understanding the Dark Side of LLMs' Intrinsic Self-Correctioncited by 42
  7. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  8. 2024Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Taskscited by 30
  9. 2024Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMscited by 28
  10. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Computecited by 14
  11. 2025Do Repetitions Matter? Strengthening Reliability in LLM Evaluationscited by 14
  12. 2026Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinementcited by 13
33 more
  1. 2025Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignmentcited by 12
  2. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMscited by 12
  3. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 12
  4. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  5. 2025A Probabilistic Inference Scaling Theory for LLM Self-Correctioncited by 9
  6. 2026The Illusion of Insight in Reasoning Modelscited by 8
  7. 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gapcited by 6
  8. 2025Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capabilitycited by 3
  9. 2025Discourse Heuristics For Paradoxically Moral Self-Correctioncited by 3
  10. 2026How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signalscited by 3
  11. 2026When Does Intrinsic Self-Correction Help? A Task-Sensitive Analysiscited by 3
  12. 2024Automating Intervention Discovery from Scientific Literature: A Progressive Ontology Prompting and Dual-LLM Frameworkcited by 2
  13. 2025From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMscited by 2
  14. 2025Unlocking Zero-Shot Geospatial Reasoning via Indirect Rewardscited by 2
  15. 2025Adaptive Multi-Agent Response Refinement in Conversational Systemscited by 2
  16. 2025MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Modelscited by 2
  17. 2025Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoningcited by 1
  18. 2026"I May Not Have Articulated Myself Clearly": Diagnosing Dynamic Instability in LLM Reasoning at Inference Timecited by 1
  19. 2026Process Supervision via Verbal Critique Improves Reasoning in Large Language Modelscited by 1
  20. 2026The Future of Facts: Tracing the Factual Generation-Verification Gapcited by 1
  21. 2026To Nuke or Not to Nuke: LLMs' (Missing) Ethical Reasoning and Actions in a High-Stakes Decision-Making Simulationcited by 1
  22. 2026Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMscited by 1
  23. 2025Towards Understanding Multi-Round Large Language Model Reasoning: Approximability, Learnability and Generalizabilitycited by 0
  24. 2025Why Stop at One Error? Benchmarking LLMs as Data Science Code Debuggers for Multi-Hop and Multi-Bug Errorscited by 0
  25. 2025Enhancing Factual Accuracy and Citation Generation in LLMs via Multi-Stage Self-Verificationcited by 0
  26. 2025Mastering Olympiad-Level Physics with Artificial Intelligencecited by 0
  27. 2026Beyond Output Critique: Self-Correction via Task Distillationcited by 0
  28. 2026Label-Consistent Data Generation for Aspect-Based Sentiment Analysis Using LLM Agentscited by 0
  29. 2026Evidence-Supported Credit Risk Report Generation Using News-Centric Financial Knowledge Graphscited by 0
  30. 2026Spectral Origins of the Self-Correction Blind Spot in Autoregressive Generationcited by 0
  31. 2026Refining Over Resampling: Test-Time Self-Correction for LLM Reasoningcited by 0
  32. 2026Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recoverycited by 0
  33. 2026Bootstrapping Video Interaction Generation with Synthetic State Transitionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
  2. 2024A Theoretical Understanding of Self-Correction through In-context Alignment
  3. 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
  4. 2025Self-Correcting Large Language Models: Generation vs. Multiple Choice
  5. 2025Language Models can perform Single-Utterance Self-Correction of Perturbed Reasoning
  6. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  7. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models
  8. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives