# Large Language Models have Intrinsic Self-Correction Ability

arXiv: https://arxiv.org/abs/2406.15673 · HTML: https://mlsanity.com/paper/2406.15673/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 29 of its 32 references that are in the map.
- First posted on arXiv 21 Jun 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Language Models can Solve Computer Tasks](https://mlsanity.com/paper/2303.17491/) (2023)
- [Large Language Models are Better Reasoners with Self-Verification](https://mlsanity.com/paper/2212.09561/) (2022)
- [Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective](https://mlsanity.com/paper/2305.15408/) (2023)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://mlsanity.com/paper/2210.08726/) (2022)
- [The Effect of Sampling Temperature on Problem Solving in Large Language Models](https://mlsanity.com/paper/2402.05201/) (2024)
- [Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization](https://mlsanity.com/paper/2305.13091/) (2023)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [Self-Verification Improves Few-Shot Clinical Information Extraction](https://mlsanity.com/paper/2306.00024/) (2023)
- [Contextual Temperature for Language Modeling](https://mlsanity.com/paper/2012.13575/) (2020)
- [MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models](https://mlsanity.com/paper/2310.12426/) (2023)
- [Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures](https://mlsanity.com/paper/2405.04700/) (2024)

## Built on by (most cited first, 30 of 46)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Confidence Improves Self-Consistency in LLMs](https://mlsanity.com/paper/2502.06233/) (2025)
- [SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning](https://mlsanity.com/paper/2503.11951/) (2025)
- [Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective](https://mlsanity.com/paper/2412.14135/) (2024)
- [Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification](https://mlsanity.com/paper/2502.01839/) (2025)
- [Understanding the Dark Side of LLMs' Intrinsic Self-Correction](https://mlsanity.com/paper/2412.14959/) (2024)
- [MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning](https://mlsanity.com/paper/2409.12147/) (2024)
- [Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks](https://mlsanity.com/paper/2410.04055/) (2024)
- [Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs](https://mlsanity.com/paper/2412.19513/) (2024)
- [The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute](https://mlsanity.com/paper/2505.14733/) (2025)
- [Do Repetitions Matter? Strengthening Reliability in LLM Evaluations](https://mlsanity.com/paper/2509.24086/) (2025)
- [Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs](https://mlsanity.com/paper/2510.16062/) (2025)
- [Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinement](https://mlsanity.com/paper/2603.08251/) (2026)
- [BAGEN: Are LLM Agents Budget-Aware?](https://mlsanity.com/paper/2606.00198/) (2026)
- [Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment](https://mlsanity.com/paper/2502.07803/) (2025)
- [Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey](https://mlsanity.com/paper/2504.14520/) (2025)
- [A Probabilistic Inference Scaling Theory for LLM Self-Correction](https://mlsanity.com/paper/2508.16456/) (2025)
- [The Illusion of Insight in Reasoning Models](https://mlsanity.com/paper/2601.00514/) (2026)
- [Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gap](https://mlsanity.com/paper/2507.00075/) (2025)
- [Meeseeks: A Feedback-Driven, Iterative Self-Correction Benchmark evaluating LLMs' Instruction Following Capability](https://mlsanity.com/paper/2504.21625/) (2025)
- [Discourse Heuristics For Paradoxically Moral Self-Correction](https://mlsanity.com/paper/2507.00985/) (2025)
- [MiST: Understanding the Role of Mid-Stage Scientific Training in Developing Chemical Reasoning Models](https://mlsanity.com/paper/2512.21231/) (2025)
- [How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals](https://mlsanity.com/paper/2604.22271/) (2026)
- [When Does Intrinsic Self-Correction Help? A Task-Sensitive Analysis](https://mlsanity.com/paper/2606.23196/) (2026)
- [Automating Intervention Discovery from Scientific Literature: A Progressive Ontology Prompting and Dual-LLM Framework](https://mlsanity.com/paper/2409.00054/) (2024)
- [From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMs](https://mlsanity.com/paper/2504.11277/) (2025)
- [Unlocking Zero-Shot Geospatial Reasoning via Indirect Rewards](https://mlsanity.com/paper/2510.00072/) (2025)
- [Adaptive Multi-Agent Response Refinement in Conversational Systems](https://mlsanity.com/paper/2511.08319/) (2025)
- [Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning](https://mlsanity.com/paper/2507.10007/) (2025)
- [Mastering Olympiad-Level Physics with Artificial Intelligence](https://mlsanity.com/paper/2511.10515/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have attracted significant attention for their exceptional abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-correction, intrinsic self-correction is considered a promising direction because it does not utilize external knowledge. However, recent works doubt the validity of LLM's ability to conduct intrinsic self-correction. In this paper, we present a novel perspective on the intrinsic self-correction capabilities of LLMs through theoretical analyses and empirical experiments. In addition, we identify two critical factors for successful self-correction: zero temperature and fair prompts. Leveraging these factors, we demonstrate that intrinsic self-correction ability is exhibited across multiple existing LLMs. Our findings offer insights into the fundamental theories underlying the self-correction behavior of LLMs and remark on the importance of unbiased prompts and zero temperature settings in harnessing their full potential.