# Understanding the Dark Side of LLMs' Intrinsic Self-Correction

arXiv: https://arxiv.org/abs/2412.14959 · HTML: https://mlsanity.com/paper/2412.14959/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 36)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Causal Abstractions of Neural Networks](https://mlsanity.com/paper/2106.02997/) (2021)
- [Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://mlsanity.com/paper/2403.09629/) (2024)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs](https://mlsanity.com/paper/2406.01297/) (2024)
- [Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View](https://mlsanity.com/paper/2310.02124/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [Ensemble Distribution Distillation](https://mlsanity.com/paper/1905.00076/) (2019)
- [Thinking Fast and Slow in Large Language Models](https://mlsanity.com/paper/2212.05206/) (2022)
- [The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation](https://mlsanity.com/paper/2312.09085/) (2023)
- [RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs](https://mlsanity.com/paper/2305.08844/) (2023)
- [Capturing Failures of Large Language Models via Human Cognitive Biases](https://mlsanity.com/paper/2202.12299/) (2022)
- [Overthinking the Truth: Understanding how Language Models Process False Demonstrations](https://mlsanity.com/paper/2307.09476/) (2023)
- [Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking](https://mlsanity.com/paper/2311.09827/) (2023)
- [Large Language Models have Intrinsic Self-Correction Ability](https://mlsanity.com/paper/2406.15673/) (2024)
- [Large Language Models Can Self-Correct with Key Condition Verification](https://mlsanity.com/paper/2405.14092/) (2024)
- [Using Captum to Explain Generative Language Models](https://mlsanity.com/paper/2312.05491/) (2023)

## Built on by (most cited first, 30 of 42)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training](https://mlsanity.com/paper/2501.11425/) (2025)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoning](https://mlsanity.com/paper/2501.02497/) (2025)
- [S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning](https://mlsanity.com/paper/2502.12853/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Challenging the Evaluator: LLM Sycophancy Under User Rebuttal](https://mlsanity.com/paper/2509.16533/) (2025)
- [Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findings](https://mlsanity.com/paper/2505.24341/) (2025)
- [A Probabilistic Inference Scaling Theory for LLM Self-Correction](https://mlsanity.com/paper/2508.16456/) (2025)
- [LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information](https://mlsanity.com/paper/2502.02095/) (2025)
- [SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modeling](https://mlsanity.com/paper/2510.05115/) (2025)
- [Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Models](https://mlsanity.com/paper/2504.02902/) (2025)
- [When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retraction](https://mlsanity.com/paper/2505.16170/) (2025)
- [Think Visually, Reason Textually: Vision-Language Synergy in ARC](https://mlsanity.com/paper/2511.15703/) (2025)
- [Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization](https://mlsanity.com/paper/2504.14858/) (2025)
- [Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessions](https://mlsanity.com/paper/2603.12123/) (2026)
- [Understanding the Dilemma of Unlearning for Large Language Models](https://mlsanity.com/paper/2509.24675/) (2025)
- [Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models](https://mlsanity.com/paper/2509.24711/) (2025)
- [More Rounds, More Noise: Why Multi-Turn Review Fails to Improve Cross-Context Verification](https://mlsanity.com/paper/2603.16244/) (2026)
- [Discourse Heuristics For Paradoxically Moral Self-Correction](https://mlsanity.com/paper/2507.00985/) (2025)
- [Concise Reasoning in the Lens of Lagrangian Optimization](https://mlsanity.com/paper/2510.10168/) (2025)
- [How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signals](https://mlsanity.com/paper/2604.22271/) (2026)
- [Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skill](https://mlsanity.com/paper/2606.06454/) (2026)
- [When Does Intrinsic Self-Correction Help? A Task-Sensitive Analysis](https://mlsanity.com/paper/2606.23196/) (2026)
- [When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicity](https://mlsanity.com/paper/2509.11141/) (2025)
- [Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domains](https://mlsanity.com/paper/2605.17359/) (2026)
- [Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Models](https://mlsanity.com/paper/2606.16999/) (2026)
- [DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization](https://mlsanity.com/paper/2605.31455/) (2026)
- [Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs](https://mlsanity.com/paper/2606.16011/) (2026)

## Abstract (arXiv)

Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. However, recent works show that LLMs' intrinsic self-correction fails without oracle labels as feedback prompts. In this paper, we aim to interpret LLMs' intrinsic self-correction for different tasks, especially for those failure cases. By including one simple task and three complex tasks with state-of-the-art (SOTA) LLMs like ChatGPT families (o1, 4o, 3.5-turbo) and Llama families (2-7B, 3-8B, and 3.1-8B), we design three interpretation methods to reveal the dark side of LLMs' intrinsic self-correction. We identify intrinsic self-correction can (1) cause LLMs to waver both intermedia and final answers and lead to prompt bias on simple factual questions; (2) introduce human-like cognitive bias on complex tasks. In light of our findings, we also provide two simple yet effective strategies for alleviation: question repeating and supervised fine-tuning with a few samples. We open-source our work at https://x-isc.info/.