mlsanityData to 7 Oct 2026

Understanding the Dark Side of LLMs' Intrinsic Self-Correction

arXiv 2412.14959 ↗First posted 19 Dec 2024cs.CLcited by 42

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Intrinsic self-correction was proposed to improve LLMs' responses via feedback prompts solely based on their inherent capability. However, recent works show that LLMs' intrinsic self-correction fails without oracle labels as feedback prompts. In this paper, we aim to interpret LLMs' intrinsic self-correction for different tasks, especially for those failure cases. By including one simple task and three complex tasks with state-of-the-art (SOTA) LLMs like ChatGPT families (o1, 4o, 3.5-turbo) and Llama families (2-7B, 3-8B, and 3.1-8B), we design three interpretation methods to reveal the dark side of LLMs' intrinsic self-correction. We identify intrinsic self-correction can (1) cause LLMs to waver both intermedia and final answers and lead to prompt bias on simple factual questions; (2) introduce human-like cognitive bias on complex tasks. In light of our findings, we also provide two simple yet effective strategies for alleviation: question repeating and supervised fine-tuning with a few samples. We open-source our work at https://x-isc.info/.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 55 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

36 of 55 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Large Language Models have Intrinsic Self-Correction Ability21 Jun
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  3. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May
  4. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  5. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 31

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  5. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  6. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  7. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  8. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  9. 2021Causal Abstractions of Neural Networkscited by 401
  10. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  11. 2022Generating Sequences by Learning to Self-Correctcited by 271
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
19 more
  1. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  2. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  3. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  4. 2022Large Language Models with Controllable Working Memorycited by 169
  5. 2019Ensemble Distribution Distillationcited by 158
  6. 2022Thinking Fast and Slow in Large Language Modelscited by 136
  7. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  8. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  9. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 97
  10. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  11. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  12. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39
  13. 2023Using Captum to Explain Generative Language Modelscited by 33
  14. 2024Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seekingcited by 24
  15. 2023Working Memory Capacity of ChatGPT: An Empirical Studycited by 23
  16. 2024Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learningcited by 20
  17. 2024Walking in Others' Shoes: How Perspective-Taking Guides Large Language Models in Reducing Toxicity and Biascited by 17
  18. 2024Course-Correction: Safety Alignment Using Synthetic Preferencescited by 13
  19. 2023Ask Again, Then Fail: Large Language Models' Vacillations in Judgmentcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 23; 2026: 19.

42 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 74
  4. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  5. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  6. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 37
  7. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  8. 2025Challenging the Evaluator: LLM Sycophancy Under User Rebuttalcited by 18
  9. 2025Exploring Multimodal Challenges in Toxic Chinese Detection: Taxonomy, Benchmark, and Findingscited by 10
  10. 2025A Probabilistic Inference Scaling Theory for LLM Self-Correctioncited by 9
  11. 2025LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Informationcited by 8
  12. 2025SAC-Opt: Semantic Anchors for Iterative Correction in Optimization Modelingcited by 8
30 more
  1. 2025Beyond Accuracy: The Role of Calibration in Self-Improving Large Language Modelscited by 7
  2. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  3. 2025Think Visually, Reason Textually: Vision-Language Synergy in ARCcited by 6
  4. 2025Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimizationcited by 5
  5. 2026Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessionscited by 5
  6. 2026More Rounds, More Noise: Why Multi-Turn Review Fails to Improve Cross-Context Verificationcited by 4
  7. 2025Discourse Heuristics For Paradoxically Moral Self-Correctioncited by 3
  8. 2025Understanding the Dilemma of Unlearning for Large Language Modelscited by 3
  9. 2025Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Modelscited by 3
  10. 2025Concise Reasoning in the Lens of Lagrangian Optimizationcited by 3
  11. 2026How LLMs Detect and Correct Their Own Errors: The Role of Internal Confidence Signalscited by 3
  12. 2026When Does Intrinsic Self-Correction Help? A Task-Sensitive Analysiscited by 3
  13. 2025When Smiley Turns Hostile: Interpreting How Emojis Trigger LLMs' Toxicitycited by 2
  14. 2026Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skillcited by 2
  15. 2026Learning Transferable Topology Priors for Multi-Agent LLM Collaboration Across Domainscited by 1
  16. 2026Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMscited by 1
  17. 2026Selection Without Signal, Recovery Through Expression: A Measurement Study of Post-Hoc Falsification Operators for Frozen Small Code Modelscited by 1
  18. 2025Unlocking Recursive Thinking of LLMs: Alignment via Refinementcited by 0
  19. 2025FineRef: Fine-Grained Error Reflection and Correction for Long-Form Generation with Citationscited by 0
  20. 2026ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modellingcited by 0
  21. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgmentscited by 0
  22. 2026No Action Without a NOD: A Heterogeneous Multi-Agent Architecture for Reliable Service Agentscited by 0
  23. 2026DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimizationcited by 0
  24. 2026Large Language Models as Optimizers: A Survey of Direct vs. Tool-Augmented Approaches and Their Performance Frontierscited by 0
  25. 2026Think Through a Bottleneck: Hourglass Reasoning for Rigorous Inductioncited by 0
  26. 2026Factorized Hypothesis Search for Evidence-to-Taxonomy Retrievalcited by 0
  27. 2026DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learningcited by 0
  28. 2026An Interpretable Memory Decision Controller for LLM Agents Based on Three-Signal Complementarity: Decoupling Confidence and Consistencycited by 0
  29. 2026When Should LLMs Trust Their Own Revisions? A Risk-Aware Study of Intrinsic Self-Correctioncited by 0
  30. 2026LAURA: Knowledge Distillation for Interpretable Ambiguous Clause Identification in Legal Contractscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs
  2. 2023LLMs cannot find reasoning errors, but can correct them given the error location
  3. 2025Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs
  4. 2024A Theoretical Understanding of Self-Correction through In-context Alignment
  5. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing
  6. 2025Confidence Improves Self-Consistency in LLMs
  7. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives
  8. 2024Direct-Inverse Prompting: Analyzing LLMs' Discriminative Capacity in Self-Improving Generation