mlsanityData to 14 Oct 2026

Large Language Models Cannot Self-Correct Reasoning Yet

arXiv 2310.01798 ↗First posted 3 Oct 2023cs.CL cs.AIcited by 1,017

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Language Models (LLMs) have emerged as a groundbreaking technology with their unparalleled text generation capabilities across various applications. Nevertheless, concerns persist regarding the accuracy and appropriateness of their generated content. A contemporary methodology, self-correction, has been proposed as a remedy to these issues. Building upon this premise, this paper critically examines the role and efficacy of self-correction within LLMs, shedding light on its true potential and limitations. Central to our investigation is the notion of intrinsic self-correction, whereby an LLM attempts to correct its initial responses based solely on its inherent capabilities, without the crutch of external feedback. In the context of reasoning, our research indicates that LLMs struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction. Drawing from these insights, we offer suggestions for future research and practical applications in this field.

Key facts. Cited by 1,017 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 46 of its 53 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

46 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Shepherd: A Critic for Language Model Generation8 Aug
  2. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies6 Aug
  3. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  5. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  6. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  7. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 39

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2023Let's Verify Step by Stepcited by 3,789
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
27 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2020Extracting Training Data from Large Language Modelscited by 2,189
  3. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  5. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  6. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  7. 2023PaLM 2 Technical Reportcited by 1,162
  8. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  9. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  10. 2023Teaching Large Language Models to Self-Debugcited by 988
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  12. 2023Large Language Models as Optimizerscited by 864
  13. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  14. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  15. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  16. 2023Language Models can Solve Computer Taskscited by 372
  17. 2021True Few-Shot Learning with Language Modelscited by 344
  18. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  19. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  20. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  21. 2022Generating Sequences by Learning to Self-Correctcited by 277
  22. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
  23. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 173
  24. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  25. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 94
  26. 2023Why Does ChatGPT Fall Short in Providing Truthful Answers?cited by 54
  27. 2023Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakagecited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 35; 2024: 220; 2025: 295; 2026: 467.

1,017 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  3. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
  4. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  5. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  6. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  7. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  8. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  9. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  10. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  11. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  12. 2024Iterative Reasoning Preference Optimizationcited by 246
38 more
  1. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  2. 2024AI Agents That Mattercited by 219
  3. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 217
  4. 2025Survey on Evaluation of LLM-based Agentscited by 204
  5. 2023Universal Self-Consistency for Large Language Model Generationcited by 203
  6. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 185
  7. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 180
  8. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 179
  9. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 177
  10. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  11. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
  12. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  13. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
  14. 2024LLM Critics Help Catch LLM Bugscited by 147
  15. 2024Stream of Search (SoS): Learning to Search in Languagecited by 145
  16. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 142
  17. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 138
  18. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 137
  19. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 134
  20. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  21. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  22. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 128
  23. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  24. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  25. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 125
  26. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  27. 2024Emergent social conventions and collective bias in LLM populationscited by 123
  28. 2025Process Reward Models That Thinkcited by 118
  29. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 114
  30. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganographycited by 113
  31. 2025Confidence Improves Self-Consistency in LLMscited by 113
  32. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 111
  33. 2023Branch-Solve-Merge Improves Large Language Model Evaluation and Generationcited by 106
  34. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 103
  35. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 103
  36. 2024On scalable oversight with weak LLMs judging strong LLMscited by 99
  37. 2025Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2cited by 97
  38. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 93

50 most cited of 1,017 All 1,017 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Correcting Large Language Models: Generation vs. Multiple Choice
  2. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  3. 2022Large Language Models Can Self-Improve
  4. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  5. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook
  6. 2025The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution
  7. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  8. 2024S^3cMath: Spontaneous Step-level Self-correction Makes Large Language Models Better Mathematical Reasoners