mlsanityData to 7 Oct 2026

GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problems

arXiv 2310.12397 ↗First posted 19 Oct 2023cs.AIcited by 88

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

There has been considerable divergence of opinion on the reasoning abilities of Large Language Models (LLMs). While the initial optimism that reasoning might emerge automatically with scale has been tempered thanks to a slew of counterexamples, a wide spread belief in their iterative self-critique capabilities persists. In this paper, we set out to systematically investigate the effectiveness of iterative prompting of LLMs in the context of Graph Coloring, a canonical NP-complete reasoning problem that is related to propositional satisfiability as well as practical problems like scheduling and allocation. We present a principled empirical study of the performance of GPT4 in solving graph coloring instances or verifying the correctness of candidate colorings. In iterative modes, we experiment with the model critiquing its own answers and an external correct reasoner verifying proposed solutions. In both cases, we analyze whether the content of the criticisms actually affects bottom line performance. The study seems to indicate that (i) LLMs are bad at solving graph coloring instances (ii) they are no better at verifying a solution--and thus are not effective in iterative modes with LLMs critiquing LLM-generated solutions (iii) the correctness and content of the criticisms--whether by LLMs or external solvers--seems largely irrelevant to the performance of iterative prompting. We show that the observed increase in effectiveness is largely due to the correct solution being fortuitously present in the top-k completions of the prompt (and being recognized as such by an external verifier). Our results thus call into question claims about the self-critiquing capabilities of state of the art LLMs.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 14 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

10 of 14 references.

References · 10

  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023Teaching Large Language Models to Self-Debugcited by 963
  6. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  7. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  8. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  9. 2023ARB: Advanced Reasoning Benchmark for Large Language Modelscited by 60
  10. 2023GPT-4 Can't Reasoncited by 39

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 14; 2024: 40; 2025: 25; 2026: 9.

88 papers.

Sort:

By area:

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  4. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  5. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
  6. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  7. 2025Process Reward Models That Thinkcited by 118
  8. 2024Let Your Graph Do the Talking: Encoding Structured Data for LLMscited by 112
  9. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinementcited by 111
  10. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  11. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  12. 2024Position: Graph Foundation Models are Already Herecited by 84
38 more
  1. 2024Can Large Language Models Reason and Plan?cited by 82
  2. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  3. 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 78
  4. 2023NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classescited by 78
  5. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 73
  6. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agentscited by 72
  7. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectivescited by 71
  8. 2023CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generationcited by 65
  9. 2024Understanding Transformer Reasoning Capabilities via Graph Algorithmscited by 64
  10. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  11. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 52
  12. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  13. 2024Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Modelscited by 47
  14. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 46
  15. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correctioncited by 45
  16. 2025Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMscited by 45
  17. 2024Enhance Reasoning for Large Language Models in the Game Werewolfcited by 44
  18. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  19. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
  20. 2024MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoningcited by 40
  21. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  22. 2025LLMs as Planning Formalizers: A Survey for Leveraging Large Language Models to Construct Automated Planning Modelscited by 38
  23. 2024Are Large Language Models Good Prompt Optimizers?cited by 37
  24. 2023Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoningcited by 30
  25. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  26. 2024Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMscited by 28
  27. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  28. 2023Structured Chemistry Reasoning with Large Language Modelscited by 27
  29. 2024SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responsescited by 24
  30. 2024Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviorscited by 23
  31. 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23
  32. 2024On Provable Length and Compositional Generalizationcited by 22
  33. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiencycited by 21
  34. 2024The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?cited by 20
  35. 2024Self-Reflection Makes Large Language Models Safer, Less Biased, and Ideologically Neutralcited by 20
  36. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 19
  37. 2024Counterfactual Debating with Preset Stances for Hallucination Elimination of LLMscited by 17
  38. 2024Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attackscited by 17

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
  2. 2025Evaluating the Systematic Reasoning Abilities of Large Language Models through Graph Coloring
  3. 2023Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought
  4. 2024Graph Reasoning with Large Language Models via Pseudo-code Prompting
  5. 2025Rethinking and Benchmarking Large Language Models for Graph Reasoning
  6. 2025Improving LLMs' Generalized Reasoning Abilities by Graph Problems
  7. 2024Can LLMs perform structured graph reasoning?
  8. 2023Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis