mlsanityData to 7 Oct 2026

Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought

arXiv 2210.01240 ↗First posted 3 Oct 2022cs.CLcited by 334

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts (examples with intermediate reasoning steps). Existing benchmarks measure reasoning ability indirectly, by evaluating accuracy on downstream tasks such as mathematical reasoning. However, it is unclear how these models obtain the answers and whether they rely on simple heuristics rather than the generated chain-of-thought. To enable systematic exploration of the reasoning ability of LLMs, we present a new synthetic question-answering dataset called PrOntoQA, where each example is generated from a synthetic world model represented in first-order logic. This allows us to parse the generated chain-of-thought into symbolic proofs for formal analysis. Our analysis on InstructGPT and GPT-3 shows that LLMs are quite capable of making correct individual deduction steps, and so are generally capable of reasoning, even in fictional contexts. However, they have difficulty with proof planning: When multiple valid deduction steps are available, they are not able to systematically explore the different options.

Key facts. Cited by 334 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 30 of its 36 references that are in the map. First posted on arXiv 3 Oct 2022.

Builds on

30 of 36 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
  2. 2022Faithful Reasoning Using Large Language Models30 Aug
  3. 2022Language models show human-like content effects on reasoning tasks14 Jul
  4. 2022On the Paradox of Learning to Reason from Data23 May
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
  6. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoning15 Feb

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  10. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  11. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  12. 2019Abductive Commonsense Reasoningcited by 300
12 more
  1. 2022Exploring Length Generalization in Large Language Modelscited by 204
  2. 2017End-to-End Differentiable Provingcited by 159
  3. 2019Neural Logic Machinescited by 144
  4. 2021Saturated Transformers are Constant-Depth Threshold Circuitscited by 137
  5. 2022Language Model Cascadescited by 104
  6. 2021NaturalProofs: Mathematical Theorem Proving in Natural Languagecited by 80
  7. 2020Leveraging Abstract Meaning Representation for Knowledge Base Question Answeringcited by 53
  8. 2021Can NLI Models Verify QA Systems' Predictions?cited by 46
  9. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  10. 2020Critical Thinking for Language Modelscited by 31
  11. 2021Flexible Generation of Natural Language Deductionscited by 28
  12. 2021Towards General Natural Language Understanding with Probabilistic Worldbuildingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 61; 2024: 89; 2025: 107; 2026: 71.

334 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  3. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  4. 2022A Survey on In-context Learningcited by 717
  5. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  6. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  7. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  8. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  9. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  10. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  11. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  12. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
38 more
  1. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  2. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  3. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystackcited by 211
  4. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  5. 2024Retrieval-Augmented Generation with Graphs (GraphRAG)cited by 170
  6. 2023A Categorical Archive of ChatGPT Failurescited by 167
  7. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  8. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  9. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  10. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  11. 2025BIG-Bench Extra Hardcited by 123
  12. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  13. 2024Large Language Models Can Learn Temporal Reasoningcited by 110
  14. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 102
  15. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
  16. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 100
  17. 2024Faithful Logical Reasoning via Symbolic Chain-of-Thoughtcited by 99
  18. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  19. 2025Efficient Reasoning Models: A Surveycited by 93
  20. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  21. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  22. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  23. 2025Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoningcited by 89
  24. 2025Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoningcited by 87
  25. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  26. 2024Position: Graph Foundation Models are Already Herecited by 84
  27. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 83
  28. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 82
  29. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilitiescited by 82
  30. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 74
  31. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  32. 2023Can GPT-3 Perform Statutory Reasoning?cited by 71
  33. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 71
  34. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  35. 2024Evaluating Mathematical Reasoning Beyond Accuracycited by 69
  36. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 67
  37. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Languagecited by 66
  38. 2023Large Language Models can Learn Rulescited by 66

50 most cited of 334 All 334 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
  2. 2024General Purpose Verification for Chain of Thought Prompting
  3. 2025Shape of Thought: When Distribution Matters More than Correctness in Reasoning Tasks
  4. 2025Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoning
  5. 2025Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework
  6. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
  7. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chains
  8. 2025Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation