mlsanityData to 14 Oct 2026

Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey

arXiv 2404.01869 ↗First posted 2 Apr 2024cs.CL cs.AIcited by 83

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have recently shown impressive performance on tasks involving reasoning, leading to a lively debate on whether these models possess reasoning capabilities similar to humans. However, despite these successes, the depth of LLMs' reasoning abilities remains uncertain. This uncertainty partly stems from the predominant focus on task performance, measured through shallow accuracy metrics, rather than a thorough investigation of the models' reasoning behavior. This paper seeks to address this gap by providing a comprehensive review of studies that go beyond task accuracy, offering deeper insights into the models' reasoning processes. Furthermore, we survey prevalent methodologies to evaluate the reasoning behavior of LLMs, emphasizing current trends and efforts towards more nuanced reasoning analyses. Our review suggests that LLMs tend to rely on surface-level patterns and correlations in their training data, rather than on sophisticated reasoning abilities. Additionally, we identify the need for further research that delineates the key differences between human and LLM-based reasoning. Through this survey, we aim to shed light on the complex reasoning processes within LLMs.

Key facts. Cited by 83 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 69 of its 116 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

69 of 116 references.

Closest ancestors · 28

Same direction, newest first

  1. 2024Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoning20 Feb
  2. 2024LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models1 Jan
  3. 2023Assessing Logical Reasoning Capabilities of Encoder-Only Transformer Models18 Dec
  4. 2023A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models1 Nov
  5. 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models23 Oct
  6. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement12 Oct
  7. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks5 Jul
  8. 2023Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases21 Jun
  9. 2023Inductive reasoning in humans and large language models11 Jun
  10. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance26 May
  11. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples24 May
  12. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate22 May
16 more
  1. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativeness21 Apr
  2. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-47 Apr
  3. 2023Natural Language Reasoning, A Survey26 Mar
  4. 2023Logical Reasoning over Natural Language as Knowledge Representation: A Survey21 Mar
  5. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  6. 2023Augmented Language Models: a Survey15 Feb
  7. 2022Language Models as Inductive Reasoners21 Dec
  8. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  9. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning15 Dec
  10. 2022Thinking Fast and Slow in Large Language Models10 Dec
  11. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  12. 2022Language models show human-like content effects on reasoning tasks14 Jul
  13. 2022RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoning25 May
  14. 2022On the Paradox of Learning to Reason from Data23 May
  15. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks11 May
  16. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning6 May

Other references · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  11. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
29 more
  1. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  3. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  4. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  5. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  6. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  7. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  8. 2023Inverse Scaling: When Bigger Isn't Bettercited by 207
  9. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  10. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  11. 2022Talking About Large Language Modelscited by 175
  12. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  13. 2023Can Large Language Models Infer Causation from Correlation?cited by 146
  14. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  15. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  16. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
  17. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 72
  18. 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
  19. 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Biascited by 53
  20. 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoningcited by 53
  21. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 34
  22. 2022Can Foundation Models Talk Causality?cited by 30
  23. 2023Counterfactual reasoning: Testing language models' understanding of hypothetical scenarioscited by 27
  24. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 26
  25. 2023Mathematical Language Models: A Surveycited by 24
  26. 2023Is Bigger and Deeper Always Better? Probing LLaMA Across Scales and Layerscited by 20
  27. 2022Towards Understanding How Machines Can Learn Causal Overhypothesescited by 14
  28. 2023CLOMO: Counterfactual Logical Modification with Large Language Modelscited by 7
  29. 2023Understanding Natural Language Understanding Systems. A Critical Analysiscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 18; 2025: 43; 2026: 22.

83 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  3. 2024Can Language Models Learn to Skip Steps?cited by 59
  4. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  5. 2025A Survey of Large Language Model Agents for Question Answeringcited by 48
  6. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  7. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?cited by 38
  8. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  9. 2025Generative to Agentic AI: Survey, Conceptualization, and Challengescited by 33
  10. 2025MINERVA: Evaluating Complex Video Reasoningcited by 32
  11. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Modelscited by 31
  12. 2025The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofscited by 30
38 more
  1. 2024TS-Reasoner: Domain-Oriented Time Series Inference Agents for Reasoning and Automated Analysiscited by 29
  2. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  3. 2024100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instancescited by 24
  4. 2025FlairGPT: Repurposing LLMs for Interior Designscited by 24
  5. 2024iLLM-TSC: Integration reinforcement learning and large language model for traffic signal control policy improvementcited by 23
  6. 2024DetectiveQA: Evaluating Long-Context Reasoning on Detective Novelscited by 21
  7. 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Modelscited by 20
  8. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 16
  9. 2024Uncovering Latent Chain of Thought Vectors in Language Modelscited by 14
  10. 2025Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignmentcited by 12
  11. 2025How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMscited by 12
  12. 2026AgenticPay: A Multi-Agent LLM Negotiation System for Buyer-Seller Transactionscited by 12
  13. 2025Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoningcited by 9
  14. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  15. 2025ReasonBENCH: Benchmarking the (In)Stability of LLM Reasoningcited by 9
  16. 2026Advancing General-Purpose Reasoning Models with Modular Gradient Surgerycited by 9
  17. 2025PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehuntscited by 8
  18. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 8
  19. 2024Q-Adapter: Customizing Pre-trained LLMs to New Preferences with Forgetting Mitigationcited by 7
  20. 2025Beyond Accuracy: Dissecting Mathematical Reasoning for LLMs Under Reinforcement Learningcited by 7
  21. 2024Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approachcited by 6
  22. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Modelscited by 6
  23. 2026Reasoning emerges from constrained inference manifolds in large language modelscited by 6
  24. 2024LLM-based Discriminative Reasoning for Knowledge Graph Question Answeringcited by 5
  25. 2025OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Modelscited by 5
  26. 2025Agent-to-Agent Theory of Mind: Testing Interlocutor Awareness among Large Language Modelscited by 5
  27. 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoningcited by 5
  28. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbationscited by 5
  29. 2024DeFine: Decision-Making with Analogical Reasoning over Factor Profilescited by 4
  30. 2025Explaining Large Language Models with gSMILEcited by 4
  31. 2025Counterfactual reasoning: an analysis of in-context emergencecited by 4
  32. 2025Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generationcited by 4
  33. 2025"To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarioscited by 3
  34. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Modelscited by 2
  35. 2025LLM-ODDR: A Large Language Model Framework for Joint Order Dispatching and Driver Repositioningcited by 2
  36. 2025SCOPE: Stochastic and Counterbiased Option Placement for Evaluating Large Language Modelscited by 2
  37. 2025Beyond Ethical Alignment: Evaluating LLMs as Artificial Moral Assistantscited by 2
  38. 2025On the Notion that Language Models Reasoncited by 2

50 most cited of 83 All 83 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning
  2. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  3. 2025Reasoning Capabilities and Invariability of Large Language Models
  4. 2025Beyond Chains of Thought: Benchmarking Latent-Space Reasoning Abilities in Large Language Models
  5. 2025Towards Reasoning Ability of Small Language Models
  6. 2026Large Language Model Reasoning Failures
  7. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus
  8. 2025A Survey of Scaling in Large Language Model Reasoning