mlsanityData to 7 Oct 2026

Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks

arXiv 2307.02477 ↗First posted 5 Jul 2023cs.CL cs.AIcited by 243

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

The impressive performance of recent language models across a wide range of tasks suggests that they possess a degree of abstract reasoning skills. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? To disentangle these effects, we propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks. Across a suite of 11 tasks, we observe nontrivial performance on the counterfactual variants, but nevertheless find that performance substantially and consistently degrades compared to the default conditions. This suggests that while current LMs may possess abstract task-solving skills to an extent, they often also rely on narrow, non-transferable procedures for task-solving. These results motivate a more careful interpretation of language model performance that teases apart these aspects of behavior.

Key facts. Cited by 243 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 86 of its 133 references that are in the map. First posted on arXiv 5 Jul 2023.

Builds on

86 of 133 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought22 Jun
  2. 2023Probing Physical Reasoning with Counter-Commonsense Context4 Jun
  3. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners24 May
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debate23 May
  5. 2023PaLM 2 Technical Report17 May
  6. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
  7. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
  8. 2022Language models show human-like content effects on reasoning tasks14 Jul
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 77

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
38 more
  1. 2020Extracting Training Data from Large Language Modelscited by 2,144
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  4. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  5. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  7. 2017Counterfactual Fairnesscited by 802
  8. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  11. 2019Designing and Interpreting Probes with Control Taskscited by 608
  12. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  13. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  14. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
  15. 2023Simple and Controllable Music Generationcited by 480
  16. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  17. 2023MusicLM: Generating Music From Textcited by 442
  18. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  19. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  20. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  21. 2021Causal Abstractions of Neural Networkscited by 401
  22. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  23. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  24. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  25. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  26. 2023How Language Model Hallucinations Can Snowballcited by 310
  27. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  28. 2020Transformers as Soft Reasoners over Languagecited by 265
  29. 2022Data Contamination: From Memorization to Exploitationcited by 234
  30. 2023Inverse Scaling: When Bigger Isn't Bettercited by 205
  31. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  32. 2017Fair Inference On Outcomescited by 143
  33. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  34. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 108
  35. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  36. 2020Syntactic Structure from Deep Learningcited by 99
  37. 2019Counterfactual Story Reasoning and Generationcited by 87
  38. 2022Meaning without reference in large language modelscited by 76

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 38; 2024: 94; 2025: 79; 2026: 32.

243 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  3. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  4. 2023Challenges and Applications of Large Language Modelscited by 251
  5. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  6. 2023What Algorithms can Transformers Learn? A Study in Length Generalizationcited by 198
  7. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
  8. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 151
  9. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Modelscited by 143
  10. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  11. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Datacited by 124
  12. 2024Empowering Biomedical Discovery with AI Agentscited by 117
38 more
  1. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 117
  2. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  3. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  4. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 100
  5. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 89
  6. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 88
  7. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 88
  8. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 82
  9. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  10. 2023A Benchmark for Learning to Translate a New Language from One Grammar Bookcited by 74
  11. 2024Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelscited by 67
  12. 2023Large Language Models can Learn Rulescited by 66
  13. 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 61
  14. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 54
  15. 2023Prompt Engineering a Prompt Engineercited by 53
  16. 2024Measuring and Controlling Instruction (In)Stability in Language Model Dialogscited by 51
  17. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  18. 2023Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoningcited by 49
  19. 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 49
  20. 2023In-Context Learning Learns Label Relationships but Is Not Conventional Learningcited by 48
  21. 2023From task structures to world models: What do LLMs know?cited by 46
  22. 2024VLind-Bench: Measuring Language Priors in Large Vision-Language Modelscited by 44
  23. 2024LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planningcited by 44
  24. 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 44
  25. 2024CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilitiescited by 43
  26. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approachescited by 43
  27. 2024Towards Scientific Discovery with Generative AI: Progress, Opportunities, and Challengescited by 41
  28. 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?cited by 40
  29. 2024Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Modelscited by 38
  30. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?cited by 36
  31. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  32. 2024Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Modelscited by 34
  33. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 33
  34. 2023Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledgecited by 32
  35. 2023Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encodingcited by 32
  36. 2024Finding Blind Spots in Evaluator LLMs with Interpretable Checklistscited by 32
  37. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Modelscited by 31
  38. 2025Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Modelscited by 31

50 most cited of 243 All 243 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026A Systematic Study of Small Language Models on Abstract Reasoning Tasks
  2. 2022Counterfactual reasoning: Do language models need world knowledge for causal understanding?
  3. 2025Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning
  4. 2023Prompting Large Language Models for Counterfactual Generation: An Empirical Study
  5. 2024Revisiting the Superficial Alignment Hypothesis
  6. 2025Reasoning Capabilities of Large Language Models on Dynamic Tasks
  7. 2025On the Limits of Innate Planning in Large Language Models
  8. 2025Language Mixing in Reasoning Language Models: Patterns, Impact, and Internal Causes