mlsanityData to 7 Oct 2026

On the Paradox of Learning to Reason from Data

arXiv 2205.11502 ↗First posted 23 May 2022cs.CL cs.AIcited by 93

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Logical reasoning is needed in a wide range of NLP tasks. Can a BERT model be trained end-to-end to solve logical reasoning problems presented in natural language? We attempt to answer this question in a confined problem space where there exists a set of parameters that perfectly simulates logical reasoning. We make observations that seem to contradict each other: BERT attains near-perfect accuracy on in-distribution test examples while failing to generalize to other data distributions over the exact same problem space. Our study provides an explanation for this paradox: instead of learning to emulate the correct reasoning function, BERT has in fact learned statistical features that inherently exist in logical reasoning problems. We also show that it is infeasible to jointly remove statistical features from data, illustrating the difficulty of learning to reason in general. Our result naturally extends to other neural models and unveils the fundamental difference between learning to reason and learning to achieve high performance on NLP benchmarks using statistical features.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 35 of its 54 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

35 of 54 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  11. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  12. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
23 more
  1. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  2. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  3. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  4. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  5. 2019Deep Learning for Symbolic Mathematicscited by 266
  6. 2020Transformers as Soft Reasoners over Languagecited by 265
  7. 2020Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learningcited by 216
  8. 2020ReClor: A Reading Comprehension Dataset Requiring Logical Reasoningcited by 211
  9. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 198
  10. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  11. 2019What Can Neural Networks Reason About?cited by 155
  12. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 155
  13. 2021On Calibration and Out-of-domain Generalizationcited by 134
  14. 2019Unlearn Dataset Bias in Natural Language Inference by Fitting the Residualcited by 119
  15. 2021NaturalProofs: Mathematical Theorem Proving in Natural Languagecited by 80
  16. 2019Reasoning Over Paragraph Effects in Situationscited by 79
  17. 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
  18. 2020Teaching Temporal Logics to Neural Networkscited by 51
  19. 2020Measuring Systematic Generalization in Neural Proof Generation with Transformerscited by 43
  20. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  21. 2019Improving Graph Neural Network Representations of Logical Formulae with Subgraph Poolingcited by 20
  22. 2020On the Robustness of Language Encoders against Grammatical Errorscited by 18
  23. 2020An Experimental Study of Formula Embeddings for Automated Theorem Proving in First-Order Logiccited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 25; 2024: 28; 2025: 20; 2026: 9.

93 papers.

Sort:

By area:

  1. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  4. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  5. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  6. 2023Challenges and Applications of Large Language Modelscited by 251
  7. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  8. 2024The pitfalls of next-token predictioncited by 156
  9. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 146
  10. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  11. 2022Faithful Reasoning Using Large Language Modelscited by 130
  12. 2023On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)cited by 85
38 more
  1. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  2. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 82
  3. 2024Formal Mathematical Reasoning: A New Frontier in AIcited by 80
  4. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 71
  5. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Modelscited by 67
  6. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Languagecited by 66
  7. 2023Tractable Control for Autoregressive Language Generationcited by 65
  8. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  9. 2023Natural Language Reasoning, A Surveycited by 61
  10. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 56
  11. 2023BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Informationcited by 42
  12. 2024Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Modelscited by 38
  13. 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
  14. 2022Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge networkcited by 34
  15. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  16. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  17. 2025Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoningcited by 29
  18. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  19. 2023Dissociating language and thought in large language modelscited by 27
  20. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 26
  21. 2022RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoningcited by 22
  22. 2024LLMs Are Prone to Fallacies in Causal Inferencecited by 21
  23. 2025JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Modelscited by 21
  24. 2025Frontier LLMs Still Struggle with Simple Reasoning Taskscited by 20
  25. 2026Large Language Model Reasoning Failurescited by 20
  26. 2024Transformers Struggle to Learn to Searchcited by 19
  27. 2022Unpacking Large Language Models with Conceptual Consistencycited by 18
  28. 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logiccited by 18
  29. 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programmingcited by 18
  30. 2024Can Transformers Reason Logically? A Study in SAT Solvingcited by 16
  31. 2024Transformer-Based Models Are Not Yet Perfect At Learning to Emulate Structural Recursioncited by 14
  32. 2024Reliable Reasoning Beyond Natural Languagecited by 14
  33. 2024SocraSynth: Multi-LLM Reasoning with Conditional Statisticscited by 13
  34. 2025Have Large Language Models Learned to Reason? A Characterization via 3-SAT Phase Transitioncited by 13
  35. 2023Explicit Planning Helps Language Models in Logical Reasoningcited by 12
  36. 2022A Simple, Yet Effective Approach to Finding Biases in Code Generationcited by 8
  37. 2023Certified Deductive Reasoning with Language Modelscited by 8
  38. 2024Can Large Language Models Act as Symbolic Reasoners?cited by 8

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Benchmarking LLM Competence on Logical Inference over Probability Operators
  2. 2025On the Notion that Language Models Reason
  3. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
  4. 2022Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
  5. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
  6. 2025Learning to Reason via Mixture-of-Thought for Logical Reasoning
  7. 2023A Simple Generative Model of Logical Reasoning and Statistical Learning
  8. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning