# Machine Reading, Fast and Slow: When Do Models "Understand" Language?

arXiv: https://arxiv.org/abs/2209.07430 · HTML: https://mlsanity.com/paper/2209.07430/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 88 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 30 of 55)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Pretrained Transformers Improve Out-of-Distribution Robustness](https://mlsanity.com/paper/2004.06100/) (2020)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions](https://mlsanity.com/paper/2005.06676/) (2020)
- [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://mlsanity.com/paper/1908.08962/) (2019)
- [How Can We Accelerate Progress Towards Human-like Linguistic Generalization?](https://mlsanity.com/paper/2005.00955/) (2020)
- [Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning](https://mlsanity.com/paper/1908.05803/) (2019)
- [Competency Problems: On Finding and Removing Artifacts in Language Data](https://mlsanity.com/paper/2104.08646/) (2021)

## Built on by (most cited first, 13 of 13)

- [The Debate Over Understanding in AI's Large Language Models](https://mlsanity.com/paper/2210.13966/) (2022)
- [Position: Key Claims in LLM Research Have a Long Tail of Footnotes](https://mlsanity.com/paper/2308.07120/) (2023)
- [Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses](https://mlsanity.com/paper/2305.11662/) (2023)
- [CLEAR: Can Language Models Really Understand Causal Graphs?](https://mlsanity.com/paper/2406.16605/) (2024)
- [Being Right for Whose Right Reasons?](https://mlsanity.com/paper/2306.00639/) (2023)
- [Natural Language Processing RELIES on Linguistics](https://mlsanity.com/paper/2405.05966/) (2024)
- [From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency](https://mlsanity.com/paper/2404.12145/) (2024)
- [The Two Word Test: A Semantic Benchmark for Large Language Models](https://mlsanity.com/paper/2306.04610/) (2023)
- [Implicit causality in GPT-2: a case study](https://mlsanity.com/paper/2212.04348/) (2022)
- [Research Community Perspectives on "Intelligence" and Large Language Models](https://mlsanity.com/paper/2505.20959/) (2025)
- [Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities](https://mlsanity.com/paper/2510.16815/) (2025)
- [The Overlooked Repetitive Lengthening Form in Sentiment Analysis](https://mlsanity.com/paper/2604.01268/) (2026)
- [FAC2E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognition](https://mlsanity.com/paper/2403.00126/) (2024)

## Abstract (arXiv)

Two of the most fundamental challenges in Natural Language Understanding (NLU) at present are: (a) how to establish whether deep learning-based models score highly on NLU benchmarks for the 'right' reasons; and (b) to understand what those reasons would even be. We investigate the behavior of reading comprehension models with respect to two linguistic 'skills': coreference resolution and comparison. We propose a definition for the reasoning steps expected from a system that would be 'reading slowly', and compare that with the behavior of five models of the BERT family of various sizes, observed through saliency scores and counterfactual explanations. We find that for comparison (but not coreference) the systems based on larger encoders are more likely to rely on the 'right' information, but even they struggle with generalization, suggesting that they still learn specific lexical patterns rather than the general principles of comparison.