# Distinct patterns of syntactic agreement errors in recurrent networks and humans

arXiv: https://arxiv.org/abs/1807.06882 · HTML: https://mlsanity.com/paper/1807.06882/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2018.

## Built on by (most cited first, 13 of 13)

- [Refining Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/2104.09635/) (2021)
- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT](https://mlsanity.com/paper/2209.10538/) (2022)
- [Blackbird's language matrices (BLMs): a new benchmark to investigate disentangled generalisation in neural networks](https://mlsanity.com/paper/2205.10866/) (2022)
- [Grammatical information in BERT sentence embeddings as two-dimensional arrays](https://mlsanity.com/paper/2312.09890/) (2023)
- [A computational psycholinguistic evaluation of the syntactic abilities of Galician BERT models at the interface of dependency resolution and training time](https://mlsanity.com/paper/2206.02440/) (2022)
- [Evaluating Models of Robust Word Recognition with Serial Reproduction](https://mlsanity.com/paper/2101.09788/) (2021)
- [Diverging Transformer Predictions for Human Sentence Processing: A Comprehensive Analysis of Agreement Attraction Effects](https://mlsanity.com/paper/2603.16574/) (2026)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Is the Computation of Abstract Sameness Relations Human-Like in Neural Language Models?](https://mlsanity.com/paper/2205.06149/) (2022)
- [Fixing the problems of deep neural networks will require better training data and learning algorithms](https://mlsanity.com/paper/2311.12819/) (2023)
- [Contextualized word senses: from attention to compositionality](https://mlsanity.com/paper/2312.00680/) (2023)
- [Hitting "Probe"rty with Non-Linearity, and More](https://mlsanity.com/paper/2402.16168/) (2024)

## Abstract (arXiv)

Determining the correct form of a verb in context requires an understanding of the syntactic structure of the sentence. Recurrent neural networks have been shown to perform this task with an error rate comparable to humans, despite the fact that they are not designed with explicit syntactic representations. To examine the extent to which the syntactic representations of these networks are similar to those used by humans when processing sentences, we compare the detailed pattern of errors that RNNs and humans make on this task. Despite significant similarities (attraction errors, asymmetry between singular and plural subjects), the error patterns differed in important ways. In particular, in complex sentences with relative clauses error rates increased in RNNs but decreased in humans. Furthermore, RNNs showed a cumulative effect of attractors but humans did not. We conclude that at least in some respects the syntactic representations acquired by RNNs are fundamentally different from those used by humans.