# Is the Computation of Abstract Sameness Relations Human-Like in Neural Language Models?

arXiv: https://arxiv.org/abs/2205.06149 · HTML: https://mlsanity.com/paper/2205.06149/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 12 May 2022.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [Good-Enough Compositional Data Augmentation](https://mlsanity.com/paper/1904.09545/) (2019)
- [Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?](https://mlsanity.com/paper/2010.12725/) (2020)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Open Sesame: Getting Inside BERT's Linguistic Knowledge](https://mlsanity.com/paper/1906.01698/) (2019)
- [What do RNN Language Models Learn about Filler-Gap Dependencies?](https://mlsanity.com/paper/1809.00042/) (2018)
- [Span-based Semantic Parsing for Compositional Generalization](https://mlsanity.com/paper/2009.06040/) (2020)
- [Learning to Recombine and Resample Data for Compositional Generalization](https://mlsanity.com/paper/2010.03706/) (2020)
- [Meta-Learning to Compositionally Generalize](https://mlsanity.com/paper/2106.04252/) (2021)
- [Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs](https://mlsanity.com/paper/1909.02597/) (2019)
- [Compositional generalization in a deep seq2seq model by separating syntax and semantics](https://mlsanity.com/paper/1904.09708/) (2019)
- [Compositional Generalization for Primitive Substitutions](https://mlsanity.com/paper/1910.02612/) (2019)
- [Examining the Inductive Bias of Neural Language Models with Artificial Languages](https://mlsanity.com/paper/2106.01044/) (2021)
- [Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages](https://mlsanity.com/paper/1903.06400/) (2019)
- [Understanding Human Intelligence through Human Limitations](https://mlsanity.com/paper/2009.14050/) (2020)
- [Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures](https://mlsanity.com/paper/1808.08946/) (2018)
- [Tree-structured composition in neural networks without tree-structured architectures](https://mlsanity.com/paper/1506.04834/) (2015)
- [The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages](https://mlsanity.com/paper/1710.03838/) (2017)
- [Distinct patterns of syntactic agreement errors in recurrent networks and humans](https://mlsanity.com/paper/1807.06882/) (2018)
- [Transformer-Based Neural Text Generation with Syntactic Guidance](https://mlsanity.com/paper/2010.01737/) (2020)

## Abstract (arXiv)

In recent years, deep neural language models have made strong progress in various NLP tasks. This work explores one facet of the question whether state-of-the-art NLP models exhibit elementary mechanisms known from human cognition. The exploration is focused on a relatively primitive mechanism for which there is a lot of evidence from various psycholinguistic experiments with infants. The computation of "abstract sameness relations" is assumed to play an important role in human language acquisition and processing, especially in learning more complex grammar rules. In order to investigate this mechanism in BERT and other pre-trained language models (PLMs), the experiment designs from studies with infants were taken as the starting point. On this basis, we designed experimental settings in which each element from the original studies was mapped to a component of language models. Even though the task in our experiments was relatively simple, the results suggest that the cognitive faculty of computing abstract sameness relations is stronger in infants than in all investigated PLMs.