# On the proper role of linguistically-oriented deep net analysis in linguistic theorizing

arXiv: https://arxiv.org/abs/2106.08694 · HTML: https://mlsanity.com/paper/2106.08694/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 69 references that are in the map.
- First posted on arXiv 16 Jun 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [What you can cram into a single vector: Probing sentence embeddings for linguistic properties](https://mlsanity.com/paper/1805.01070/) (2018)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Targeted Syntactic Evaluation of Language Models](https://mlsanity.com/paper/1808.09031/) (2018)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [A Convolutional Neural Network for Modelling Sentences](https://mlsanity.com/paper/1404.2188/) (2014)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Explainable Deep Learning: A Field Guide for the Uninitiated](https://mlsanity.com/paper/2004.14545/) (2020)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [The emergence of number and syntax units in LSTM language models](https://mlsanity.com/paper/1903.07435/) (2019)
- [BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance](https://mlsanity.com/paper/1911.02969/) (2019)
- [Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks](https://mlsanity.com/paper/2001.03632/) (2020)
- [Distributional Semantics and Linguistic Theory](https://mlsanity.com/paper/1905.01896/) (2019)
- [Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans](https://mlsanity.com/paper/2006.11098/) (2020)
- [Memorize or generalize? Searching for a compositional RNN in a haystack](https://mlsanity.com/paper/1802.06467/) (2018)
- [RNNs Implicitly Implement Tensor Product Representations](https://mlsanity.com/paper/1812.08718/) (2018)
- [What they do when in doubt: a study of inductive biases in seq2seq learners](https://mlsanity.com/paper/2006.14953/) (2020)

## Built on by (most cited first, 30 of 43)

- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2504.08165/) (2025)
- [A fine-grained comparison of pragmatic language understanding in humans and language models](https://mlsanity.com/paper/2212.06801/) (2022)
- [Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://mlsanity.com/paper/2109.14989/) (2021)
- [Prompting is not a substitute for probability measurements in large language models](https://mlsanity.com/paper/2305.13264/) (2023)
- [Dissociating language and thought in large language models](https://mlsanity.com/paper/2301.06627/) (2023)
- [Can Language Models Learn Typologically Implausible Languages?](https://mlsanity.com/paper/2502.12317/) (2025)
- [A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction](https://mlsanity.com/paper/2301.12564/) (2023)
- [A Sentence is Worth a Thousand Pictures: Can Large Language Models Understand Hum4n L4ngu4ge and the W0rld behind W0rds?](https://mlsanity.com/paper/2308.00109/) (2023)
- [What Can String Probability Tell Us About Grammaticality?](https://mlsanity.com/paper/2510.16227/) (2025)
- [Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Reference](https://mlsanity.com/paper/2404.14883/) (2024)
- [Predicting Fine-Tuning Performance with Probing](https://mlsanity.com/paper/2210.07352/) (2022)
- [A Method for Studying Semantic Construal in Grammatical Constructions with Interpretable Contextual Embedding Spaces](https://mlsanity.com/paper/2305.18598/) (2023)
- [Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations](https://mlsanity.com/paper/2405.13828/) (2024)
- [Scope Ambiguities in Large Language Models](https://mlsanity.com/paper/2404.04332/) (2024)
- [Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?](https://mlsanity.com/paper/2212.10879/) (2022)
- [Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)](https://mlsanity.com/paper/2308.03228/) (2023)
- [Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities](https://mlsanity.com/paper/2310.19626/) (2023)
- [From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency](https://mlsanity.com/paper/2404.12145/) (2024)
- [From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks](https://mlsanity.com/paper/2405.15164/) (2024)
- [Schr"odinger's Tree -- On Syntax and Neural Language Models](https://mlsanity.com/paper/2110.08887/) (2021)
- [Learning the meanings of function words from grounded language using a visual question answering model](https://mlsanity.com/paper/2308.08628/) (2023)
- [Transformers, Contextualism, and Polysemy](https://mlsanity.com/paper/2404.09577/) (2024)
- [Philosophy of Cognitive Science in the Age of Deep Learning](https://mlsanity.com/paper/2405.04048/) (2024)
- [Computational Models to Study Language Processing in the Human Brain: A Survey](https://mlsanity.com/paper/2403.13368/) (2024)
- [Interpretability of Language Models via Task Spaces](https://mlsanity.com/paper/2406.06441/) (2024)
- [Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models](https://mlsanity.com/paper/2505.05970/) (2025)
- [How Does Adversarial Fine-Tuning Benefit BERT?](https://mlsanity.com/paper/2108.13602/) (2021)
- [Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossible](https://mlsanity.com/paper/2510.07178/) (2025)

## Abstract (arXiv)

A lively research field has recently emerged that uses experimental methods to probe the linguistic behavior of modern deep networks. While work in this tradition often reports intriguing results about the grammatical skills of deep nets, it is not clear what their implications for linguistic theorizing should be. As a consequence, linguistically-oriented deep net analysis has had very little impact on linguistics at large. In this chapter, I suggest that deep networks should be treated as theories making explicit predictions about the acceptability of linguistic utterances. I argue that, if we overcome some obstacles standing in the way of seriously pursuing this idea, we will gain a powerful new theoretical tool, complementary to mainstream algebraic approaches.