Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)
Not on the map. Why?
Abstract (arXiv)
We present a critical assessment of Piantadosi's (2023) claim that "Modern language models refute Chomsky's approach to language," focusing on four main points. First, despite the impressive performance and utility of large language models (LLMs), humans achieve their capacity for language after exposure to several orders of magnitude less data. The fact that young children become competent, fluent speakers of their native languages with relatively little exposure to them is the central mystery of language learning to which Chomsky initially drew attention, and LLMs currently show little promise of solving this mystery. Second, what can the artificial reveal about the natural? Put simply, the implications of LLMs for our understanding of the cognitive structures and mechanisms underlying language and its acquisition are like the implications of airplanes for understanding how birds fly. Third, LLMs cannot constitute scientific theories of language for several reasons, not least of which is that scientific theories must provide interpretable explanations, not just predictions. This leads to our final point: to even determine whether the linguistic and cognitive capabilities of LLMs rival those of humans requires explicating what humans' capacities actually are. In other words, it requires a separate theory of language and cognition; generative linguistics provides precisely such a theory. As such, we conclude that generative linguistics as a scientific discipline will remain indispensable throughout the 21st century and beyond.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 32 of its 128 references that are in the map. First posted on arXiv 6 Aug 2023.
Builds on
32 of 128 references.
References · 32
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2018Neural Network Acceptability Judgmentscited by 958
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
- 2020A Primer in BERTology: What we know about how BERT workscited by 815
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
- 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
- 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
- 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
20 more
- 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
- 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
- 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
- 2020When Do You Need Billions of Words of Pretraining Data?cited by 104
- 2020Syntactic Structure from Deep Learningcited by 99
- 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
- 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
- 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
- 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
- 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
- 2021What if This Modified That? Syntactic Interventions via Counterfactual Embeddingscited by 39
- 2020Artificial neural networks for neuroscientists: A primercited by 26
- 2020What they do when in doubt: a study of inductive biases in seq2seq learnerscited by 20
- 2020GAIT-prop: A biologically plausible learning rule derived from backpropagation of errorcited by 15
- 2019Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflectioncited by 13
- 2020Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German pluralscited by 13
- 2014Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networkscited by 11
- 2017Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasetscited by 11
- 2020Overestimation of Syntactic Representationin Neural Language Modelscited by 4
- 2021The Greedy and Recursive Search for Morphological Productivitycited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 5; 2025: 3.
9 papers.
- 2025Can Language Models Learn Typologically Implausible Languages?cited by 23
- 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 18
- 2025Large Language Models as Proxies for Theories of Human Linguistic Cognitioncited by 10
- 2024A systematic framework for generating novel experimental hypotheses from language modelscited by 9
- 2024No Such Thing as a General Learner: Language models and their dual optimizationcited by 1
- 2024Non-native speakers of English or ChatGPT: Who thinks better?cited by 1
- 2024LLMs as Function Approximators: Terminology, Taxonomy, and Questions for Evaluationcited by 0
- 2024On the Compatibility of Generative AI and Generative Linguisticscited by 0
- 2025Generative Linguistics, Large Language Models, and the Social Nature of Scientific Successcited by 0
- 2025Generative Linguistics, Large Language Models, and the Social Nature of Scientific Success25 Mar 2025
- 2025Can Language Models Learn Typologically Implausible Languages?17 Feb 2025
- 2025Large Language Models as Proxies for Theories of Human Linguistic Cognition11 Feb 2025
- 2024Non-native speakers of English or ChatGPT: Who thinks better?30 Nov 2024
- 2024On the Compatibility of Generative AI and Generative Linguistics15 Nov 2024
- 2024No Such Thing as a General Learner: Language models and their dual optimization18 Aug 2024
- 2024A systematic framework for generating novel experimental hypotheses from language models9 Aug 2024
- 2024LLMs as Function Approximators: Terminology, Taxonomy, and Questions for Evaluation18 Jul 2024
- 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models12 Sep 2023