mlsanityData to 14 Oct 2026

Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)

arXiv 2308.03228 ↗First posted 6 Aug 2023cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We present a critical assessment of Piantadosi's (2023) claim that "Modern language models refute Chomsky's approach to language," focusing on four main points. First, despite the impressive performance and utility of large language models (LLMs), humans achieve their capacity for language after exposure to several orders of magnitude less data. The fact that young children become competent, fluent speakers of their native languages with relatively little exposure to them is the central mystery of language learning to which Chomsky initially drew attention, and LLMs currently show little promise of solving this mystery. Second, what can the artificial reveal about the natural? Put simply, the implications of LLMs for our understanding of the cognitive structures and mechanisms underlying language and its acquisition are like the implications of airplanes for understanding how birds fly. Third, LLMs cannot constitute scientific theories of language for several reasons, not least of which is that scientific theories must provide interpretable explanations, not just predictions. This leads to our final point: to even determine whether the linguistic and cognitive capabilities of LLMs rival those of humans requires explicating what humans' capacities actually are. In other words, it requires a separate theory of language and cognition; generative linguistics provides precisely such a theory. As such, we conclude that generative linguistics as a scientific discipline will remain indispensable throughout the 21st century and beyond.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 32 of its 128 references that are in the map. First posted on arXiv 6 Aug 2023.

Builds on

32 of 128 references.

References · 32

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  6. 2018Neural Network Acceptability Judgmentscited by 958
  7. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  8. 2020A Primer in BERTology: What we know about how BERT workscited by 815
  9. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  10. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
  11. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
  12. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
20 more
  1. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  2. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  3. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
  4. 2020When Do You Need Billions of Words of Pretraining Data?cited by 104
  5. 2020Syntactic Structure from Deep Learningcited by 99
  6. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  7. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  8. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  9. 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
  10. 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
  11. 2021What if This Modified That? Syntactic Interventions via Counterfactual Embeddingscited by 39
  12. 2020Artificial neural networks for neuroscientists: A primercited by 26
  13. 2020What they do when in doubt: a study of inductive biases in seq2seq learnerscited by 20
  14. 2020GAIT-prop: A biologically plausible learning rule derived from backpropagation of errorcited by 15
  15. 2019Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflectioncited by 13
  16. 2020Inflecting when there's no majority: Limitations of encoder-decoder neural networks as cognitive models for German pluralscited by 13
  17. 2014Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networkscited by 11
  18. 2017Being Negative but Constructively: Lessons Learnt from Creating Better Visual Question Answering Datasetscited by 11
  19. 2020Overestimation of Syntactic Representationin Neural Language Modelscited by 4
  20. 2021The Greedy and Recursive Search for Morphological Productivitycited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 5; 2025: 3.

9 papers.