mlsanityData to 7 Oct 2026

How Linguistics Learned to Stop Worrying and Love the Language Models

arXiv 2501.17047 ↗First posted 28 Jan 2025cs.CLcited by 50

Not on the map. Why?

Abstract (arXiv)

Language models can produce fluent, grammatical text. Nonetheless, some maintain that language models don't really learn language and also that, even if they did, that would not be informative for the study of human learning and processing. On the other side, there have been claims that the success of LMs obviates the need for studying linguistic theory and structure. We argue that both extremes are wrong. LMs can contribute to fundamental questions about linguistic structure, language processing, and learning. They force us to rethink arguments and ways of thinking that have been foundational in linguistics. While they do not replace linguistic structure and theory, they serve as model systems and working proofs of concept for gradient, usage-based approaches to language. We offer an optimistic take on the relationship between language models and linguistics.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 78 of its 356 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

78 of 356 references.

References · 78

  1. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  3. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  4. 2024The Platonic Representation Hypothesiscited by 362
  5. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  6. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  7. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  8. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
  9. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  10. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  11. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  12. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 137
38 more
  1. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  2. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  3. 2018On the Practical Computational Power of Finite Precision RNNs for Language Recognitioncited by 117
  4. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 111
  5. 2018What do RNN Language Models Learn about Filler-Gap Dependencies?cited by 86
  6. 2023Physics of Language Models: Part 1, Learning Hierarchical Language Structurescited by 83
  7. 2022Training Trajectories of Language Models Across Scalescited by 80
  8. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 75
  9. 2024Why are Sensitive Functions Hard for Transformers?cited by 74
  10. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 70
  11. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 69
  12. 2016Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learnercited by 68
  13. 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
  14. 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
  15. 2020RNNs can generate bounded hierarchical languages with optimal memorycited by 59
  16. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 57
  17. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  18. 2024Language models align with human judgments on key grammatical constructionscited by 49
  19. 2024People cannot distinguish GPT-4 from a human in a Turing testcited by 46
  20. 2023Testing AI on language comprehension tasks reveals insensitivity to underlying meaningcited by 45
  21. 2015Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortexcited by 44
  22. 2022Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systemscited by 44
  23. 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
  24. 2021Word Acquisition in Neural Language Modelscited by 41
  25. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 40
  26. 2024Towards a theory of how the structure of language is acquired by deep neural networkscited by 38
  27. 2017Deep Learning Models of the Retinal Response to Natural Scenescited by 36
  28. 2019On the Computational Power of RNNscited by 35
  29. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 35
  30. 2024Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidencecited by 35
  31. 2023Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stabilitycited by 34
  32. 2018Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networkscited by 33
  33. 2023A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learningcited by 32
  34. 2024Mission: Impossible Language Modelscited by 31
  35. 2024Code Pretraining Improves Entity Tracking Abilities of Language Modelscited by 31
  36. 2023BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language modelscited by 29
  37. 2020Universal linguistic inductive biases via meta-learningcited by 28
  38. 2021Sensitivity as a Complexity Measure for Sequence Classification Taskscited by 28

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 30.

50 papers.

Sort:

By area:

  1. 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 28
  2. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  3. 2025Anything Goes? A Crosslinguistic Study of (Im)possible Language Learning in LMscited by 20
  4. 2025Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Modelscited by 18
  5. 2025LLMs as a synthesis between symbolic and distributed approaches to languagecited by 13
  6. 2025Developmentally-plausible Working Memory Shapes a Critical Period for Language Acquisitioncited by 10
  7. 2025Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communicationcited by 9
  8. 2025On the Acquisition of Shared Grammatical Representations in Bilingual Language Modelscited by 7
  9. 2025Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learningcited by 7
  10. 2025Constructions are Revealed in Word Distributionscited by 6
  11. 2026Differential syntactic and semantic encoding in LLMscited by 6
  12. 2025Human-like fleeting memory improves language learning but impairs reading time prediction in transformer language modelscited by 3
38 more
  1. 2025UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?cited by 3
  2. 2025Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossiblecited by 3
  3. 2026Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representationscited by 3
  4. 2025Leveraging Human Production-Interpretation Asymmetries to Test LLM Cognitive Plausibilitycited by 2
  5. 2025Language Models Identify Ambiguities and Exploit Loopholescited by 2
  6. 2025From Zipf's Law to Neural Scaling through Heaps' Law and Hilberg's Hypothesiscited by 2
  7. 2026Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Taskscited by 2
  8. 2025The exponential distribution of the order of demonstrative, numeral, adjective and nouncited by 1
  9. 2025WUGNECTIVES: Novel Entity Inferences of Language Models from Discourse Connectivescited by 1
  10. 2025HSKBenchmark: Modeling and Benchmarking Chinese Second Language Acquisition in Large Language Models through Curriculum Tuningcited by 1
  11. 2026Differences in Typological Alignment in Language Models' Treatment of Differential Argument Markingcited by 1
  12. 2026Perturbation: A simple and efficient adversarial tracer for representation learning in language modelscited by 1
  13. 2026Syntactically-guided Information Maintenance in Sentence Comprehensioncited by 1
  14. 2026Implicit Representations of Grammaticality in Language Modelscited by 1
  15. 2026Developmental approach reveals the statistical learning of Neural Language Models: Transformers generalize from the most abstract statistical patternscited by 1
  16. 2025Do Large Language Models know who did what to whom?cited by 0
  17. 2026Bears, all bears, and some bears. Language Constraints on Language Models' Inductive Inferencescited by 0
  18. 2026Which course? Discourse! Teaching Discourse and Generation in the Era of LLMscited by 0
  19. 2026CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Modelscited by 0
  20. 2026France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctionscited by 0
  21. 2026Vocabulary shapes cross-lingual variation of word-order learnability in language modelscited by 0
  22. 2026Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMscited by 0
  23. 2026Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?cited by 0
  24. 2026Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Modelscited by 0
  25. 2026Tracing the ongoing emergence of human-like reasoning in Large Language Modelscited by 0
  26. 2026Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructionscited by 0
  27. 2026Large Language Models as Modal Models in Linguisticscited by 0
  28. 2026When transformers learn "impossible" languages, what do they learn?cited by 0
  29. 2026Linear representations of grammaticality in neural language modelscited by 0
  30. 2026Computational models of pragmatic reasoning with flexible generation of meaning and expression alternativescited by 0
  31. 2026Cognitive Convergence: Deep Similarities Between Large Language Models and Human Cognitioncited by 0
  32. 2026Children, but not language models, show accelerating returns in word learningcited by 0
  33. 2026Informational Antilocality and the Locality Bias in LLMscited by 0
  34. 2026A Formal Limitation on Learning Human Language From Textual Corporacited by 0
  35. 2026Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Modelscited by 0
  36. 2026Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralizationcited by 0
  37. 2026A retrospective analysis on the use of LLMs to study infant syntax learningcited by 0
  38. 2026Neural Language Models Learn the Contextual Distributions of Dependency Structures: a statistical learning theory to compositionalitycited by 0