mlsanityData to 14 Oct 2026

Language models as tools for investigating the distinction between possible and impossible natural languages

arXiv 2512.09394 ↗First posted 10 Dec 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We argue that language models (LMs) have strong potential as investigative tools for probing the distinction between possible and impossible natural languages and thus uncovering the inductive biases that support human language learning. We outline a phased research program in which LM architectures are iteratively refined to better discriminate between possible and impossible languages, supporting linking hypotheses to human cognition.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 98 of its 356 references that are in the map. First posted on arXiv 10 Dec 2025.

Builds on

98 of 356 references.

References · 98

  1. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  2. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  3. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  4. 2024The Platonic Representation Hypothesiscited by 382
  5. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  6. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 288
  7. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  8. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
  9. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  10. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  11. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  12. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 138
38 more
  1. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  2. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  3. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 118
  4. 2018On the Practical Computational Power of Finite Precision RNNs for Language Recognitioncited by 117
  5. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  6. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 88
  7. 2018What do RNN Language Models Learn about Filler-Gap Dependencies?cited by 86
  8. 2023Physics of Language Models: Part 1, Learning Hierarchical Language Structurescited by 83
  9. 2022Training Trajectories of Language Models Across Scalescited by 81
  10. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
  11. 2024Why are Sensitive Functions Hard for Transformers?cited by 74
  12. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 71
  13. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 70
  14. 2016Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learnercited by 68
  15. 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
  16. 2023What Do Self-Supervised Speech Models Know About Words?cited by 64
  17. 2020RNNs can generate bounded hierarchical languages with optimal memorycited by 59
  18. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languagescited by 59
  19. 2023Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generatorscited by 56
  20. 2025Deep Learning is Not So Mysterious or Differentcited by 56
  21. 2024Language models align with human judgments on key grammatical constructionscited by 49
  22. 2024People cannot distinguish GPT-4 from a human in a Turing testcited by 48
  23. 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Modelscited by 47
  24. 2015Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortexcited by 45
  25. 2023Testing AI on language comprehension tasks reveals insensitivity to underlying meaningcited by 45
  26. 2022Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systemscited by 44
  27. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 42
  28. 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
  29. 2021Word Acquisition in Neural Language Modelscited by 41
  30. 2024Towards a theory of how the structure of language is acquired by deep neural networkscited by 39
  31. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 37
  32. 2017Deep Learning Models of the Retinal Response to Natural Scenescited by 36
  33. 2023Probing self-supervised speech models for phonetic and phonemic information: a case study in aspirationcited by 36
  34. 2019On the Computational Power of RNNscited by 35
  35. 2023Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stabilitycited by 35
  36. 2024Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidencecited by 35
  37. 2018Revisiting the poverty of the stimulus: hierarchical generalization without a hierarchical bias in recurrent neural networkscited by 33
  38. 2023A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learningcited by 33

50 most cited of 98 All 98 (JSON)

Built on by

Not cited yet.