mlsanityData to 7 Oct 2026

Generalizations across filler-gap dependencies in neural language models

arXiv 2410.18225 ↗First posted 23 Oct 2024cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Humans develop their grammars by making structural generalizations from finite input. We ask how filler-gap dependencies, which share a structural generalization despite diverse surface forms, might arise from the input. We explicitly control the input to a neural language model (NLM) to uncover whether the model posits a shared representation for filler-gap dependencies. We show that while NLMs do have success differentiating grammatical from ungrammatical filler-gap dependencies, they rely on superficial properties of the input, rather than on a shared generalization. Our work highlights the need for specific linguistic inductive biases to model language acquisition.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 10 of its 38 references that are in the map. First posted on arXiv 23 Oct 2024.

Builds on

10 of 38 references.

References · 10

  1. 2018Colorless green recurrent networks dream hierarchicallycited by 210
  2. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 137
  3. 2020Syntactic Structure from Deep Learningcited by 99
  4. 2018What do RNN Language Models Learn about Filler-Gap Dependencies?cited by 86
  5. 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
  6. 2023Grokking of Hierarchical Structure in Vanilla Transformerscited by 53
  7. 2024Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidencecited by 35
  8. 2021Language Models Use Monotonicity to Assess NPI Licensingcited by 30
  9. 2019Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Modelscited by 29
  10. 2019Quantity doesn't buy quality syntax with neural language modelscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 50
  2. 2025For GPT-4 as with Humans: Information Structure Predicts Acceptability of Long-Distance Dependenciescited by 1
  3. 2025mGRADE: Minimal Recurrent Gating Meets Delay Convolutions for Lightweight Sequence Modelingcited by 0
  4. 2025Language models as tools for investigating the distinction between possible and impossible natural languagescited by 0
  5. 2025Large language models have learned to use languagecited by 0
  6. 2025Large language models are not about natural languagecited by 0
  7. 2025Linguists should learn to love speech-based deep learning modelscited by 0
  8. 2026Are Language Models Models?cited by 0
  9. 2026Across the Levels of Analysis: Explaining Predictive Processing in Humans Requires More Than Machine-Estimated Probabilitiescited by 0
  10. 2026You Can't Fight in Here! This is BBS!cited by 0
  11. 2026Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMscited by 0
  12. 2026Not-So-Strange Love: Language Models and Generative Linguistic Theories are More Compatible than They Appearcited by 0