mlsanityData to 14 Oct 2026

Colorless green recurrent networks dream hierarchically

arXiv 1803.11138 ↗First posted 29 Mar 2018cs.CLcited by 213

Not on the map. Why?

Abstract (arXiv)

Recurrent neural networks (RNNs) have achieved impressive results in a variety of linguistic processing tasks, suggesting that they can induce non-trivial properties of language. We investigate here to what extent RNNs learn to track abstract hierarchical syntactic structure. We test whether RNNs trained with a generic language modeling objective in four languages (Italian, English, Hebrew, Russian) can predict long-distance number agreement in various constructions. We include in our evaluation nonsensical sentences where RNNs cannot rely on semantic or lexical cues ("The colorless green ideas I ate with the chair sleep furiously"), and, for Italian, we compare model performance to human intuitions. Our language-model-trained RNNs make reliable predictions about long-distance agreement, and do not lag much behind human performance. We thus bring support to the hypothesis that RNNs are not just shallow-pattern extractors, but they also acquire deeper grammatical competence.

Key facts. Cited by 213 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 60; 2022: 41; 2023: 34; 2024: 25; 2025: 20; 2026: 33.

213 papers.

Sort:

By area:

  1. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  4. 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 177
  5. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  6. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  7. 2023The Learnability of In-Context Learningcited by 127
  8. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  9. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  10. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  11. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  12. 2021Language Model Evaluation Beyond Perplexitycited by 78
38 more
  1. 2021Understanding by Understanding Not: Modeling Negation in Language Modelscited by 73
  2. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 71
  3. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 59
  4. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  5. 2022Probing for the Usage of Grammatical Numbercited by 59
  6. 2022Linear Connectivity Reveals Generalization Strategiescited by 56
  7. 2021CLiMP: A Benchmark for Chinese Language Model Evaluationcited by 46
  8. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  9. 2021Structural Persistence in Language Models: Priming as a Window into Abstract Language Representationscited by 44
  10. 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
  11. 2023CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Modelscited by 43
  12. 2023Modeling rapid language learning by distilling Bayesian priors into artificial neural networkscited by 42
  13. 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
  14. 2021Refining Targeted Syntactic Evaluation of Language Modelscited by 41
  15. 2021Word Acquisition in Neural Language Modelscited by 41
  16. 2023Language acquisition: do children and language models follow similar learning stages?cited by 41
  17. 2023Neither hype nor gloom do DNNs justicecited by 40
  18. 2022Prompting Language Models for Linguistic Structurecited by 39
  19. 2022Analyzing Encoded Concepts in Transformer Language Modelscited by 38
  20. 2022On the Explainability of Natural Language Processing Deep Modelscited by 38
  21. 2024Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidencecited by 35
  22. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  23. 2023Do large language models resemble humans in language use?cited by 34
  24. 2021A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humanscited by 32
  25. 2021Language Models Use Monotonicity to Assess NPI Licensingcited by 31
  26. 2022Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguitiescited by 30
  27. 2024Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilitiescited by 30
  28. 2023Dissociating language and thought in large language modelscited by 29
  29. 2021Do Syntactic Probes Probe Syntax? Experiments with Jabberwocky Probingcited by 28
  30. 2021Sorting through the noise: Testing robustness of information processing in pre-trained language modelscited by 27
  31. 2022Neural reality of argument structure constructionscited by 25
  32. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 25
  33. 2021How is BERT surprised? Layerwise detection of linguistic anomaliescited by 24
  34. 2022When classifying grammatical role, BERT doesn't care about word order... except when it matterscited by 24
  35. 2022When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to itcited by 23
  36. 2021Relative Importance in Sentence Processingcited by 22
  37. 2023How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speechcited by 22
  38. 2023A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Constructioncited by 22

50 most cited of 213 All 213 (JSON)