mlsanityData to 7 Oct 2026

Syntactic Structure from Deep Learning

arXiv 2004.10827 ↗First posted 22 Apr 2020cs.CLcited by 99

Not on the map. Why?

Abstract (arXiv)

Modern deep neural networks achieve impressive performance in engineering applications that require extensive linguistic skills, such as machine translation. This success has sparked interest in probing whether these models are inducing human-like grammatical knowledge from the raw data they are exposed to, and, consequently, whether they can shed new light on long-standing debates concerning the innate structure necessary for language acquisition. In this article, we survey representative studies of the syntactic abilities of deep networks, and discuss the broader implications that this work has for theoretical linguistics.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 17; 2023: 18; 2024: 25; 2025: 14; 2026: 13.

99 papers.

Sort:

By area:

  1. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 243
  2. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  3. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  4. 2022Do Large Language Models know what humans know?cited by 60
  5. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  6. 2021On the proper role of linguistically-oriented deep net analysis in linguistic theorizingcited by 43
  7. 2021Disentangling Syntax and Semantics in the Brain with Deep Networkscited by 39
  8. 2023Do large language models resemble humans in language use?cited by 34
  9. 2023Dissociating language and thought in large language modelscited by 27
  10. 2021Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attentioncited by 26
  11. 2022RuCoLA: Russian Corpus of Linguistic Acceptabilitycited by 26
  12. 2024Large Language Models are Biased Because They Are Large Language Modelscited by 26
38 more
  1. 2022Neural reality of argument structure constructionscited by 25
  2. 2022Probing for Incremental Parse States in Autoregressive Language Modelscited by 24
  3. 2022Unit Testing for Concepts in Neural Networkscited by 23
  4. 2024RuBLiMP: Russian Benchmark of Linguistic Minimal Pairscited by 23
  5. 2023A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Constructioncited by 22
  6. 2024Do Language Models Exhibit Human-like Structural Priming Effects?cited by 21
  7. 2022Acceptability Judgements via Examining the Topology of Attention Mapscited by 19
  8. 2022Systematic Generalization and Emergent Structures in Transformers Trained on Structured Taskscited by 19
  9. 2023Machine Psychologycited by 16
  10. 2021The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty Through Generationcited by 15
  11. 2023Evaluating Neural Language Models as Cognitive Models of Language Acquisitioncited by 15
  12. 2024Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Due to Impenetrable Semantic Referencecited by 15
  13. 2024A Psycholinguistic Evaluation of Language Models' Sensitivity to Argument Rolescited by 14
  14. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  15. 2024Generalizations across filler-gap dependencies in neural language modelscited by 12
  16. 2021Compositional Processing Emerges in Neural Networks Solving Math Problemscited by 11
  17. 2022Probing BERT's priors with serial reproduction chainscited by 11
  18. 2022Modeling structure-building in the brain with CCG parsing and large language modelscited by 11
  19. 2024Scope Ambiguities in Large Language Modelscited by 11
  20. 2022Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?cited by 10
  21. 2023Why Linguistics Will Thrive in the 21st Century: A Reply to Piantadosi (2023)cited by 9
  22. 2023Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanitiescited by 9
  23. 2022Characterizing Verbatim Short-Term Memory in Neural Language Modelscited by 8
  24. 2021Modeling morphology with Linear Discriminative Learning: considerations and design choicescited by 7
  25. 2022Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreementcited by 7
  26. 2024From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistencycited by 7
  27. 2024From Frege to chatGPT: Compositionality in language, cognition, and deep neural networkscited by 7
  28. 2024Generative linguistics contribution to artificial intelligence: Where this contribution lies?cited by 7
  29. 2024What fifty-one years of Linguistics and Artificial Intelligence research tell us about their correlation: A scientometric analysiscited by 7
  30. 2021On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representationscited by 6
  31. 2021Schr"odinger's Tree -- On Syntax and Neural Language Modelscited by 6
  32. 2023Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalizationcited by 6
  33. 2023How Well Do Text Embedding Models Understand Syntax?cited by 6
  34. 2024Exploring syntactic information in sentence embeddings through multilingual subject-verb agreementcited by 6
  35. 2024Linguistics Theory Meets LLM: Code-Switched Text Generation via Equivalence Constrained Large Language Modelscited by 6
  36. 2025Whither symbols in the era of advanced neural networks?cited by 6
  37. 2021A phonetic model of non-native spoken word processingcited by 5
  38. 2022Grammatical cues to subjecthood are redundant in a majority of simple clauses across languagescited by 5

50 most cited of 99 All 99 (JSON)