mlsanityData to 13 Oct 2026

Unsupervised Recurrent Neural Network Grammars

arXiv 1904.03746 ↗First posted 7 Apr 2019cs.CL stat.MLcited by 47

Not on the map. Why?

Abstract (arXiv)

Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. Supervised RNNGs achieve strong language modeling and parsing performance, but require an annotated corpus of parse trees. In this work, we experiment with unsupervised learning of RNNGs. Since directly marginalizing over the space of latent trees is intractable, we instead apply amortized variational inference. To maximize the evidence lower bound, we develop an inference network parameterized as a neural CRF constituency parser. On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese. On constituency grammar induction, they are competitive with recent neural language models that induce tree structures from words through attention mechanisms.

Key facts. Cited by 47 papers in the map (data through 13 Oct 2026). First posted on arXiv 7 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 14; 2022: 9; 2023: 10; 2024: 8; 2025: 3; 2026: 3.

47 papers.

Sort:

By area:

  1. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  2. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  3. 2021Refining Targeted Syntactic Evaluation of Language Modelscited by 41
  4. 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scalecited by 40
  5. 2021VLGrammar: Grounded Grammar Induction of Vision and Languagecited by 22
  6. 2021PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbolscited by 22
  7. 2021R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modelingcited by 19
  8. 2021Transformers Generalize Linearlycited by 19
  9. 2021Interpreting Deep Learning Models in Natural Language Processing: A Reviewcited by 19
  10. 2023Pushdown Layers: Encoding Recursive Structure in Transformer Language Modelscited by 18
  11. 2021Modeling Hierarchical Structures with Continuous Recursive Neural Networkscited by 15
  12. 2022Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representationcited by 14
35 more
  1. 2024Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scalecited by 9
  2. 2023Activity Grammars for Temporal Action Segmentationcited by 8
  3. 2023Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parserscited by 7
  4. 2024Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Modelscited by 7
  5. 2021Improved Latent Tree Induction with Distant Supervision via Span Constraintscited by 6
  6. 2023Ensemble Distillation for Unsupervised Constituency Parsingcited by 6
  7. 2021The Limitations of Limited Context for Constituency Parsingcited by 5
  8. 2023Augmenting Transformers with Recursively Composed Multi-grained Representationscited by 5
  9. 2021Improving BERT Pretraining with Syntactic Supervisioncited by 4
  10. 2021Learning Syntax from Naturally-Occurring Bracketingscited by 4
  11. 2023A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classificationcited by 4
  12. 2024Graph Parsing Networkscited by 4
  13. 2021Co-training an Unsupervised Constituency Parser with Weak Supervisioncited by 3
  14. 2022Exploiting Inductive Bias in Transformers for Unsupervised Disentanglement of Syntax and Semantics with VAEscited by 3
  15. 2022Learning a Grammar Inducer from Massive Uncurated Instructional Videoscited by 3
  16. 2023Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure Buildingcited by 3
  17. 2021RL-GRIT: Reinforcement Learning for Grammar Inferencecited by 2
  18. 2021Scaling Structured Inference with Randomizationcited by 2
  19. 2022Learning with Latent Structures in Natural Language Processing: A Surveycited by 2
  20. 2023Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentencescited by 2
  21. 2022Dependency-based Mixture Language Modelscited by 1
  22. 2022Revisiting the Practical Effectiveness of Constituency Parse Extraction from Pre-trained Language Modelscited by 1
  23. 2024Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Inductioncited by 1
  24. 2024Improving Unsupervised Constituency Parsing via Maximizing Semantic Informationcited by 1
  25. 2025A Systematic Study of Compositional Syntactic Transformer Language Modelscited by 1
  26. 2022Syntactic Inductive Biases for Deep Learning Methodscited by 0
  27. 2022Unsupervised and Few-shot Parsing from Pretrained Language Modelscited by 0
  28. 2024Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsingcited by 0
  29. 2024Grammar Induction from Visual, Speech and Textcited by 0
  30. 2024On Eliciting Syntax from Language Models via Hashingcited by 0
  31. 2025Understanding Syntactic Generalization in Structure-inducing Language Modelscited by 0
  32. 2025Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Inductioncited by 0
  33. 2026GiLT: Augmenting Transformer Language Models with Dependency Graphscited by 0
  34. 2026Weave of Formal Thoughtcited by 0
  35. 2026Holographic Neural PCFG for Unsupervised Parsingcited by 0