mlsanityData to 14 Oct 2026

PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols

arXiv 2104.13727 ↗First posted 28 Apr 2021cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Probabilistic context-free grammars (PCFGs) with neural parameterization have been shown to be effective in unsupervised phrase-structure grammar induction. However, due to the cubic computational complexity of PCFG representation and parsing, previous approaches cannot scale up to a relatively large number of (nonterminal and preterminal) symbols. In this work, we present a new parameterization form of PCFGs based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols. We further use neural parameterization for the new form to improve unsupervised parsing performance. We evaluate our model across ten languages and empirically demonstrate the effectiveness of using more symbols. Our code: https://github.com/sustcsonglin/TN-PCFG

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 23 of its 47 references that are in the map. First posted on arXiv 28 Apr 2021.

Builds on

23 of 47 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Variational Inference with Normalizing Flowscited by 2,170
  3. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 659
  4. 2018Efficient Attention: Attention with Linear Complexitiescited by 368
  5. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 284
  6. 2018Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networkscited by 101
  7. 2019Compound Probabilistic Context-Free Grammars for Grammar Inductioncited by 76
  8. 2019Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoderscited by 60
  9. 2017Neural Language Modeling by Jointly Learning Syntax and Lexiconcited by 57
  10. 2020Fast and Accurate Neural CRF Constituency Parsingcited by 50
  11. 2019Unsupervised Recurrent Neural Network Grammarscited by 47
  12. 2020Torch-Struct: Deep Structured Prediction Librarycited by 40
11 more
  1. 2020Visually Grounded Compound PCFGscited by 34
  2. 2020Unsupervised Parsing via Constituency Testscited by 23
  3. 2020Scaling Hidden Markov Language Modelscited by 22
  4. 2016Unsupervised Neural Hidden Markov Modelscited by 19
  5. 2020The Return of Lexical Dependencies: Neural Lexicalized PCFGscited by 19
  6. 2020On the Role of Supervision in Unsupervised Constituency Parsingcited by 18
  7. 2018Unsupervised Learning of Syntactic Structure with Invertible Neural Projectionscited by 11
  8. 2018Benefits of over-parameterization with EMcited by 10
  9. 2020Second-Order Unsupervised Neural Dependency Parsingcited by 10
  10. 2021An Empirical Study of Compound PCFGscited by 6
  11. 2018Gaussian Mixture Latent Vector Grammarscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 6; 2024: 5; 2025: 1; 2026: 1.

22 papers.

Sort:

  1. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  2. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  3. 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
  4. 2021Neural Bi-Lexicalized PCFG Inductioncited by 18
  5. 2022Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGscited by 10
  6. 2022Low-Rank Constraints for Fast Inference in Structured Modelscited by 8
  7. 2022Hierarchical Phrase-based Sequence-to-Sequence Learningcited by 7
  8. 2022Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammarscited by 7
  9. 2023Contextual Distortion Reveals Constituency: Masked Language Models are Implicit Parserscited by 7
  10. 2022Learning a Grammar Inducer from Massive Uncurated Instructional Videoscited by 3
  11. 2023Simple Hardware-Efficient PCFGs with Independent Left and Right Productionscited by 3
  12. 2021Scaling Structured Inference with Randomizationcited by 2
10 more
  1. 2024Unsupervised Parsing by Searching for Frequent Word Sequences among Sentences with Equivalent Predicate-Argument Structurescited by 2
  2. 2022Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inferencecited by 1
  3. 2023Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraintscited by 1
  4. 2023Greedy Grammar Induction with Indirect Negative Evidencecited by 1
  5. 2024Structural Optimization Ambiguity and Simplicity Bias in Unsupervised Neural Grammar Inductioncited by 1
  6. 2024Improving Unsupervised Constituency Parsing via Maximizing Semantic Informationcited by 1
  7. 2024Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsingcited by 0
  8. 2024On Eliciting Syntax from Language Models via Hashingcited by 0
  9. 2025Probability Distribution Collapse: A Critical Bottleneck to Compact Unsupervised Neural Grammar Inductioncited by 0
  10. 2026Holographic Neural PCFG for Unsupervised Parsingcited by 0