Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints
Not on the map. Why?
Abstract (arXiv)
Neural QCFG is a grammar-based sequence-tosequence (seq2seq) model with strong inductive biases on hierarchical structures. It excels in interpretability and generalization but suffers from expensive inference. In this paper, we study two low-rank variants of Neural QCFG for faster inference with different trade-offs between efficiency and expressiveness. Furthermore, utilizing the symbolic interface provided by the grammar, we introduce two soft constraints over tree hierarchy and source coverage. We experiment with various datasets and find that our models outperform vanilla Neural QCFG in most settings.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 23 of its 45 references that are in the map. First posted on arXiv 5 Jun 2023.
Builds on
23 of 45 references.
References · 23
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2017Convolutional Sequence to Sequence Learningcited by 550
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
- 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
- 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
- 2016Modeling Coverage for Neural Machine Translationcited by 110
- 2016Recurrent Neural Network Grammarscited by 109
- 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
- 2017Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generationcited by 61
- 2018Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networkscited by 51
11 more
- 2017Introduction to Tensor Decompositions and their Applications in Machine Learningcited by 45
- 2020Torch-Struct: Deep Structured Prediction Librarycited by 40
- 2022Transformer Grammars: Augmenting Transformer Language Models with Syntactic Inductive Biases at Scalecited by 40
- 2021StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfercited by 34
- 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
- 2020Scaling Hidden Markov Language Modelscited by 22
- 2021PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbolscited by 22
- 2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
- 2022Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGscited by 10
- 2022Low-Rank Constraints for Fast Inference in Structured Modelscited by 8
- 2022Hierarchical Phrase-based Sequence-to-Sequence Learningcited by 7
Built on by
Citing papers per year: 2023: 1.
1 paper.