mlsanityData to 14 Oct 2026

Hierarchical Phrase-based Sequence-to-Sequence Learning

arXiv 2211.07906 ↗First posted 15 Nov 2022cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

We describe a neural transducer that maintains the flexibility of standard sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases as a source of inductive bias during training and as explicit constraints during inference. Our approach trains two models: a discriminative parser based on a bracketing transduction grammar whose derivation tree hierarchically aligns source and target phrases, and a neural seq2seq model that learns to translate the aligned phrases one-by-one. We use the same seq2seq model to translate at all phrase scales, which results in two inference modes: one mode in which the parser is discarded and only the seq2seq component is used at the sequence-level, and another in which the parser is combined with the seq2seq model. Decoding in the latter mode is done with the cube-pruned CKY algorithm, which is more involved but can make use of new translation rules during inference. We formalize our model as a source-conditioned synchronous grammar and develop an efficient variational inference algorithm for training. When applied on top of both randomly initialized and pretrained seq2seq models, we find that both inference modes performs well compared to baselines on small scale machine translation benchmarks.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 39 of its 77 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

39 of 77 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  7. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  8. 2016Self-critical Sequence Training for Image Captioningcited by 547
  9. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  10. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 263
  11. 2018Constituency Parsing with a Self-Attentive Encodercited by 199
  12. 2019Good-Enough Compositional Data Augmentationcited by 158
27 more
  1. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  2. 2016Data Recombination for Neural Semantic Parsingcited by 111
  3. 2019Compound Probabilistic Context-Free Grammars for Grammar Inductioncited by 76
  4. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  5. 2019Revisiting Low-Resource Neural Machine Translation: A Case Studycited by 61
  6. 2017A Minimal Span-Based Neural Constituency Parsercited by 59
  7. 2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
  8. 2020Torch-Struct: Deep Structured Prediction Librarycited by 40
  9. 2020Visually Grounded Compound PCFGscited by 34
  10. 2021On Compositional Generalization of Neural Machine Translationcited by 34
  11. 2016Neural Machine Translation with Supervised Attentioncited by 31
  12. 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
  13. 2022Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translationcited by 28
  14. 2021Substructure Substitution: Structured Data Augmentation for NLPcited by 25
  15. 2018Rao-Blackwellized Stochastic Gradients for Discrete Distributionscited by 24
  16. 2021PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbolscited by 22
  17. 2020ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalizationcited by 20
  18. 2020The Return of Lexical Dependencies: Neural Lexicalized PCFGscited by 19
  19. 2021Neural Bi-Lexicalized PCFG Inductioncited by 18
  20. 2021Structured Reordering for Modeling Latent Alignments in Sequence Transductioncited by 17
  21. 2016Incorporating Structural Alignment Biases into an Attentional Neural Translation Modelcited by 16
  22. 2022SUBS: Subtree Substitution for Compositional Semantic Parsingcited by 16
  23. 2017Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Methodcited by 12
  24. 2016Supervised Attentions for Neural Machine Translationcited by 8
  25. 2017Dependency Grammar Induction with Neural Lexicalization and Big Training Datacited by 7
  26. 2014Sentence Compression as Tree Transductioncited by 5
  27. 2022Compositional Generalization Requires Compositional Parserscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 6.

7 papers.

  1. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  2. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  3. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  4. 2022Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammarscited by 7
  5. 2023Non-autoregressive Machine Translation with Probabilistic Context-free Grammarcited by 4
  6. 2023Simple Hardware-Efficient PCFGs with Independent Left and Right Productionscited by 3
  7. 2023Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraintscited by 1