mlsanityData to 14 Oct 2026

Enhancing the Transformer Decoder with Transition-based Syntax

arXiv 2101.12640 ↗First posted 29 Jan 2021cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Notwithstanding recent advances, syntactic generalization remains a challenge for text decoders. While some studies showed gains from incorporating source-side symbolic syntactic and semantic structure into text generation Transformers, very little work addressed the decoding of such structure. We propose a general approach for tree decoding using a transition-based approach. Examining the challenging test case of incorporating Universal Dependencies syntax into machine translation, we present substantial improvements on test sets that focus on syntactic generalization, while presenting improved or comparable performance on standard MT benchmarks. Further qualitative analysis addresses cases where syntactic generalization in the vanilla Transformer decoder is inadequate and demonstrates the advantages afforded by integrating syntactic information.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 49 of its 84 references that are in the map. First posted on arXiv 29 Jan 2021.

Builds on

49 of 84 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  6. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 306
  7. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
  8. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 165
  9. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  10. 2016Modeling Coverage for Neural Machine Translationcited by 110
  11. 2016Recurrent Neural Network Grammarscited by 109
  12. 2020Syntactic Structure from Deep Learningcited by 99
37 more
  1. 2018Training Tips for the Transformer Modelcited by 85
  2. 2019Beyond BLEU: Training Neural Machine Translation with Semantic Similaritycited by 84
  3. 2019On the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 82
  4. 2018Linguistically-Informed Self-Attention for Semantic Role Labelingcited by 71
  5. 2017Graph Convolutional Encoders for Syntax-aware Neural Machine Translationcited by 70
  6. 2019Tree Transformer: Integrating Tree Structures into Self-Attentioncited by 68
  7. 2020Do Syntax Trees Help Pre-trained Transformers Extract Information?cited by 48
  8. 2020Measuring Systematic Generalization in Neural Proof Generation with Transformerscited by 43
  9. 2019Semantic Neural Machine Translation using AMRcited by 37
  10. 2019Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languagescited by 37
  11. 2018Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networkscited by 30
  12. 2019A Call for Prudent Choice of Subword Merge Operations in Neural Machine Translationcited by 30
  13. 2020Tree-structured Attention with Hierarchical Accumulationcited by 30
  14. 2016Tree-to-Sequence Attentional Neural Machine Translationcited by 28
  15. 2019Synchronous Bidirectional Neural Machine Translationcited by 27
  16. 2019Enhancing Machine Translation with Dependency-Aware Self-Attentioncited by 24
  17. 2018Asynchronous Bidirectional Decoding for Neural Machine Translationcited by 21
  18. 2019Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Datacited by 20
  19. 2017Towards String-to-Tree Neural Machine Translationcited by 19
  20. 2018A Tree-based Decoder for Neural Machine Translationcited by 18
  21. 2017Learning to Parse and Translate Improves Neural Machine Translationcited by 16
  22. 2017Nematus: a Toolkit for Neural Machine Translationcited by 16
  23. 2018On Tree-Based Neural Sentence Modelingcited by 16
  24. 2020Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answeringcited by 16
  25. 2017Predicting Target Language CCG Supertags Improves Neural Machine Translationcited by 15
  26. 2019Automatically Extracting Challenge Sets for Non local Phenomena in Neural Machine Translationcited by 15
  27. 2017Modeling Source Syntax for Neural Machine Translationcited by 12
  28. 2017Syntax-Directed Attention for Neural Machine Translationcited by 10
  29. 2018Multitask Parsing Across Semantic Representationscited by 10
  30. 2019Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representationscited by 10
  31. 2019Multi-Granularity Self-Attention for Neural Machine Translationcited by 10
  32. 2019Handling Syntactic Divergence in Low-resource Machine Translationcited by 8
  33. 2020Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Modelscited by 8
  34. 2018Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsingcited by 5
  35. 2019Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Neededcited by 2
  36. 2017Non-Projective Dependency Parsing with Non-Local Transitionscited by 1
  37. 2020Fine-grained linguistic evaluation for state-of-the-art Machine Translationcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022PreQuEL: Quality Estimation of Machine Translation Outputs in Advancecited by 9