mlsanityData to 14 Oct 2026

Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation

arXiv 2101.08942 ↗First posted 22 Jan 2021cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

The non-autoregressive models have boosted the efficiency of neural machine translation through parallelized decoding at the cost of effectiveness when comparing with the autoregressive counterparts. In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance. However, these structures are rarely considered in the existing non-autoregressive models. Inspired by this intuition, we propose to incorporate the explicit syntactic and semantic structures of languages into a non-autoregressive Transformer, for the task of neural machine translation. Moreover, we also consider the intermediate latent alignment within target sentences to better learn the long-term token dependencies. Experimental results on two real-world datasets (i.e., WMT14 En-De and WMT16 En-Ro) show that our model achieves a significantly faster speed, as well as keeps the translation quality when compared with several state-of-the-art non-autoregressive models.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 22 Jan 2021.

Builds on

27 of 33 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  6. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  7. 2017Convolutional Sequence to Sequence Learningcited by 552
  8. 2017Non-Autoregressive Neural Machine Translationcited by 463
  9. 2018Scaling Neural Machine Translationcited by 204
  10. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 195
  11. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 159
  12. 2019Insertion Transformer: Flexible Sequence Generation via Insertion Operationscited by 125
15 more
  1. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 104
  2. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  3. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  4. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  5. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  6. 2019Fast Structured Decoding for Sequence Modelscited by 59
  7. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  8. 2019Non-Autoregressive Machine Translation with Auxiliary Regularizationcited by 52
  9. 2019Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translationcited by 40
  10. 2018Non-Autoregressive Neural Machine Translation with Enhanced Decoder Inputcited by 39
  11. 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
  12. 2019Hint-Based Training for Non-Autoregressive Machine Translationcited by 28
  13. 2019KERMIT: Generative Insertion-Based Modeling for Sequencescited by 22
  14. 2020Improving Non-autoregressive Neural Machine Translation with Monolingual Datacited by 10
  15. 2020LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attentioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 3; 2024: 1; 2025: 1; 2026: 1.

13 papers.

Sort:

By area:

  1. 2021Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformercited by 69
  2. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  3. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  4. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  5. 2021Modeling Sequences as Distributions with Uncertainty for Sequential Recommendationcited by 22
  6. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  7. 2021HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarizationcited by 13
  8. 2023Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tablescited by 6
  9. 2023Non-autoregressive Machine Translation with Probabilistic Context-free Grammarcited by 4
  10. 2022Rephrasing the Reference for Non-Autoregressive Machine Translationcited by 3
  11. 2023RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformercited by 1
  12. 2024NARRepair: Non-Autoregressive Code Generation Model for Automatic Program Repaircited by 0
1 more
  1. 2026Towards Faster Language Model Inference Using Mixture-of-Experts Flow Matchingcited by 0