mlsanityData to 7 Oct 2026

Non-Autoregressive Neural Machine Translation: A Call for Clarity

arXiv 2205.10577 ↗First posted 21 May 2022cs.CL cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive approaches aim to improve the inference speed of translation models by only requiring a single forward pass to generate the output sequence instead of iteratively producing each predicted token. Consequently, their translation quality still tends to be inferior to their autoregressive counterparts due to several issues involving output token interdependence. In this work, we take a step back and revisit several techniques that have been proposed for improving non-autoregressive translation models and compare their combined translation quality and speed implications under third-party testing environments. We provide novel insights for establishing strong baselines using length prediction or CTC-based architecture variants and contribute standardized BLEU, chrF++, and TER scores using sacreBLEU on four translation tasks, which crucially have been missing as inconsistencies in the use of tokenized BLEU lead to deviations of up to 1.7 BLEU points. Our open-sourced code is integrated into fairseq for reproducibility.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 45 of its 72 references that are in the map. First posted on arXiv 21 May 2022.

Builds on

45 of 72 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  9. 2016Sequence-Level Knowledge Distillationcited by 761
  10. 2017Non-Autoregressive Neural Machine Translationcited by 457
  11. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  12. 2016Using the Output Embedding to Improve Language Modelscited by 266
33 more
  1. 2018Marian: Fast Neural Machine Translation in C++cited by 202
  2. 2019Levenshtein Transformercited by 197
  3. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  4. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  5. 2020Descending through a Crowded Valley - Benchmarking Deep Learning Optimizerscited by 118
  6. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 104
  7. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  8. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  9. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  10. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  11. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  12. 2021Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Paperscited by 70
  13. 2020Imputer: Sequence Modelling via Imputation and Dynamic Programmingcited by 65
  14. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  15. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  16. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  17. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  18. 2019Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translationcited by 39
  19. 2019Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Informationcited by 37
  20. 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
  21. 2021Non-Autoregressive Text Generation with Pre-trained Language Modelscited by 36
  22. 2019Hint-Based Training for Non-Autoregressive Machine Translationcited by 28
  23. 2020Context-Aware Cross-Attention for Non-Autoregressive Translationcited by 26
  24. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
  25. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  26. 2018Accelerating Neural Transformer via an Average Attention Networkcited by 22
  27. 2018Marian: Cost-effective High-Quality Neural Machine Translation in C++cited by 22
  28. 2020An EM Approach to Non-autoregressive Conditional Sequence Generationcited by 21
  29. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  30. 2020Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translationcited by 16
  31. 2021How Does Distilled Data Complexity Impact the Quality and Confidence of Non-Autoregressive Machine Translation?cited by 12
  32. 2022Non-Autoregressive Machine Translation: It's Not as Fast as it Seemscited by 11
  33. 2021Can Multilinguality benefit Non-autoregressive Machine Translation?cited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 1; 2025: 2.

5 papers.

  1. 2023One Wide Feedforward is All You Needcited by 15
  2. 2025PIER: A Novel Metric for Evaluating What Matters in Code-Switchingcited by 11
  3. 2022Integrating Translation Memories into Non-Autoregressive Machine Translationcited by 5
  4. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  5. 2025Confidence-Modulated Speculative Decoding for Large Language Modelscited by 1