mlsanityData to 7 Oct 2026

Multi-Granularity Optimization for Non-Autoregressive Translation

arXiv 2210.11017 ↗First posted 20 Oct 2022cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite low latency, non-autoregressive machine translation (NAT) suffers severe performance deterioration due to the naive independence assumption. This assumption is further strengthened by cross-entropy loss, which encourages a strict match between the hypothesis and the reference token by token. To alleviate this issue, we propose multi-granularity optimization for NAT, which collects model behaviors on translation segments of various granularities and integrates feedback for backpropagation. Experiments on four WMT benchmarks show that the proposed method significantly outperforms the baseline models trained with cross-entropy loss, and achieves the best performance on WMT'16 En-Ro and highly competitive results on WMT'14 En-De for fully non-autoregressive translation.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

35 of 49 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  5. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  6. 2017Non-Autoregressive Neural Machine Translationcited by 457
  7. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  8. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  9. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  10. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 103
  11. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  12. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
23 more
  1. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  2. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  3. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  4. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  5. 2019Fast Structured Decoding for Sequence Modelscited by 59
  6. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  7. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  8. 2020Understanding and Improving Lexical Choice in Non-Autoregressive Translationcited by 55
  9. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  10. 2019Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translationcited by 40
  11. 2020On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classificationcited by 40
  12. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  13. 2019Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translationcited by 39
  14. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  15. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 32
  16. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
  17. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  18. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  19. 2020An EM Approach to Non-autoregressive Conditional Sequence Generationcited by 21
  20. 2022textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generationcited by 20
  21. 2021Don't Take It Literally: An Edit-Invariant Sequence Loss for Text Generationcited by 17
  22. 2019Retrieving Sequential Information for Non-Autoregressive Neural Machine Translationcited by 15
  23. 2018Neural Machine Translation with Adequacy-Oriented Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 2.

6 papers.

  1. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
  2. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  3. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  4. 2024Improving Non-autoregressive Machine Translation with Error Exposure and Consistency Regularizationcited by 0
  5. 2024Non-autoregressive Personalized Bundle Generationcited by 0
  6. 2025Diffusion Is Your Friend in Show, Suggest and Tellcited by 0