mlsanityData to 7 Oct 2026

Imitation Learning for Non-Autoregressive Neural Machine Translation

arXiv 1906.02041 ↗First posted 5 Jun 2019cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive translation models (NAT) have achieved impressive inference speedup. A potential issue of the existing NAT algorithms, however, is that the decoding is conducted in parallel, without directly considering previous context. In this paper, we propose an imitation learning framework for non-autoregressive machine translation, which still enjoys the fast translation speed but gives comparable translation performance compared to its auto-regressive counterpart. We conduct experiments on the IWSLT16, WMT14 and WMT16 datasets. Our proposed model achieves a significant speedup over the autoregressive models, while keeping the translation quality comparable to the autoregressive models. By sampling sentence length in parallel at inference time, we achieve the performance of 31.85 BLEU on WMT16 RorightarrowEn and 30.68 BLEU on IWSLT16 EnrightarrowDe.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 13; 2022: 11; 2023: 8; 2024: 2; 2025: 1.

36 papers.

Sort:

By area:

  1. 2023Speculative Decoding with Big Little Decodercited by 163
  2. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  3. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 68
  4. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  5. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  6. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  7. 2021Non-Autoregressive Text Generation with Pre-trained Language Modelscited by 36
  8. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  9. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  10. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  11. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 27
  12. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
24 more
  1. 2024Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decodingcited by 25
  2. 2022textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generationcited by 20
  3. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  4. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  5. 2021Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translationcited by 13
  6. 2021Semi-Autoregressive Image Captioningcited by 10
  7. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  8. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  9. 2021Improving Non-autoregressive Generation with Mixup Trainingcited by 7
  10. 2022Uncertainty-Aware Image Captioningcited by 6
  11. 2021MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translationcited by 5
  12. 2022Non-Autoregressive Neural Machine Translation: A Call for Claritycited by 5
  13. 2022DePA: Improving Non-autoregressive Machine Translation with Dependency-Aware Decodercited by 3
  14. 2022Diffusion Glancing Transformer for Parallel Sequence to Sequence Learningcited by 3
  15. 2023Extending Multilingual Machine Translation through Imitation Learningcited by 3
  16. 2022PARAGEN : A Parallel Generation Toolkitcited by 2
  17. 2023TreePiece: Faster Semantic Parsing via Tree Tokenizationcited by 2
  18. 2023Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translationcited by 2
  19. 2021Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translationcited by 1
  20. 2023RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformercited by 1
  21. 2021Learning Energy-Based Approximate Inference Networks for Structured Applications in NLPcited by 0
  22. 2023Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTCcited by 0
  23. 2023Attentive Multi-Layer Perceptron for Non-autoregressive Generationcited by 0
  24. 2023Quick Back-Translation for Unsupervised Machine Translationcited by 0