mlsanityData to 14 Oct 2026

Non-Autoregressive Neural Machine Translation

arXiv 1711.02281 ↗First posted 7 Nov 2017cs.CL cs.LGcited by 463

Not on the map. Why?

Cited by 69 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing approaches to neural machine translation condition each output word on previously generated outputs. We introduce a model that avoids this autoregressive property and produces its outputs in parallel, allowing an order of magnitude lower latency during inference. Through knowledge distillation, the use of input token fertilities as a latent variable, and policy gradient fine-tuning, we achieve this at a cost of as little as 2.0 BLEU points relative to the autoregressive Transformer network used as a teacher. We demonstrate substantial cumulative improvements associated with each of the three aspects of our training strategy, and validate our approach on IWSLT 2016 English-German and two WMT language pairs. By sampling fertilities in parallel at inference time, our non-autoregressive model achieves near-state-of-the-art performance of 29.8 BLEU on WMT 2016 English-Romanian.

Key facts. Cited by 463 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 91; 2022: 100; 2023: 85; 2024: 67; 2025: 46; 2026: 73.

463 papers.

Sort:

By area:

  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  3. 2021Going deeper with Image Transformerscited by 764
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  5. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  6. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  7. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  8. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
  9. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 221
  10. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  11. 2022Continuous diffusion for categorical datacited by 214
  12. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
38 more
  1. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  2. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  3. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 176
  4. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 171
  5. 2023Speculative Decoding with Big Little Decodercited by 164
  6. 2024The pitfalls of next-token predictioncited by 160
  7. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  8. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  9. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 149
  10. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  11. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  12. 2021A Unified Generative Framework for Various NER Subtaskscited by 141
  13. 2021End-to-End Human Object Interaction Detection with HOI Transformercited by 135
  14. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  15. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124
  16. 2022Pretrained Language Models for Text Generation: A Surveycited by 122
  17. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
  18. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 114
  19. 2023SoundStorm: Efficient Parallel Audio Generationcited by 114
  20. 2023f-Divergence Minimization for Sequence-Level Knowledge Distillationcited by 114
  21. 2024Image and Video Tokenization with Binary Spherical Quantizationcited by 112
  22. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  23. 2023Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systemscited by 102
  24. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrappingcited by 98
  25. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 96
  26. 2021Finetuning Pretrained Transformers into RNNscited by 96
  27. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  28. 2024Energy-Based Diffusion Language Models for Text Generationcited by 89
  29. 2023Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipelinecited by 86
  30. 2022PiFold: Toward effective and efficient protein inverse foldingcited by 83
  31. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  32. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  33. 2024GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decodingcited by 75
  34. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 74
  35. 2021Aligning Latent and Image Spaces to Connect the Unconnectablecited by 70
  36. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  37. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  38. 2023Structure-informed Language Models Are Protein Designerscited by 68

50 most cited of 463 All 463 (JSON)