mlsanityData to 7 Oct 2026

Non-Autoregressive Machine Translation with Latent Alignments

arXiv 2004.07437 ↗First posted 16 Apr 2020cs.CL cs.LGcited by 96

Not on the map. Why?

Abstract (arXiv)

This paper presents two strong methods, CTC and Imputer, for non-autoregressive machine translation that model latent alignments with dynamic programming. We revisit CTC for machine translation and demonstrate that a simple CTC model can achieve state-of-the-art for single-step non-autoregressive machine translation, contrary to what prior work indicates. In addition, we adapt the Imputer model for non-autoregressive machine translation and demonstrate that Imputer with just 4 generation steps can match the performance of an autoregressive Transformer baseline. Our latent alignment models are simpler than many existing non-autoregressive translation baselines; for example, we do not require target length prediction or re-scoring with an autoregressive model. On the competitive WMT'14 EnrightarrowDe task, our CTC model achieves 25.7 BLEU with a single generation step, while Imputer achieves 27.5 BLEU with 2 generation steps, and 28.0 BLEU with 4 generation steps. This compares favourably to the autoregressive Transformer baseline at 27.8 BLEU.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 35; 2023: 22; 2024: 11; 2025: 4; 2026: 3.

96 papers.

Sort:

By area:

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  3. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  4. 2022Continuous diffusion for categorical datacited by 202
  5. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 169
  6. 2021Learning to Efficiently Sample from Diffusion Probabilistic Modelscited by 129
  7. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 123
  8. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  9. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  10. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  11. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  12. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
38 more
  1. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  2. 2022DiffusER: Discrete Diffusion via Edit-based Reconstructioncited by 41
  3. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  4. 2023Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracycited by 37
  5. 2021A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generationcited by 27
  6. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  7. 2022Revisiting End-to-End Speech-to-Text Translation From Scratchcited by 26
  8. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  9. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  10. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  11. 2021Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translationcited by 23
  12. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noisecited by 21
  13. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
  14. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  15. 2024StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learningcited by 20
  16. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  17. 2022Lossless Acceleration for Seq2seq Generation with Aggressive Decodingcited by 18
  18. 2021Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialogcited by 17
  19. 2022Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translationcited by 16
  20. 2021Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translationcited by 13
  21. 2022A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarizationcited by 13
  22. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  23. 2023Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translationcited by 13
  24. 2022Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarizationcited by 12
  25. 2022Non-Autoregressive Machine Translation: It's Not as Fast as it Seemscited by 11
  26. 2023Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translationcited by 11
  27. 2022ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generationcited by 10
  28. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  29. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  30. 2022LegoNN: Building Modular Encoder-Decoder Modelscited by 9
  31. 2022SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognitioncited by 9
  32. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  33. 2023DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translationcited by 9
  34. 2021An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognitioncited by 8
  35. 2021Self-Distillation Mixup Training for Non-autoregressive Neural Machine Translationcited by 8
  36. 2022ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 8
  37. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8
  38. 2022Multi-Granularity Optimization for Non-Autoregressive Translationcited by 6

50 most cited of 96 All 96 (JSON)