mlsanityData to 7 Oct 2026

Directed Acyclic Transformer for Non-Autoregressive Machine Translation

arXiv 2205.07459 ↗First posted 16 May 2022cs.CLcited by 44

Not on the map. Why?

Abstract (arXiv)

Non-autoregressive Transformers (NATs) significantly reduce the decoding latency by generating all tokens in parallel. However, such independent predictions prevent NATs from capturing the dependencies between the tokens for generating multiple possible translations. In this paper, we propose Directed Acyclic Transfomer (DA-Transformer), which represents the hidden states in a Directed Acyclic Graph (DAG), where each path of the DAG corresponds to a specific translation. The whole DAG simultaneously captures multiple translations and facilitates fast predictions in a non-autoregressive fashion. Experiments on the raw training data of WMT benchmark show that DA-Transformer substantially outperforms previous NATs by about 3 BLEU on average, which is the first NAT model that achieves competitive results with autoregressive Transformers without relying on knowledge distillation.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

34 of 56 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  6. 2016Sequence-Level Knowledge Distillationcited by 761
  7. 2017Non-Autoregressive Neural Machine Translationcited by 457
  8. 2019Levenshtein Transformercited by 197
  9. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  10. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  11. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 104
  12. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 103
22 more
  1. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  2. 2020Non-Autoregressive Machine Translation with Latent Alignmentscited by 96
  3. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  4. 2019Mixture Models for Diverse Machine Translation: Tricks of the Tradecited by 77
  5. 2018End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classificationcited by 72
  6. 2020Aligned Cross Entropy for Non-Autoregressive Machine Translationcited by 65
  7. 2019Fast Structured Decoding for Sequence Modelscited by 59
  8. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  9. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  10. 2020Understanding and Improving Lexical Choice in Non-Autoregressive Translationcited by 55
  11. 2019Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translationcited by 40
  12. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  13. 2018Non-Autoregressive Neural Machine Translation with Enhanced Decoder Inputcited by 39
  14. 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
  15. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  16. 2020Semi-Autoregressive Training Improves Mask-Predict Decodingcited by 33
  17. 2014First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNscited by 27
  18. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
  19. 2020An EM Approach to Non-autoregressive Conditional Sequence Generationcited by 21
  20. 2019Non-autoregressive Transformer by Position Learningcited by 17
  21. 2021The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21cited by 14
  22. 2020Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 17; 2024: 9; 2025: 7; 2026: 4.

44 papers.

Sort:

By area:

  1. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  2. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  3. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  4. 2022Empowering Diffusion Models on the Embedding Space for Text Generationcited by 40
  5. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  6. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  7. 2022Chaining Simultaneous Thoughts for Numerical Reasoningcited by 14
  8. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  9. 2024BiTA: Bi-Directional Tuning for Lossless Acceleration in Large Language Modelscited by 13
  10. 2023Non-autoregressive Streaming Transformer for Simultaneous Translationcited by 12
  11. 2023Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translationcited by 11
  12. 2024A Theory for Length Generalization in Learning to Reasoncited by 11
32 more
  1. 2024A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translationcited by 10
  2. 2023DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translationcited by 9
  3. 2023Simultaneous Machine Translation with Tailored Referencecited by 9
  4. 2025Continuous Autoregressive Language Modelscited by 9
  5. 2023Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generationcited by 8
  6. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8
  7. 2023Non-Autoregressive Math Word Problem Solver with Unified Tree Structurecited by 7
  8. 2024Beyond Next Token Prediction: Patch-Level Training for Large Language Modelscited by 7
  9. 2023AMOM: Adaptive Masking over Masking for Conditional Masked Language Modelcited by 5
  10. 2023Non-autoregressive Machine Translation with Probabilistic Context-free Grammarcited by 4
  11. 2025Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencingcited by 4
  12. 2022Rephrasing the Reference for Non-Autoregressive Machine Translationcited by 3
  13. 2022Diffusion Glancing Transformer for Parallel Sequence to Sequence Learningcited by 3
  14. 2023Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translationcited by 2
  15. 2025ExLM: Rethinking the Impact of [MASK] Tokens in Masked Language Modelscited by 2
  16. 2026DOS: Dependency-Oriented Sampler for Masked Diffusion Language Modelscited by 2
  17. 2023Shared Latent Space by Both Languages in Non-Autoregressive Neural Machine Translationcited by 1
  18. 2023Non-Autoregressive Document-Level Machine Translationcited by 1
  19. 2023Optimizing Non-Autoregressive Transformers with Contrastive Learningcited by 1
  20. 2024Control-DAG: Constrained Decoding for Non-Autoregressive Directed Acyclic T5 using Weighted Finite State Automatacited by 1
  21. 2024What Have We Achieved on Non-autoregressive Translation?cited by 1
  22. 2025Multilingual Non-Autoregressive Machine Translation without Knowledge Distillationcited by 1
  23. 2025Confidence-Modulated Speculative Decoding for Large Language Modelscited by 1
  24. 2022DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translationcited by 0
  25. 2023Improving Non-autoregressive Translation Quality with Pretrained Language Model, Embedding Distillation and Upsampling Strategy for CTCcited by 0
  26. 2023Learning Socio-Temporal Graphs for Multi-Agent Trajectory Predictioncited by 0
  27. 2024On the Information Redundancy in Non-Autoregressive Translationcited by 0
  28. 2024DiffNorm: Self-Supervised Normalization for Non-autoregressive Speech-to-speech Translationcited by 0
  29. 2025RTGen: Real-Time Generative Detection Transformercited by 0
  30. 2026Reconsidering Positional Supervision in Masked Diffusion Language Model Trainingcited by 0
  31. 2026Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detectioncited by 0
  32. 2026DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Modelscited by 0