mlsanityData to 14 Oct 2026

Glancing Transformer for Non-Autoregressive Neural Machine Translation

arXiv 2008.07905 ↗First posted 18 Aug 2020cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

Recent work on non-autoregressive neural machine translation (NAT) aims at improving the efficiency by parallel decoding without sacrificing the quality. However, existing NAT methods are either inferior to Transformer or require multiple decoding passes, leading to reduced speedup. We propose the Glancing Language Model (GLM), a method to learn word interdependency for single-pass parallel generation models. With GLM, we develop Glancing Transformer (GLAT) for machine translation. With only single-pass parallel decoding, GLAT is able to generate high-quality translation with 8-15 times speedup. Experiments on multiple WMT language directions show that GLAT outperforms all previous single pass non-autoregressive methods, and is nearly comparable to Transformer, reducing the gap to 0.25-0.9 BLEU points.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). First posted on arXiv 18 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 37; 2023: 29; 2024: 14; 2025: 7; 2026: 6.

103 papers.

Sort:

By area:

  1. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  2. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  3. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
  4. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 178
  5. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 171
  6. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  7. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  8. 2023MMM: Generative Masked Motion Modelcited by 131
  9. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124
  10. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  11. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  12. 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 55
38 more
  1. 2022Bootstrapped Transformer for Offline Reinforcement Learningcited by 37
  2. 2022Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learningcited by 32
  3. 2022Draft-and-Revise: Effective Image Generation with Contextual RQ-Transformercited by 30
  4. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  5. 2021Non-Autoregressive Translation by Learning Target Categorical Codescited by 25
  6. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  7. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  8. 2023Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generationcited by 21
  9. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
  10. 2022textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generationcited by 20
  11. 2022One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translationcited by 20
  12. 2022Lossless Acceleration for Seq2seq Generation with Aggressive Decodingcited by 18
  13. 2022Non-Monotonic Latent Alignments for CTC-Based Non-Autoregressive Machine Translationcited by 16
  14. 2021The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21cited by 14
  15. 2022Recent Advances in Neural Text Generation: A Task-Agnostic Surveycited by 14
  16. 2024SurfPro: Functional Protein Design Based on Continuous Surfacecited by 14
  17. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  18. 2022Bayes risk CTC: Controllable CTC alignment in Sequence-to-Sequence taskscited by 13
  19. 2023Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translationcited by 13
  20. 2022Learning Non-Autoregressive Models from Search for Unsupervised Sentence Summarizationcited by 12
  21. 2023Importance Weighted Expectation-Maximization for Protein Sequence Designcited by 12
  22. 2023Non-autoregressive Streaming Transformer for Simultaneous Translationcited by 12
  23. 2023IPAD: Iterative, Parallel, and Diffusion-based Network for Scene Text Recognitioncited by 12
  24. 2022Non-Autoregressive Machine Translation: It's Not as Fast as it Seemscited by 11
  25. 2023Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translationcited by 11
  26. 2023Glancing Future for Simultaneous Machine Translationcited by 10
  27. 2024A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translationcited by 10
  28. 2021Duplex Sequence-to-Sequence Learning for Reversible Machine Translationcited by 9
  29. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  30. 2023CTC-based Non-autoregressive Speech Translationcited by 9
  31. 2023DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translationcited by 9
  32. 2023Simultaneous Machine Translation with Tailored Referencecited by 9
  33. 2021NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecastingcited by 8
  34. 2022ngram-OAXE: Phrase-Based Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 8
  35. 2023Regeneration Learning: A Learning Paradigm for Data Generationcited by 8
  36. 2023Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generationcited by 8
  37. 2023T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classificationcited by 8
  38. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8

50 most cited of 103 All 103 (JSON)