# The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21

arXiv: https://arxiv.org/abs/2109.11247 · HTML: https://mlsanity.com/paper/2109.11247/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [Bridging the Gap between Training and Inference for Neural Machine Translation](https://mlsanity.com/paper/1906.02448/) (2019)
- [Glancing Transformer for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2008.07905/) (2020)
- [Vocabulary Learning via Optimal Transport for Neural Machine Translation](https://mlsanity.com/paper/2012.15671/) (2020)
- [Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation](https://mlsanity.com/paper/1911.03362/) (2019)
- [Joint Training for Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1803.00353/) (2018)
- [WeChat Neural Machine Translation Systems for WMT20](https://mlsanity.com/paper/2010.00247/) (2020)
- [The Volctrans Machine Translation System for WMT20](https://mlsanity.com/paper/2010.14806/) (2020)

## Built on by (most cited first, 14 of 14)

- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises](https://mlsanity.com/paper/2302.10025/) (2023)
- [Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2205.07459/) (2022)
- [Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning](https://mlsanity.com/paper/2308.12219/) (2023)
- [On the Learning of Non-Autoregressive Transformers](https://mlsanity.com/paper/2206.05975/) (2022)
- [textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generation](https://mlsanity.com/paper/2204.02030/) (2022)
- [Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2303.06662/) (2023)
- [INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2306.06381/) (2023)
- [On the Effectiveness of Automated Metrics for Text Generation Systems](https://mlsanity.com/paper/2210.13025/) (2022)
- [Accurate Knowledge Distillation with n-best Reranking](https://mlsanity.com/paper/2305.12057/) (2023)
- [PARAGEN : A Parallel Generation Toolkit](https://mlsanity.com/paper/2210.03405/) (2022)
- [Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2303.17910/) (2023)
- [Attentive Multi-Layer Perceptron for Non-autoregressive Generation](https://mlsanity.com/paper/2310.09512/) (2023)

## Abstract (arXiv)

This paper describes the Volctrans' submission to the WMT21 news translation shared task for German->English translation. We build a parallel (i.e., non-autoregressive) translation system using the Glancing Transformer, which enables fast and accurate parallel decoding in contrast to the currently prevailing autoregressive models. To the best of our knowledge, this is the first parallel translation system that can be scaled to such a practical scenario like WMT competition. More importantly, our parallel translation system achieves the best BLEU score (35.0) on German->English translation task, outperforming all strong autoregressive counterparts.