# The NiuTrans System for the WMT21 Efficiency Task

arXiv: https://arxiv.org/abs/2109.08003 · HTML: https://mlsanity.com/paper/2109.08003/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 31 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [HAT: Hardware-Aware Transformers for Efficient Natural Language Processing](https://mlsanity.com/paper/2005.14187/) (2020)
- [A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation](https://mlsanity.com/paper/2009.13818/) (2020)
- [Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model](https://mlsanity.com/paper/1906.00532/) (2019)
- [FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference](https://mlsanity.com/paper/2101.05615/) (2021)
- [Shallow-to-Deep Training for Neural Machine Translation](https://mlsanity.com/paper/2010.03737/) (2020)
- [Towards Fully 8-bit Integer Inference for the Transformer Model](https://mlsanity.com/paper/2009.08034/) (2020)
- [Weight Distillation: Transferring the Knowledge in Neural Network Parameters](https://mlsanity.com/paper/2009.09152/) (2020)
- [Learning Light-Weight Translation Models from Deep Transformer](https://mlsanity.com/paper/2012.13866/) (2020)
- [RankNAS: Efficient Neural Architecture Search by Pairwise Ranking](https://mlsanity.com/paper/2109.07383/) (2021)
- [An Efficient Transformer Decoder with Compressed Sub-layers](https://mlsanity.com/paper/2101.00542/) (2021)
- [Bag of Tricks for Optimizing Transformer Efficiency](https://mlsanity.com/paper/2109.04030/) (2021)
- [Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation](https://mlsanity.com/paper/2010.08265/) (2020)

## Built on by (most cited first, 2 of 2)

- [Learning Evaluation Models from Large Language Models for Sequence Generation](https://mlsanity.com/paper/2308.04386/) (2023)
- [The RoyalFlush System for the WMT 2022 Efficiency Task](https://mlsanity.com/paper/2212.01543/) (2022)

## Abstract (arXiv)

This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt.org/wmt21/efficiency-task.html). Following last year's work, we explore various techniques to improve efficiency while maintaining translation quality. We investigate the combinations of lightweight Transformer architectures and knowledge distillation strategies. Also, we improve the translation efficiency with graph optimization, low precision, dynamic batching, and parallel pre/post-processing. Our system can translate 247,000 words per second on an NVIDIA A100, being 3× faster than last year's system. Our system is the fastest and has the lowest memory consumption on the GPU-throughput track. The code, model, and pipeline will be available at NiuTrans.NMT (https://github.com/NiuTrans/NiuTrans.NMT).