# Imitation Learning for Non-Autoregressive Neural Machine Translation

arXiv: https://arxiv.org/abs/1906.02041 · HTML: https://mlsanity.com/paper/1906.02041/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 30 of 36)

- [Speculative Decoding with Big Little Decoder](https://mlsanity.com/paper/2302.07863/) (2023)
- [LLM Inference Unveiled: Survey and Roofline Model Insights](https://mlsanity.com/paper/2402.16363/) (2024)
- [Autoregressive Knowledge Distillation through Imitation Learning](https://mlsanity.com/paper/2009.07253/) (2020)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2205.07459/) (2022)
- [Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervision](https://mlsanity.com/paper/2110.07515/) (2021)
- [Non-Autoregressive Text Generation with Pre-trained Language Models](https://mlsanity.com/paper/2102.08220/) (2021)
- [Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation](https://mlsanity.com/paper/2106.00903/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [On the Learning of Non-Autoregressive Transformers](https://mlsanity.com/paper/2206.05975/) (2022)
- [Non-Autoregressive Translation by Learning Target Categorical Codes](https://mlsanity.com/paper/2103.11405/) (2021)
- [Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding](https://mlsanity.com/paper/2402.13720/) (2024)
- [textitlatent-GLAT: Glancing at Latent Variables for Parallel Text Generation](https://mlsanity.com/paper/2204.02030/) (2022)
- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Sequence-Level Training for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2106.08122/) (2021)
- [Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation](https://mlsanity.com/paper/2101.08942/) (2021)
- [Semi-Autoregressive Image Captioning](https://mlsanity.com/paper/2110.05342/) (2021)
- [Duplex Sequence-to-Sequence Learning for Reversible Machine Translation](https://mlsanity.com/paper/2105.03458/) (2021)
- [Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring](https://mlsanity.com/paper/2109.04411/) (2021)
- [Improving Non-autoregressive Generation with Mixup Training](https://mlsanity.com/paper/2110.11115/) (2021)
- [Uncertainty-Aware Image Captioning](https://mlsanity.com/paper/2211.16769/) (2022)
- [MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2108.08447/) (2021)
- [Non-Autoregressive Neural Machine Translation: A Call for Clarity](https://mlsanity.com/paper/2205.10577/) (2022)
- [DePA: Improving Non-autoregressive Machine Translation with Dependency-Aware Decoder](https://mlsanity.com/paper/2203.16266/) (2022)
- [Diffusion Glancing Transformer for Parallel Sequence to Sequence Learning](https://mlsanity.com/paper/2212.10240/) (2022)
- [Extending Multilingual Machine Translation through Imitation Learning](https://mlsanity.com/paper/2311.08538/) (2023)
- [PARAGEN : A Parallel Generation Toolkit](https://mlsanity.com/paper/2210.03405/) (2022)
- [TreePiece: Faster Semantic Parsing via Tree Tokenization](https://mlsanity.com/paper/2303.17161/) (2023)
- [Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2303.17910/) (2023)

## Abstract (arXiv)

Non-autoregressive translation models (NAT) have achieved impressive inference speedup. A potential issue of the existing NAT algorithms, however, is that the decoding is conducted in parallel, without directly considering previous context. In this paper, we propose an imitation learning framework for non-autoregressive machine translation, which still enjoys the fast translation speed but gives comparable translation performance compared to its auto-regressive counterpart. We conduct experiments on the IWSLT16, WMT14 and WMT16 datasets. Our proposed model achieves a significant speedup over the autoregressive models, while keeping the translation quality comparable to the autoregressive models. By sampling sentence length in parallel at inference time, we achieve the performance of 31.85 BLEU on WMT16 RorightarrowEn and 30.68 BLEU on IWSLT16 EnrightarrowDe.