# Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation

arXiv: https://arxiv.org/abs/2006.05165 · HTML: https://mlsanity.com/paper/2006.05165/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2020.

## Built on by (most cited first, 19 of 19)

- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Sequence-Level Training for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2106.08122/) (2021)
- [Lossless Acceleration for Seq2seq Generation with Aggressive Decoding](https://mlsanity.com/paper/2205.10350/) (2022)
- [A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model](https://mlsanity.com/paper/2201.10707/) (2022)
- [Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2210.05193/) (2022)
- [Candidate Soups: Fusing Candidate Results Improves Translation Quality for Non-Autoregressive Translation](https://mlsanity.com/paper/2301.11503/) (2023)
- [Semi-Autoregressive Transformer for Image Captioning](https://mlsanity.com/paper/2106.09436/) (2021)
- [Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2303.06662/) (2023)
- [Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding](https://mlsanity.com/paper/2601.20430/) (2026)
- [CTC-based Non-autoregressive Speech Translation](https://mlsanity.com/paper/2305.17358/) (2023)
- [Modeling Coverage for Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/2104.11897/) (2021)
- [DePA: Improving Non-autoregressive Machine Translation with Dependency-Aware Decoder](https://mlsanity.com/paper/2203.16266/) (2022)
- [Hybrid-Regressive Neural Machine Translation](https://mlsanity.com/paper/2210.10416/) (2022)
- [FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version](https://mlsanity.com/paper/2604.05551/) (2026)
- [Non-Autoregressive Document-Level Machine Translation](https://mlsanity.com/paper/2305.12878/) (2023)
- [Progressive Neural Network for Multi-Horizon Time Series Forecasting](https://mlsanity.com/paper/2310.19322/) (2023)
- [D-Loop: Looped Diffusion Drafting for Speculative Decoding](https://mlsanity.com/paper/2610.06011/) (2026)

## Abstract (arXiv)

Non-autoregressive neural machine translation (NAT) predicts the entire target sequence simultaneously and significantly accelerates inference process. However, NAT discards the dependency information in a sentence, and thus inevitably suffers from the multi-modality problem: the target tokens may be provided by different possible translations, often causing token repetitions or missing. To alleviate this problem, we propose a novel semi-autoregressive model RecoverSAT in this work, which generates a translation as a sequence of segments. The segments are generated simultaneously while each segment is predicted token-by-token. By dynamically determining segment length and deleting repetitive segments, RecoverSAT is capable of recovering from repetitive and missing token errors. Experimental results on three widely-used benchmark datasets show that our proposed model achieves more than 4× speedup while maintaining comparable performance compared with the corresponding autoregressive model.