# Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation

arXiv: https://arxiv.org/abs/2009.07177 · HTML: https://mlsanity.com/paper/2009.07177/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 13 of 13)

- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://mlsanity.com/paper/2210.17432/) (2022)
- [Step-unrolled Denoising Autoencoders for Text Generation](https://mlsanity.com/paper/2112.06749/) (2021)
- [Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline](https://mlsanity.com/paper/2305.13144/) (2023)
- [Directed Acyclic Transformer for Non-Autoregressive Machine Translation](https://mlsanity.com/paper/2205.07459/) (2022)
- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate](https://mlsanity.com/paper/2109.06481/) (2021)
- [Consistency Training with Virtual Adversarial Discrete Perturbation](https://mlsanity.com/paper/2104.07284/) (2021)
- [Nearest Neighbor Non-autoregressive Text Generation](https://mlsanity.com/paper/2208.12496/) (2022)
- [Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines](https://mlsanity.com/paper/2407.21046/) (2024)
- [Learning and Analyzing Generation Order for Undirected Sequence Models](https://mlsanity.com/paper/2112.09097/) (2021)
- [DiMS: Distilling Multiple Steps of Iterative Non-Autoregressive Transformers for Machine Translation](https://mlsanity.com/paper/2206.02999/) (2022)
- [NADIR: Differential Attention Flow for Non-Autoregressive Transliteration in Indic Languages](https://mlsanity.com/paper/2601.12389/) (2026)

## Abstract (arXiv)

We propose an efficient inference procedure for non-autoregressive machine translation that iteratively refines translation purely in the continuous space. Given a continuous latent variable model for machine translation (Shu et al., 2020), we train an inference network to approximate the gradient of the marginal log probability of the target sentence, using only the latent variable as input. This allows us to use gradient-based optimization to find the target sentence at inference time that approximately maximizes its marginal probability. As each refinement step only involves computation in the latent space of low dimensionality (we use 8 in our experiments), we avoid computational overhead incurred by existing non-autoregressive inference procedures that often refine in token space. We compare our approach to a recently proposed EM-like inference procedure (Shu et al., 2020) that optimizes in a hybrid space, consisting of both discrete and continuous variables. We evaluate our approach on WMT'14 En-De, WMT'16 Ro-En and IWSLT'16 De-En, and observe two advantages over the EM-like inference: (1) it is computationally efficient, i.e. each refinement step is twice as fast, and (2) it is more effective, resulting in higher marginal probabilities and BLEU scores with the same number of refinement steps. On WMT'14 En-De, for instance, our approach is able to decode 6.2 times faster than the autoregressive model with minimal degradation to translation quality (0.9 BLEU).