# Generating Diverse Translation from Model Distribution with Dropout

arXiv: https://arxiv.org/abs/2010.08178 · HTML: https://mlsanity.com/paper/2010.08178/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [One Reference Is Not Enough: Diverse Distillation with Reference Selection for Non-Autoregressive Translation](https://mlsanity.com/paper/2205.14333/) (2022)
- [Mixup Decoding for Diverse Machine Translation](https://mlsanity.com/paper/2109.03402/) (2021)
- [To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation](https://mlsanity.com/paper/2401.01419/) (2024)
- [WeTS: A Benchmark for Translation Suggestion](https://mlsanity.com/paper/2110.05151/) (2021)
- [Controlling Styles in Neural Machine Translation with Activation Prompt](https://mlsanity.com/paper/2212.08909/) (2022)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [Generating Diverse Translation with Perturbed kNN-MT](https://mlsanity.com/paper/2402.09344/) (2024)
- [EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation](https://mlsanity.com/paper/2203.02180/) (2022)
- [Diverse Sign Language Translation](https://mlsanity.com/paper/2410.19586/) (2024)
- [DiffSLT: Enhancing Diversity in Sign Language Translation via Diffusion Model](https://mlsanity.com/paper/2411.17248/) (2024)

## Abstract (arXiv)

Despite the improvement of translation quality, neural machine translation (NMT) often suffers from the lack of diversity in its generation. In this paper, we propose to generate diverse translations by deriving a large number of possible models with Bayesian modelling and sampling models from them for inference. The possible models are obtained by applying concrete dropout to the NMT model and each of them has specific confidence for its prediction, which corresponds to a posterior model distribution under specific training data in the principle of Bayesian modeling. With variational inference, the posterior model distribution can be approximated with a variational distribution, from which the final models for inference are sampled. We conducted experiments on Chinese-English and English-German translation tasks and the results shows that our method makes a better trade-off between diversity and accuracy.