# Data Diversification: A Simple Strategy For Neural Machine Translation

arXiv: https://arxiv.org/abs/1911.01986 · HTML: https://mlsanity.com/paper/1911.01986/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2019.

## Built on by (most cited first, 6 of 6)

- [Self-supervised and Supervised Joint Training for Resource-rich Machine Translation](https://mlsanity.com/paper/2106.04060/) (2021)
- [CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2204.00665/) (2022)
- [Nearest Neighbour Few-Shot Learning for Cross-lingual Classification](https://mlsanity.com/paper/2109.02221/) (2021)
- [BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation](https://mlsanity.com/paper/2204.07837/) (2022)
- [Improving English to Sinhala Neural Machine Translation using Part-of-Speech Tag](https://mlsanity.com/paper/2202.08882/) (2022)
- [Dict-NMT: Bilingual Dictionary based NMT for Extremely Low Resource Languages](https://mlsanity.com/paper/2206.04439/) (2022)

## Abstract (arXiv)

We introduce Data Diversification: a simple but effective strategy to boost neural machine translation (NMT) performance. It diversifies the training data by using the predictions of multiple forward and backward models and then merging them with the original dataset on which the final NMT model is trained. Our method is applicable to all NMT models. It does not require extra monolingual data like back-translation, nor does it add more computations and parameters like ensembles of models. Our method achieves state-of-the-art BLEU scores of 30.7 and 43.7 in the WMT'14 English-German and English-French translation tasks, respectively. It also substantially improves on 8 other translation tasks: 4 IWSLT tasks (English-German and English-French) and 4 low-resource translation tasks (English-Nepali and English-Sinhala). We demonstrate that our method is more effective than knowledge distillation and dual learning, it exhibits strong correlation with ensembles of models, and it trades perplexity off for better BLEU score. We have released our source code at https://github.com/nxphi47/data_diversification