# Neural Machine Translation Training in a Multi-Domain Scenario

arXiv: https://arxiv.org/abs/1708.08712 · HTML: https://mlsanity.com/paper/1708.08712/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Aug 2017.

## Built on by (most cited first, 5 of 5)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [Robust Domain Adaptation for Pre-trained Multilingual Neural Machine Translation Models](https://mlsanity.com/paper/2210.14979/) (2022)
- [Specializing Multi-domain NMT via Penalizing Low Mutual Information](https://mlsanity.com/paper/2210.12910/) (2022)
- [Utilizing Out-Domain Datasets to Enhance Multi-Task Citation Analysis](https://mlsanity.com/paper/2202.10884/) (2022)

## Abstract (arXiv)

In this paper, we explore alternative ways to train a neural machine translation system in a multi-domain scenario. We investigate data concatenation (with fine tuning), model stacking (multi-level fine tuning), data selection and multi-model ensemble. Our findings show that the best translation quality can be achieved by building an initial system on a concatenation of available out-of-domain data and then fine-tuning it on in-domain data. Model stacking works best when training begins with the furthest out-of-domain data and the model is incrementally fine-tuned with the next furthest domain and so on. Data selection did not give the best results, but can be considered as a decent compromise between training time and translation quality. A weighted ensemble of different individual models performed better than data selection. It is beneficial in a scenario when there is no time for fine-tuning an already trained model.