# Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing

arXiv: https://arxiv.org/abs/1911.02692 · HTML: https://mlsanity.com/paper/1911.02692/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2019.

## Built on by (most cited first, 14 of 14)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters](https://mlsanity.com/paper/2110.09574/) (2021)
- [Improving Both Domain Robustness and Domain Adaptability in Machine Translation](https://mlsanity.com/paper/2112.08288/) (2021)
- [GTrans: Grouping and Fusing Transformer Layers for Neural Machine Translation](https://mlsanity.com/paper/2207.14467/) (2022)
- [NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts](https://mlsanity.com/paper/2305.16023/) (2023)
- [CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing](https://mlsanity.com/paper/2204.06625/) (2022)
- [Token-wise Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2103.11088/) (2021)
- [DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain Translation](https://mlsanity.com/paper/2504.20371/) (2025)
- [Translation Transformers Rediscover Inherent Data Domains](https://mlsanity.com/paper/2109.07864/) (2021)
- [Specializing Multi-domain NMT via Penalizing Low Mutual Information](https://mlsanity.com/paper/2210.12910/) (2022)
- [Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translation](https://mlsanity.com/paper/2407.01126/) (2024)
- [Label-Free Multi-Domain Machine Translation with Stage-wise Training](https://mlsanity.com/paper/2305.03949/) (2023)
- [Source Prompt: Coordinated Pre-training of Language Models on Diverse Corpora from Multiple Sources](https://mlsanity.com/paper/2311.09732/) (2023)
- [PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation](https://mlsanity.com/paper/2608.03077/) (2026)

## Abstract (arXiv)

Many multi-domain neural machine translation (NMT) models achieve knowledge transfer by enforcing one encoder to learn shared embedding across domains. However, this design lacks adaptation to individual domains. To overcome this limitation, we propose a novel multi-domain NMT model using individual modules for each domain, on which we apply word-level, adaptive and layer-wise domain mixing. We first observe that words in a sentence are often related to multiple domains. Hence, we assume each word has a domain proportion, which indicates its domain preference. Then word representations are obtained by mixing their embedding in individual domains based on their domain proportions. We show this can be achieved by carefully designing multi-head dot-product attention modules for different domains, and eventually taking weighted averages of their parameters by word-level layer-wise domain proportions. Through this, we can achieve effective domain knowledge sharing, and capture fine-grained domain-specific knowledge as well. Our experiments show that our proposed model outperforms existing ones in several NMT tasks.