# Specializing Multi-domain NMT via Penalizing Low Mutual Information

arXiv: https://arxiv.org/abs/2210.12910 · HTML: https://mlsanity.com/paper/2210.12910/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Simple, Scalable Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1909.08478/) (2019)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Domain Control for Neural Machine Translation](https://mlsanity.com/paper/1612.06140/) (2016)
- [It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information](https://mlsanity.com/paper/2005.02354/) (2020)
- [Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing](https://mlsanity.com/paper/1911.02692/) (2019)
- [Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2203.02951/) (2022)
- [Neural Machine Translation Training in a Multi-Domain Scenario](https://mlsanity.com/paper/1708.08712/) (2017)
- [Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2105.12523/) (2021)

## Built on by (most cited first, 1 of 1)

- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)

## Abstract (arXiv)

Multi-domain Neural Machine Translation (NMT) trains a single model with multiple domains. It is appealing because of its efficacy in handling multiple domains within one model. An ideal multi-domain NMT should learn distinctive domain characteristics simultaneously, however, grasping the domain peculiarity is a non-trivial task. In this paper, we investigate domain-specific information through the lens of mutual information (MI) and propose a new objective that penalizes low MI to become higher. Our method achieved the state-of-the-art performance among the current competitive multi-domain NMT models. Also, we empirically show our objective promotes low MI to be higher resulting in domain-specialized multi-domain NMT.