# Finding Sparse Structures for Domain Specific Neural Machine Translation

arXiv: https://arxiv.org/abs/2012.10586 · HTML: https://mlsanity.com/paper/2012.10586/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2020.

## Built on by (most cited first, 3 of 3)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Pruning-then-Expanding Model for Domain Adaptation of Neural Machine Translation](https://mlsanity.com/paper/2103.13678/) (2021)
- [Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions](https://mlsanity.com/paper/2211.01542/) (2022)

## Abstract (arXiv)

Neural machine translation often adopts the fine-tuning approach to adapt to specific domains. However, nonrestricted fine-tuning can easily degrade on the general domain and over-fit to the target domain. To mitigate the issue, we propose Prune-Tune, a novel domain adaptation method via gradual pruning. It learns tiny domain-specific sub-networks during fine-tuning on new domains. Prune-Tune alleviates the over-fitting and the degradation problem without model modification. Furthermore, Prune-Tune is able to sequentially learn a single network with multiple disjoint domain-specific sub-networks for multiple domains. Empirical experiment results show that Prune-Tune outperforms several strong competitors in the target domain test set without sacrificing the quality on the general domain in both single and multi-domain settings. The source code and data are available at https://github.com/ohlionel/Prune-Tune.