# Freezing Subnetworks to Analyze Domain Adaptation in Neural Machine Translation

arXiv: https://arxiv.org/abs/1809.05218 · HTML: https://mlsanity.com/paper/1809.05218/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2018.

## Built on by (most cited first, 10 of 10)

- [Modular Deep Learning](https://mlsanity.com/paper/2302.11529/) (2023)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?](https://mlsanity.com/paper/2410.09644/) (2024)
- [Breaking Down Multilingual Machine Translation](https://mlsanity.com/paper/2110.08130/) (2021)
- [Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translation](https://mlsanity.com/paper/2104.05848/) (2021)
- [Reinforcement Learning with Large Action Spaces for Neural Machine Translation](https://mlsanity.com/paper/2210.03053/) (2022)
- [Omnidirectional Video Super-Resolution using Deep Learning](https://mlsanity.com/paper/2506.14803/) (2025)
- [More Parameters? No Thanks!](https://mlsanity.com/paper/2107.09622/) (2021)
- [Thread-Efficient Decoding for Neural Texture Compression](https://mlsanity.com/paper/2608.27888/) (2026)

## Abstract (arXiv)

To better understand the effectiveness of continued training, we analyze the major components of a neural machine translation system (the encoder, decoder, and each embedding space) and consider each component's contribution to, and capacity for, domain adaptation. We find that freezing any single component during continued training has minimal impact on performance, and that performance is surprisingly good when a single component is adapted while holding the rest of the model fixed. We also find that continued training does not move the model very far from the out-of-domain model, compared to a sensitivity analysis metric, suggesting that the out-of-domain model can provide a good generic initialization for the new domain.