mlsanityData to 14 Oct 2026

Finding the Right Recipe for Low Resource Domain Adaptation in Neural Machine Translation

arXiv 2206.01137 ↗First posted 2 Jun 2022cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

General translation models often still struggle to generate accurate translations in specialized domains. To guide machine translation practitioners and characterize the effectiveness of domain adaptation methods under different data availability scenarios, we conduct an in-depth empirical exploration of monolingual and parallel data approaches to domain adaptation of pre-trained, third-party, NMT models in settings where architecture change is impractical. We compare data centric adaptation methods in isolation and combination. We study method effectiveness in very low resource (8k parallel examples) and moderately low resource (46k parallel examples) conditions and propose an ensemble approach to alleviate reductions in original domain translation quality. Our work includes three domains: consumer electronic, clinical, and biomedical and spans four language pairs - Zh-En, Ja-En, Es-En, and Ru-En. We also make concrete recommendations for achieving high in-domain performance and release our consumer electronic and medical domain datasets for all languages and make our code publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

25 of 38 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2019Cross-lingual Language Model Pretrainingcited by 1,090
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  11. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  12. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
13 more
  1. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 347
  2. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 273
  3. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  4. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 150
  5. 2016Dual Learning for Machine Translationcited by 148
  6. 2019Adaptation of Deep Bidirectional Multilingual Transformers for Russian Languagecited by 109
  7. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  8. 2019Tagged Back-Translationcited by 77
  9. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  10. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  11. 2018Unsupervised Statistical Machine Translationcited by 51
  12. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  13. 2019Domain Differential Adaptation for Neural Machine Translationcited by 10

Built on by

Not cited yet.