mlsanityData to 7 Oct 2026

Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

arXiv 2104.06951 ↗First posted 14 Apr 2021cs.CLcited by 46

Not on the map. Why?

Abstract (arXiv)

The development of deep learning techniques has allowed Neural Machine Translation (NMT) models to become extremely powerful, given sufficient training data and training time. However, systems struggle when translating text from a new domain with a distinct style or vocabulary. Fine-tuning on in-domain data allows good domain adaptation, but requires sufficient relevant bilingual data. Even if this is available, simple fine-tuning can cause overfitting to new data and `catastrophic forgetting' of previously learned behaviour. We concentrate on robust approaches to domain adaptation for NMT, particularly where a system may need to translate across multiple domains. We divide techniques into those revolving around data selection or generation, model architecture, parameter adaptation procedure, and inference procedure. We finally highlight the benefits of domain adaptation and multi-domain adaptation techniques to other lines of NMT research.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 190 of its 367 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

190 of 367 references.

References · 190

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  12. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
38 more
  1. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  4. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  5. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  6. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  7. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  8. 2016Sequence-Level Knowledge Distillationcited by 761
  9. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  10. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  11. 2019Gradient based sample selection for online continual learningcited by 568
  12. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  13. 2017Six Challenges for Neural Machine Translationcited by 334
  14. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  15. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  16. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  17. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  18. 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
  19. 2019Evaluating Gender Bias in Machine Translationcited by 247
  20. 2019Massively Multilingual Neural Machine Translationcited by 246
  21. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  22. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 234
  23. 2020Nearest Neighbor Machine Translationcited by 209
  24. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  25. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  26. 2020Unsupervised Domain Clusters in Pretrained Language Modelscited by 160
  27. 2015On Using Monolingual Corpora in Neural Machine Translationcited by 149
  28. 2020Neural Unsupervised Domain Adaptation in NLP---A Surveycited by 142
  29. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  30. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  31. 2015Character-Aware Neural Language Modelscited by 129
  32. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  33. 2018Analyzing Uncertainty in Neural Machine Translationcited by 126
  34. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  35. 2021Gender Bias in Machine Translationcited by 116
  36. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  37. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  38. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113

50 most cited of 190 All 190 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 9; 2023: 9; 2024: 15; 2025: 5; 2026: 5.

46 papers.

Sort:

By area:

  1. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  2. 2023Modular Deep Learningcited by 82
  3. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  4. 2023Federated Domain Generalization: A Surveycited by 25
  5. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  6. 2022Chunk-based Nearest Neighbor Machine Translationcited by 21
  7. 2024Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuningcited by 11
  8. 2022Improving Automated Program Repair with Domain Adaptationcited by 10
  9. 2023Domain-adaptive Message Passing Graph Neural Networkcited by 10
  10. 2023Supervised Learning and Large Language Model Benchmarks on Mental Health Datasets: Cognitive Distortions and Suicidal Risks in Chinese Social Mediacited by 9
  11. 2022Adaptor: Objective-Centric Adaptation Framework for Language Modelscited by 7
  12. 2022Efficient Machine Translation Domain Adaptationcited by 7
34 more
  1. 2022When does Parameter-Efficient Transfer Learning Work for Machine Translation?cited by 7
  2. 2021Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modelingcited by 5
  3. 2024Low-resource Machine Translation: what for? who for? An observational study on a dedicated Tetun language translation servicecited by 5
  4. 2021Selecting Parallel In-domain Sentences for Neural Machine Translation Using Monolingual Textscited by 4
  5. 2023Tailoring Domain Adaptation for Machine Translation Quality Estimationcited by 4
  6. 2023Adaptation Speed Analysis for Fairness-aware Causal Modelscited by 4
  7. 2024What the Weight?! A Unified Framework for Zero-Shot Knowledge Compositioncited by 4
  8. 2025Specification-Aware Machine Translation and Evaluation for Purpose Alignmentcited by 4
  9. 2024Domain Adaptation for Time series Transformers using One-step fine-tuningcited by 3
  10. 2024QAEncoder: Towards Aligned Representation Learning in Question Answering Systemscited by 3
  11. 2024Scalable Multi-Domain Adaptation of Language Models using Modular Expertscited by 3
  12. 2021SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shiftscited by 2
  13. 2024Can Humans Identify Domains?cited by 2
  14. 2024On Instruction-Finetuning Neural Machine Translation Modelscited by 2
  15. 2024Domain adapted machine translation: What does catastrophic forgetting forget and why?cited by 2
  16. 2026KD4MT: A Survey of Knowledge Distillation for Machine Translationcited by 2
  17. 2023Domain Adaptation for Arabic Machine Translation: The Case of Financial Textscited by 1
  18. 2024A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-Field Perspectivecited by 1
  19. 2024F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translationcited by 1
  20. 2024Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translationcited by 1
  21. 2024From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translationcited by 1
  22. 2025Beyond English: Evaluating Automated Measurement of Moral Foundations in Non-English Discourse with a Chinese Case Studycited by 1
  23. 2022Finding the Right Recipe for Low Resource Domain Adaptation in Neural Machine Translationcited by 0
  24. 2022Vernacular Search Query Translation with Unsupervised Domain Adaptationcited by 0
  25. 2022Soft Alignment Objectives for Robust Adaptation of Language Generationcited by 0
  26. 2024IMO: Greedy Layer-Wise Sparse Representation Learning for Out-of-Distribution Text Classification with Pre-trained Modelscited by 0
  27. 2024Defining Boundaries: The Impact of Domain Specification on Cross-Language and Cross-Domain Transfer in Machine Translationcited by 0
  28. 2025In-Domain African Languages Translation Using LLMs and Multi-armed Banditscited by 0
  29. 2025TULUN: Transparent and Adaptable Low-resource Machine Translationcited by 0
  30. 2025Optimal Corpus Aware Training for Neural Machine Translationcited by 0
  31. 2026BYOL: Bring Your Own Language Into LLMscited by 0
  32. 2026Consensus-Aligned Neuron Efficient Fine-Tuning Large Language Models for Multi-Domain Machine Translationcited by 0
  33. 2026LLMs in the Real World: Evaluating "AI" in Emergency Contextscited by 0
  34. 2026PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translationcited by 0