mlsanityData to 14 Oct 2026

Domain Robustness in Neural Machine Translation

arXiv 1911.03109 ↗First posted 8 Nov 2019cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

Translating text that diverges from the training domain is a key challenge for machine translation. Domain robustness---the generalization of models to unseen test domains---is low for both statistical (SMT) and neural machine translation (NMT). In this paper, we study the performance of SMT and NMT models on out-of-domain test sets. We find that in unknown domains, SMT and NMT suffer from very different problems: SMT systems are mostly adequate but not fluent, while NMT systems are mostly fluent, but not adequate. For NMT, we identify such hallucinations (translations that are fluent but unrelated to the source) as a key reason for low domain robustness. To mitigate this problem, we empirically compare methods that are reported to improve adequacy or in-domain robustness in terms of their effectiveness at improving domain robustness. In experiments on German to English OPUS data, and German to Romansh (a low-resource setting) we find that several methods improve domain robustness. While those methods do lead to higher BLEU scores overall, they only slightly increase the adequacy of translations compared to SMT.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 17; 2023: 15; 2024: 8; 2025: 3; 2026: 2.

65 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  3. 2021The Curious Case of Hallucinations in Neural Machine Translationcited by 161
  4. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 132
  5. 2023Hallucinations in Large Multilingual Translation Modelscited by 129
  6. 2021Survey of Low-Resource Machine Translationcited by 92
  7. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarkscited by 76
  8. 2022Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Bettercited by 75
  9. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
  10. 2021PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domainscited by 61
  11. 2022Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translationcited by 61
  12. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
38 more
  1. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  2. 2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
  3. 2023HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translationcited by 35
  4. 2023BLESS: Benchmarking Large Language Models on Sentence Simplificationcited by 35
  5. 2021Prevent the Language Model from being Overconfident in Neural Machine Translationcited by 34
  6. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  7. 2023Improving Translation Faithfulness of Large Language Models via Augmenting Instructionscited by 28
  8. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  9. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  10. 2023Generative Models as a Complex Systems Science: How can we make sense of large language model behavior?cited by 19
  11. 2022Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrievalcited by 17
  12. 2024An Audit on the Perspectives and Challenges of Hallucinations in NLPcited by 17
  13. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  14. 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
  15. 2022Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translationcited by 13
  16. 2023Why LLMs Hallucinate, and How to Get (Evidential) Closure: Perceptual, Intensional, and Extensional Learning for Faithful Natural Language Generationcited by 13
  17. 2023Data Augmentation for Neural Machine Translation using Generative Language Modelcited by 12
  18. 2021Improving Both Domain Robustness and Domain Adaptability in Machine Translationcited by 11
  19. 2021Rethinking Data Augmentation for Low-Resource Neural Machine Translation: A Multi-Task Learning Approachcited by 8
  20. 2024Simpson's Paradox and the Accuracy-Fluency Tradeoff in Translationcited by 8
  21. 2024Large Language Models for Dysfluency Detection in Stuttered Speechcited by 8
  22. 2023Hallucination Detection for Grounded Instruction Generationcited by 7
  23. 2022Anticipation-Free Training for Simultaneous Machine Translationcited by 5
  24. 2022Learning ReLU networks to high uniform accuracy is intractablecited by 5
  25. 2022Reducing Hallucinations in Neural Machine Translation with Feature Attributioncited by 5
  26. 2021The Highs and Lows of Simple Lexical Domain Adaptation Approaches for Neural Machine Translationcited by 4
  27. 2022Rethinking Round-Trip Translation for Machine Translation Evaluationcited by 4
  28. 2025Expanding the WMT24++ Benchmark with Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, and Valladercited by 4
  29. 2021Improving Lexically Constrained Neural Machine Translation with Source-Conditioned Masked Span Predictioncited by 3
  30. 2022Checks and Strategies for Enabling Code-Switched Machine Translationcited by 3
  31. 2022Domain Adaptation of Machine Translation with Crowdworkerscited by 3
  32. 2023Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequencescited by 3
  33. 2023Does mBERT understand Romansh? Evaluating word embeddings using word alignmentcited by 3
  34. 2025The Mediomatix Corpus: Parallel Data for Romansh Language Varieties via Comparable Schoolbookscited by 3
  35. 2021Distributionally Robust Recurrent Decoders with Random Network Distillationcited by 2
  36. 2022Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?cited by 2
  37. 2023TreeSwap: Data Augmentation for Machine Translation via Dependency Subtree Swappingcited by 2
  38. 2024Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Correctionscited by 2

50 most cited of 65 All 65 (JSON)