mlsanityData to 7 Oct 2026

Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translation

arXiv 2412.19522 ↗First posted 27 Dec 2024cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) systems built on multilingual sequence-to-sequence Language Models (msLMs) fail to deliver expected results when the amount of parallel data for a language, as well as the language's representation in the model are limited. This restricts the capabilities of domain-specific NMT systems for low-resource languages (LRLs). As a solution, parallel data from auxiliary domains can be used either to fine-tune or to further pre-train the msLM. We present an evaluation of the effectiveness of these two techniques in the context of domain-specific LRL-NMT. We also explore the impact of domain divergence on NMT model performance. We recommend several strategies for utilizing auxiliary parallel data in building domain-specific NMT models for LRLs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 27 Dec 2024.

Builds on

32 of 50 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  6. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  7. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  8. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  9. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 214
  10. 2017Unsupervised Neural Machine Translationcited by 148
  11. 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
  12. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
20 more
  1. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  2. 2021Survey of Low-Resource Machine Translationcited by 92
  3. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 57
  4. 2022Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP Worldcited by 55
  5. 2021MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairscited by 41
  6. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 40
  7. 2020Language Model Prior for Low-Resource Neural Machine Translationcited by 37
  8. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  9. 2016Neural Machine Translation with Pivot Languagescited by 25
  10. 2020Domain Divergences: a Survey and Empirical Analysiscited by 24
  11. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
  12. 2021nmT5 -- Is parallel data still relevant for pre-training massively multilingual language models?cited by 21
  13. 2021PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretrainingcited by 21
  14. 2020Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translationcited by 16
  15. 2024Predicting Machine Translation Performance on Low-Resource Languages: The Role of Domain Similaritycited by 15
  16. 2024Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corporacited by 13
  17. 2019Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translationcited by 11
  18. 2021Cross-lingual Intermediate Fine-tuning improves Dialogue State Trackingcited by 11
  19. 2023Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translationcited by 5
  20. 2021Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translationcited by 1
  2. 2026Mix-MoE: Improving Multilingual Machine Translation of Large Language Models through Mixed MoEscited by 0