mlsanityData to 14 Oct 2026

Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translation

arXiv 2306.01382 ↗First posted 2 Jun 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing or under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

26 of 38 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  5. 2019How multilingual is Multilingual BERT?cited by 645
  6. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  7. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  8. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  9. 2018Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Taskscited by 215
  10. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  11. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  12. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
14 more
  1. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  2. 2019The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-Englishcited by 62
  3. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 42
  4. 2020A Call for More Rigor in Unsupervised Cross-lingual Learningcited by 37
  5. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  6. 2021Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translationcited by 29
  7. 2022Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?cited by 24
  8. 2021ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generationcited by 23
  9. 2021PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretrainingcited by 21
  10. 2018Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabularycited by 18
  11. 2020Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translationcited by 16
  12. 2022Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translationcited by 14
  13. 2022X-SCITLDR: Cross-Lingual Extreme Summarization of Scholarly Documentscited by 12
  14. 2019Exploiting Out-of-Domain Parallel Data through Multilingual Transfer Learning for Low-Resource Neural Machine Translationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 1.

5 papers.

  1. 2024Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corporacited by 13
  2. 2024SiTSE: Sinhala Text Simplification Dataset and Evaluationcited by 3
  3. 2024Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translationcited by 2
  4. 2025Beyond Vanilla Fine-Tuning: Leveraging Multistage, Multilingual, and Domain-Specific Methods for Low-Resource Machine Translationcited by 2
  5. 2026EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translationcited by 1