mlsanityData to 14 Oct 2026

Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?

arXiv 2203.08850 ↗First posted 16 Mar 2022cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

What can pre-trained multilingual sequence-to-sequence models like mBART contribute to translating low-resource languages? We conduct a thorough empirical experiment in 10 languages to ascertain this, considering five factors: (1) the amount of fine-tuning data, (2) the noise in the fine-tuning data, (3) the amount of pre-training data in the model, (4) the impact of domain mismatch, and (5) language typology. In addition to yielding several heuristics, the experiments form a framework for evaluating the data sensitivities of machine translation systems. While mBART is robust to domain differences, its translations for unseen and typologically distant languages remain below 3.0 BLEU. In answer to our title's question, mBART is not a low-resource panacea; we therefore encourage shifting the emphasis from new models to new data.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

29 of 47 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  7. 2019How multilingual is Multilingual BERT?cited by 645
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  9. 2017Six Challenges for Neural Machine Translationcited by 335
  10. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  11. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
  12. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
17 more
  1. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  2. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 109
  3. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  4. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  5. 2020Analyzing the Source and Target Contributions to Predictions in Neural Machine Translationcited by 71
  6. 2019Investigating Multilingual NMT Representations at Scalecited by 68
  7. 2021IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generationcited by 57
  8. 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
  9. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 42
  10. 2021The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translationcited by 38
  11. 2018Filtering and Mining Parallel Data in a Joint Multilingual Spacecited by 32
  12. 2021Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translationcited by 29
  13. 2019Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Setcited by 16
  14. 2020Data Augmentation and Terminology Integration for Domain-Specific Sinhala-English-Tamil Statistical Machine Translationcited by 16
  15. 2019Benchmarking Neural Machine Translation for Southern African Languagescited by 9
  16. 2019Evaluating the Supervised and Zero-shot Performance of Multi-lingual Translation Modelscited by 6
  17. 2021Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 12; 2025: 5; 2026: 1.

24 papers.

Sort:

  1. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  2. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
  3. 2024IrokoBench: A New Benchmark for African Languages in the Age of Large Language Modelscited by 43
  4. 2023Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMscited by 25
  5. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18
  6. 2024Quality Does Matter: A Detailed Look at the Quality and Utility of Web-Mined Parallel Corporacited by 13
  7. 2023Towards a Better Understanding of Variations in Zero-Shot Neural Machine Translation Performancecited by 10
  8. 2024Sinhala Transliteration: A Comparative Analysis Between Rule-based and Seq2Seq Approachescited by 6
  9. 2023Leveraging Auxiliary Domain Parallel Data in Intermediate Task Fine-tuning for Low-resource Translationcited by 5
  10. 2024Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMscited by 5
  11. 2023Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Modelscited by 4
  12. 2024Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translationcited by 4
12 more
  1. 2024How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translationcited by 4
  2. 2025Specification-Aware Machine Translation and Evaluation for Purpose Alignmentcited by 4
  3. 2024SiTSE: Sinhala Text Simplification Dataset and Evaluationcited by 3
  4. 2025Language Fusion for Parameter-Efficient Cross-lingual Transfercited by 3
  5. 2024Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translationcited by 2
  6. 2025Beyond Vanilla Fine-Tuning: Leveraging Multistage, Multilingual, and Domain-Specific Methods for Low-Resource Machine Translationcited by 2
  7. 2025Zero-Shot Performance Prediction for Probabilistic Scaling Lawscited by 2
  8. 2024Harnessing the Power of Artificial Intelligence to Vitalize Endangered Indigenous Languages: Technologies and Experiencescited by 1
  9. 2024ChatZero:Zero-shot Cross-Lingual Dialogue Generation via Pseudo-Target Languagecited by 1
  10. 2024A Data Selection Approach for Enhancing Low Resource Machine Translation Using Cross-Lingual Sentence Representationscited by 1
  11. 2025SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languagescited by 1
  12. 2026EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translationcited by 1