# Predicting Machine Translation Performance on Low-Resource Languages: The Role of Domain Similarity

arXiv: https://arxiv.org/abs/2402.02633 · HTML: https://mlsanity.com/paper/2402.02633/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2024.

## Built on by (most cited first, 15 of 15)

- [IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models](https://mlsanity.com/paper/2406.03368/) (2024)
- [URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base](https://mlsanity.com/paper/2409.18472/) (2024)
- [Low-resource Machine Translation: what for? who for? An observational study on a dedicated Tetun language translation service](https://mlsanity.com/paper/2411.12262/) (2024)
- [Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+](https://mlsanity.com/paper/2510.19217/) (2025)
- [Exploiting Domain-Specific Parallel Data on Multilingual Language Models for Low-resource Language Translation](https://mlsanity.com/paper/2412.19522/) (2024)
- [Zero-Shot Performance Prediction for Probabilistic Scaling Laws](https://mlsanity.com/paper/2510.16743/) (2025)
- [Simple Additions, Substantial Gains: Expanding Scripts, Languages, and Lineage Coverage in URIEL+](https://mlsanity.com/paper/2510.27183/) (2025)
- [Dialectal and Low-Resource Machine Translation for Aromanian](https://mlsanity.com/paper/2410.17728/) (2024)
- [Leveraging the Cross-Domain & Cross-Linguistic Corpus for Low Resource NMT: A Case Study On Bhili-Hindi-English Parallel Corpus](https://mlsanity.com/paper/2511.00486/) (2025)
- [EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation](https://mlsanity.com/paper/2609.29511/) (2026)
- [Generating Completions for Broca's Aphasic Sentences Using Large Language Models](https://mlsanity.com/paper/2412.17669/) (2024)
- [TULUN: Transparent and Adaptable Low-resource Machine Translation](https://mlsanity.com/paper/2505.18683/) (2025)
- [Can We Reliably Rank Model Performance across Domains without Labeled Data?](https://mlsanity.com/paper/2510.09519/) (2025)
- [Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG](https://mlsanity.com/paper/2601.09982/) (2026)
- [Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation](https://mlsanity.com/paper/2605.07533/) (2026)

## Abstract (arXiv)

Fine-tuning and testing a multilingual large language model is expensive and challenging for low-resource languages (LRLs). While previous studies have predicted the performance of natural language processing (NLP) tasks using machine learning methods, they primarily focus on high-resource languages, overlooking LRLs and shifts across domains. Focusing on LRLs, we investigate three factors: the size of the fine-tuning corpus, the domain similarity between fine-tuning and testing corpora, and the language similarity between source and target languages. We employ classical regression models to assess how these factors impact the model's performance. Our results indicate that domain similarity has the most critical impact on predicting the performance of Machine Translation models.