# The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English

arXiv: https://arxiv.org/abs/1902.01382 · HTML: https://mlsanity.com/paper/1902.01382/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2019.

## Built on by (most cited first, 30 of 62)

- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages](https://mlsanity.com/paper/2205.15960/) (2022)
- [From LLM to NMT: Advancing Low-Resource Machine Translation with Claude](https://mlsanity.com/paper/2404.13813/) (2024)
- [Some Languages are More Equal than Others: Probing Deeper into the Linguistic Disparity in the NLP World](https://mlsanity.com/paper/2210.08523/) (2022)
- [AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages](https://mlsanity.com/paper/2104.08726/) (2021)
- [MURAL: Multimodal, Multitask Retrieval Across Languages](https://mlsanity.com/paper/2109.05125/) (2021)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [The Amazon Nova Family of Models: Technical Report and Model Card](https://mlsanity.com/paper/2506.12103/) (2025)
- [Multilingual Mathematical Autoformalization](https://mlsanity.com/paper/2311.03755/) (2023)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2105.03953/) (2021)
- [The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results](https://mlsanity.com/paper/2110.04392/) (2021)
- [AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages](https://mlsanity.com/paper/2109.04715/) (2021)
- [A Large-Scale Study of Machine Translation in the Turkic Languages](https://mlsanity.com/paper/2109.04593/) (2021)
- [DEEP: DEnoising Entity Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2111.07393/) (2021)
- [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2103.10531/) (2021)
- [Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN](https://mlsanity.com/paper/2107.01366/) (2021)
- [Alternative Input Signals Ease Transfer in Multilingual Machine Translation](https://mlsanity.com/paper/2110.07804/) (2021)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Consistent Human Evaluation of Machine Translation across Language Pairs](https://mlsanity.com/paper/2205.08533/) (2022)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture](https://mlsanity.com/paper/2509.19274/) (2025)
- [WYWEB: A NLP Evaluation Benchmark For Classical Chinese](https://mlsanity.com/paper/2305.14150/) (2023)
- [Multilingual Representation Distillation with Contrastive Learning](https://mlsanity.com/paper/2210.05033/) (2022)
- [FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation](https://mlsanity.com/paper/2408.13585/) (2024)

## Abstract (arXiv)

For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available. Besides the technical challenges of learning with limited supervision, it is difficult to evaluate methods trained on low-resource language pairs because of the lack of freely and publicly available benchmarks. In this work, we introduce the FLoRes evaluation datasets for Nepali-English and Sinhala-English, based on sentences translated from Wikipedia. Compared to English, these are languages with very different morphology and syntax, for which little out-of-domain parallel data is available and for which relatively large amounts of monolingual data are freely available. We describe our process to collect and cross-check the quality of translations, and we report baseline performance using several learning settings: fully supervised, weakly supervised, semi-supervised, and fully unsupervised. Our experiments demonstrate that current state-of-the-art methods perform rather poorly on this benchmark, posing a challenge to the research community working on low-resource MT. Data and code to reproduce our experiments are available at https://github.com/facebookresearch/flores.