# SMOL: Professionally translated parallel data for 115 under-represented languages

arXiv: https://arxiv.org/abs/2502.12301 · HTML: https://mlsanity.com/paper/2502.12301/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 23 of 23)

- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges](https://mlsanity.com/paper/1907.05019/) (2019)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [PMIndia -- A Collection of Parallel Corpora of Languages of India](https://mlsanity.com/paper/2001.09907/) (2020)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks](https://mlsanity.com/paper/2210.14712/) (2022)
- [A Large-Scale Study of Machine Translation in the Turkic Languages](https://mlsanity.com/paper/2109.04593/) (2021)
- [FFR v1.1: Fon-French Neural Machine Translation](https://mlsanity.com/paper/2006.09217/) (2020)
- [Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedi](https://mlsanity.com/paper/2003.04986/) (2020)
- [Igbo-English Machine Translation: An Evaluation Benchmark](https://mlsanity.com/paper/2004.00648/) (2020)
- [Extended Parallel Corpus for Amharic-English Machine Translation](https://mlsanity.com/paper/2104.03543/) (2021)
- [Evaluating Multiway Multilingual NMT in the Turkic Languages](https://mlsanity.com/paper/2109.06262/) (2021)
- [English-Twi Parallel Corpus for Machine Translation](https://mlsanity.com/paper/2103.15625/) (2021)
- [Towards a parallel corpus of Portuguese and the Bantu language Emakhuwa of Mozambique](https://mlsanity.com/paper/2104.05753/) (2021)
- [English2Gbe: A multilingual machine translation model for {Fon/Ewe}Gbe](https://mlsanity.com/paper/2112.11482/) (2021)
- [Domain-specific MT for Low-resource Languages: The case of Bambara-French](https://mlsanity.com/paper/2104.00041/) (2021)
- [Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in text](https://mlsanity.com/paper/2311.06440/) (2023)

## Built on by (most cited first, 12 of 12)

- [Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languages](https://mlsanity.com/paper/2511.06531/) (2025)
- [The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation](https://mlsanity.com/paper/2505.13090/) (2025)
- [TransBench: Benchmarking Machine Translation for Industrial-Scale Applications](https://mlsanity.com/paper/2505.14244/) (2025)
- [TULUN: Transparent and Adaptable Low-resource Machine Translation](https://mlsanity.com/paper/2505.18683/) (2025)
- [Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG](https://mlsanity.com/paper/2601.09982/) (2026)
- [BYOL: Bring Your Own Language Into LLMs](https://mlsanity.com/paper/2601.10804/) (2026)
- [AfroScope: A Framework for Studying the Linguistic Landscape of Africa](https://mlsanity.com/paper/2601.13346/) (2026)
- [Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction](https://mlsanity.com/paper/2604.07320/) (2026)
- [Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast India](https://mlsanity.com/paper/2604.19778/) (2026)
- [BM25-Augmented Many-Shot Translation for Low-Resource North-Eastern Indian Languages](https://mlsanity.com/paper/2608.13722/) (2026)
- [TranslatePsy-AfriSLM: High-Quality Data Scaling For Low-Resource Machine Translation](https://mlsanity.com/paper/2608.18655/) (2026)
- [MixLID: Token-Level Language Identification in Mixed-Language Text](https://mlsanity.com/paper/2610.16601/) (2026)

## Abstract (arXiv)

We open-source SMOL (Set of Maximal Overall Leverage), a suite of training data to unlock machine translation for low-resource languages. SMOL has been translated into 124 (and growing) under-resourced languages (125 language pairs), including many for which there exist no previous public resources, for a total of 6.1M translated tokens. SMOL comprises two sub-datasets, each carefully chosen for maximum impact given its size: SMOLSENT, a set of sentences chosen for broad unique token coverage, and SMOLDOC, a document-level resource focusing on a broad topic coverage. They join the already released GATITOS for a trifecta of paragraph, sentence, and token-level content. We demonstrate that using SMOL to prompt or fine-tune Large Language Models yields robust chrF improvements. In addition to translation, we provide factuality ratings and rationales for all documents in SMOLDOC, yielding the first factuality datasets for most of these languages.