mlsanityData to 14 Oct 2026

SMOL: Professionally translated parallel data for 115 under-represented languages

arXiv 2502.12301 ↗First posted 17 Feb 2025cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

We open-source SMOL (Set of Maximal Overall Leverage), a suite of training data to unlock machine translation for low-resource languages. SMOL has been translated into 124 (and growing) under-resourced languages (125 language pairs), including many for which there exist no previous public resources, for a total of 6.1M translated tokens. SMOL comprises two sub-datasets, each carefully chosen for maximum impact given its size: SMOLSENT, a set of sentences chosen for broad unique token coverage, and SMOLDOC, a document-level resource focusing on a broad topic coverage. They join the already released GATITOS for a trifecta of paragraph, sentence, and token-level content. We demonstrate that using SMOL to prompt or fine-tune Large Language Models yields robust chrF improvements. In addition to translation, we provide factuality ratings and rationales for all documents in SMOLDOC, yielding the first factuality datasets for most of these languages.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 23 of its 51 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

23 of 51 references.

References · 23

  1. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  3. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  4. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  5. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  6. 2023MADLAD-400: A Multilingual And Document-Level Large Audited Datasetcited by 181
  7. 2020The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MTcited by 128
  8. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  9. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  10. 2020PMIndia -- A Collection of Parallel Corpora of Languages of Indiacited by 42
  11. 2021The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translationcited by 38
  12. 2022Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Taskscited by 28
11 more
  1. 2021A Large-Scale Study of Machine Translation in the Turkic Languagescited by 21
  2. 2020FFR v1.1: Fon-French Neural Machine Translationcited by 16
  3. 2020Investigating an approach for low resource language dataset creation, curation and classification: Setswana and Sepedicited by 15
  4. 2020Igbo-English Machine Translation: An Evaluation Benchmarkcited by 14
  5. 2021Extended Parallel Corpus for Amharic-English Machine Translationcited by 10
  6. 2021Evaluating Multiway Multilingual NMT in the Turkic Languagescited by 8
  7. 2021English-Twi Parallel Corpus for Machine Translationcited by 7
  8. 2021Towards a parallel corpus of Portuguese and the Bantu language Emakhuwa of Mozambiquecited by 5
  9. 2021English2Gbe: A multilingual machine translation model for {Fon/Ewe}Gbecited by 5
  10. 2021Domain-specific MT for Low-resource Languages: The case of Bambara-Frenchcited by 4
  11. 2023Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in textcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025Ibom NLP: A Step Toward Inclusive Natural Language Processing for Nigeria's Minority Languagescited by 6
  2. 2025The Effect of Language Diversity When Fine-Tuning Large Language Models for Translationcited by 1
  3. 2025TransBench: Benchmarking Machine Translation for Industrial-Scale Applicationscited by 1
  4. 2025TULUN: Transparent and Adaptable Low-resource Machine Translationcited by 0
  5. 2026Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAGcited by 0
  6. 2026BYOL: Bring Your Own Language Into LLMscited by 0
  7. 2026AfroScope: A Framework for Studying the Linguistic Landscape of Africacited by 0
  8. 2026Evaluating In-Context Translation with Synchronous Context-Free Grammar Transductioncited by 0
  9. 2026Towards High-Quality Machine Translation for Kokborok: A Low-Resource Tibeto-Burman Language of Northeast Indiacited by 0
  10. 2026BM25-Augmented Many-Shot Translation for Low-Resource North-Eastern Indian Languagescited by 0
  11. 2026TranslatePsy-AfriSLM: High-Quality Data Scaling For Low-Resource Machine Translationcited by 0
  12. 2026MixLID: Token-Level Language Identification in Mixed-Language Textcited by 0