# The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT

arXiv: https://arxiv.org/abs/2010.06354 · HTML: https://mlsanity.com/paper/2010.06354/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 30 of 128)

- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task](https://mlsanity.com/paper/2304.01097/) (2023)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://mlsanity.com/paper/2302.13942/) (2023)
- [The paradox of the compositionality of natural language: a neural machine translation case study](https://mlsanity.com/paper/2108.05885/) (2021)
- [Democratizing Neural Machine Translation with OPUS-MT](https://mlsanity.com/paper/2212.01936/) (2022)
- [Adaptive Machine Translation with Large Language Models](https://mlsanity.com/paper/2301.13294/) (2023)
- [Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation](https://mlsanity.com/paper/2105.08504/) (2021)
- [Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages](https://mlsanity.com/paper/2404.11553/) (2024)
- [Frustratingly Easy Label Projection for Cross-lingual Transfer](https://mlsanity.com/paper/2211.15613/) (2022)
- [Quantifying the Dialect Gap and its Correlates Across Languages](https://mlsanity.com/paper/2310.15135/) (2023)
- [When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages](https://mlsanity.com/paper/2311.09205/) (2023)
- [Many-to-English Machine Translation Tools, Data, and Pretrained Models](https://mlsanity.com/paper/2104.00290/) (2021)
- [How BPE Affects Memorization in Transformers](https://mlsanity.com/paper/2110.02782/) (2021)
- [Goldfish: Monolingual Language Models for 350 Languages](https://mlsanity.com/paper/2408.10441/) (2024)
- [Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs](https://mlsanity.com/paper/2502.14830/) (2025)
- [Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2311.08380/) (2023)
- [A Large-Scale Study of Machine Translation in the Turkic Languages](https://mlsanity.com/paper/2109.04593/) (2021)
- [Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data](https://mlsanity.com/paper/2212.09171/) (2022)
- [Poro 34B and the Blessing of Multilinguality](https://mlsanity.com/paper/2404.01856/) (2024)
- [Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction](https://mlsanity.com/paper/2109.06798/) (2021)
- [SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages](https://mlsanity.com/paper/2210.11621/) (2022)
- [EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models](https://mlsanity.com/paper/2409.17892/) (2024)
- [An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT)](https://mlsanity.com/paper/2503.10267/) (2025)
- [Low-Rank Softmax Can Have Unargmaxable Classes in Theory but Rarely in Practice](https://mlsanity.com/paper/2203.06462/) (2022)
- [Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification](https://mlsanity.com/paper/2311.13937/) (2023)
- [Endangered Languages are not Low-Resourced!](https://mlsanity.com/paper/2103.09567/) (2021)

## Abstract (arXiv)

This paper describes the development of a new benchmark for machine translation that provides training and test data for thousands of language pairs covering over 500 languages and tools for creating state-of-the-art translation models from that collection. The main goal is to trigger the development of open translation tools and models with a much broader coverage of the World's languages. Using the package it is possible to work on realistic low-resource scenarios avoiding artificially reduced setups that are common when demonstrating zero-shot or few-shot learning. For the first time, this package provides a comprehensive collection of diverse data sets in hundreds of languages with systematic language and script annotation and data splits to extend the narrow coverage of existing benchmarks. Together with the data release, we also provide a growing number of pre-trained baseline models for individual language pairs and selected language groups.