# WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia

arXiv: https://arxiv.org/abs/1907.05791 · HTML: https://mlsanity.com/paper/1907.05791/

## Key facts

- Cited by 223 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2019.

## Built on by (most cited first, 30 of 223)

- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus](https://mlsanity.com/paper/2104.08758/) (2021)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Relative representations enable zero-shot latent space communication](https://mlsanity.com/paper/2209.15430/) (2022)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages](https://mlsanity.com/paper/2309.09400/) (2023)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation](https://mlsanity.com/paper/2205.02022/) (2022)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders](https://mlsanity.com/paper/2104.08027/) (2021)
- [XLM-E: Cross-lingual Language Model Pre-training via ELECTRA](https://mlsanity.com/paper/2106.16138/) (2021)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Extrapolating Large Language Models to Non-English by Aligning Languages](https://mlsanity.com/paper/2308.04948/) (2023)
- [NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages](https://mlsanity.com/paper/2205.15960/) (2022)
- [LLMs Are Few-Shot In-Context Low-Resource Language Learners](https://mlsanity.com/paper/2403.16512/) (2024)
- [One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia](https://mlsanity.com/paper/2203.13357/) (2022)
- [Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions](https://mlsanity.com/paper/2305.15083/) (2023)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [MURAL: Multimodal, Multitask Retrieval Across Languages](https://mlsanity.com/paper/2109.05125/) (2021)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs](https://mlsanity.com/paper/2104.08692/) (2021)
- [Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment](https://mlsanity.com/paper/2106.06381/) (2021)
- [AfroLID: A Neural Language Identification Tool for African Languages](https://mlsanity.com/paper/2210.11744/) (2022)
- [The Effect of Domain and Diacritics in Yor`ub'a-English Neural Machine Translation](https://mlsanity.com/paper/2103.08647/) (2021)
- [A Survey on Cross-Lingual Summarization](https://mlsanity.com/paper/2203.12515/) (2022)

## Abstract (arXiv)

We present an approach based on multilingual sentence embeddings to automatically extract parallel sentences from the content of Wikipedia articles in 85 languages, including several dialects or low-resource languages. We do not limit the the extraction process to alignments with English, but systematically consider all possible language pairs. In total, we are able to extract 135M parallel sentences for 1620 different language pairs, out of which only 34M are aligned with English. This corpus of parallel sentences is freely available at https://github.com/facebookresearch/LASER/tree/master/tasks/WikiMatrix. To get an indication on the quality of the extracted bitexts, we train neural MT baseline systems on the mined data only for 1886 languages pairs, and evaluate them on the TED corpus, achieving strong BLEU scores for many language pairs. The WikiMatrix bitexts seem to be particularly interesting to train MT systems between distant languages without the need to pivot through English.