# Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages

arXiv: https://arxiv.org/abs/2210.09984 · HTML: https://mlsanity.com/paper/2210.09984/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization](https://mlsanity.com/paper/2003.11080/) (2020)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [Document Ranking with a Pretrained Sequence-to-Sequence Model](https://mlsanity.com/paper/2003.06713/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering](https://mlsanity.com/paper/2007.15207/) (2020)
- [XOR QA: Cross-lingual Open-Retrieval Question Answering](https://mlsanity.com/paper/2010.11856/) (2020)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval](https://mlsanity.com/paper/2203.05765/) (2022)
- [Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models](https://mlsanity.com/paper/2201.08471/) (2022)
- [MS MARCO: Benchmarking Ranking Models in the Large-Data Regime](https://mlsanity.com/paper/2105.04021/) (2021)
- [Towards Best Practices for Training Multilingual Dense Retrieval Models](https://mlsanity.com/paper/2204.02363/) (2022)
- [Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning](https://mlsanity.com/paper/1912.13080/) (2019)

## Built on by (most cited first, 30 of 47)

- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [Zero-Shot Listwise Document Reranking with a Large Language Model](https://mlsanity.com/paper/2305.02156/) (2023)
- [HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution](https://mlsanity.com/paper/2307.16883/) (2023)
- [GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation](https://mlsanity.com/paper/2502.01113/) (2025)
- [Training Sparse Mixture Of Experts Text Embedding Models](https://mlsanity.com/paper/2502.07972/) (2025)
- [Rethinking the Role of Token Retrieval in Multi-Vector Retrieval](https://mlsanity.com/paper/2304.01982/) (2023)
- [AttributionBench: How Hard is Automatic Attribution Evaluation?](https://mlsanity.com/paper/2402.15089/) (2024)
- [AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languages](https://mlsanity.com/paper/2305.06897/) (2023)
- [The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design](https://mlsanity.com/paper/2408.12503/) (2024)
- [Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?](https://mlsanity.com/paper/2409.02727/) (2024)
- [TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documents](https://mlsanity.com/paper/2312.01279/) (2023)
- [Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions](https://mlsanity.com/paper/2407.20243/) (2024)
- [Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Task](https://mlsanity.com/paper/2504.03616/) (2025)
- [Evaluating Embedding APIs for Information Retrieval](https://mlsanity.com/paper/2305.06300/) (2023)
- [Language Models are Universal Embedders](https://mlsanity.com/paper/2310.08232/) (2023)
- [Search-Adaptor: Embedding Customization for Information Retrieval](https://mlsanity.com/paper/2310.08750/) (2023)
- [Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings](https://mlsanity.com/paper/2506.01435/) (2025)
- [Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks](https://mlsanity.com/paper/2411.01192/) (2024)
- [DAPR: A Benchmark on Document-Aware Passage Retrieval](https://mlsanity.com/paper/2305.13915/) (2023)
- [MegaWika: Millions of reports and their sources across 50 diverse languages](https://mlsanity.com/paper/2307.07049/) (2023)
- [Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models](https://mlsanity.com/paper/2408.10151/) (2024)
- [KazQAD: Kazakh Open-Domain Question Answering Dataset](https://mlsanity.com/paper/2404.04487/) (2024)
- [Alloprof: a new French question-answer education dataset and its use in an information retrieval case study](https://mlsanity.com/paper/2302.07738/) (2023)
- [AToMiC: An Image/Text Retrieval Test Collection to Support Multimedia Content Creation](https://mlsanity.com/paper/2304.01961/) (2023)
- [IndicIRSuite: Multilingual Dataset and Neural Information Models for Indian Languages](https://mlsanity.com/paper/2312.09508/) (2023)
- [Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challenges](https://mlsanity.com/paper/2406.06938/) (2024)
- [Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentations](https://mlsanity.com/paper/2504.04771/) (2025)
- [M3Retrieve: Benchmarking Multimodal Retrieval for Medicine](https://mlsanity.com/paper/2510.06888/) (2025)
- [RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets](https://mlsanity.com/paper/2511.01386/) (2025)
- [Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking](https://mlsanity.com/paper/2212.01340/) (2022)

## Abstract (arXiv)

MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval across 18 different languages, which collectively encompass over three billion native speakers around the world. These languages have diverse typologies, originate from many different language families, and are associated with varying amounts of available resources -- including what researchers typically characterize as high-resource as well as low-resource languages. Our dataset is designed to support the creation and evaluation of models for monolingual retrieval, where the queries and the corpora are in the same language. In total, we have gathered over 700k high-quality relevance judgments for around 77k queries over Wikipedia in these 18 languages, where all assessments have been performed by native speakers hired by our team. Our goal is to spur research that will improve retrieval across a continuum of languages, thus enhancing information access capabilities for diverse populations around the world, particularly those that have been traditionally underserved. This overview paper describes the dataset and baselines that we share with the community. The MIRACL website is live at http://miracl.ai/.