mlsanityData to 7 Oct 2026

Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages

arXiv 2210.09984 ↗First posted 18 Oct 2022cs.IR cs.CLcited by 47

Not on the map. Why?

Abstract (arXiv)

MIRACL (Multilingual Information Retrieval Across a Continuum of Languages) is a multilingual dataset we have built for the WSDM 2023 Cup challenge that focuses on ad hoc retrieval across 18 different languages, which collectively encompass over three billion native speakers around the world. These languages have diverse typologies, originate from many different language families, and are associated with varying amounts of available resources -- including what researchers typically characterize as high-resource as well as low-resource languages. Our dataset is designed to support the creation and evaluation of models for monolingual retrieval, where the queries and the corpora are in the same language. In total, we have gathered over 700k high-quality relevance judgments for around 77k queries over Wikipedia in these 18 languages, where all assessments have been performed by native speakers hired by our team. Our goal is to spur research that will improve retrieval across a continuum of languages, thus enhancing information access capabilities for diverse populations around the world, particularly those that have been traditionally underserved. This overview paper describes the dataset and baselines that we share with the community. The MIRACL website is live at http://miracl.ai/.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

22 of 29 references.

References · 22

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  9. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  10. 2019Passage Re-ranking with BERTcited by 629
  11. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  12. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
10 more
  1. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  2. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
  3. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  4. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  5. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  6. 2022Tevatron: An Efficient and Flexible Toolkit for Dense Retrievalcited by 47
  7. 2022Transfer Learning Approaches for Building Cross-Language Dense Retrieval Modelscited by 36
  8. 2021MS MARCO: Benchmarking Ranking Models in the Large-Data Regimecited by 28
  9. 2022Towards Best Practices for Training Multilingual Dense Retrieval Modelscited by 26
  10. 2019Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learningcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 15; 2024: 16; 2025: 10; 2026: 5.

47 papers.

Sort:

By area:

  1. 2024Generative Representational Instruction Tuningcited by 227
  2. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  3. 2023HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attributioncited by 55
  4. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  5. 2025Training Sparse Mixture Of Experts Text Embedding Modelscited by 32
  6. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  7. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  8. 2023AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languagescited by 23
  9. 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model designcited by 20
  10. 2024Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?cited by 19
  11. 2023TextGenSHAP: Scalable Post-hoc Explanations in Text Generation with Long Documentscited by 17
  12. 2024Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensionscited by 16
35 more
  1. 2025Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Taskcited by 12
  2. 2023Evaluating Embedding APIs for Information Retrievalcited by 11
  3. 2023Language Models are Universal Embedderscited by 11
  4. 2023Search-Adaptor: Embedding Customization for Information Retrievalcited by 10
  5. 2024Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarkscited by 10
  6. 2025Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddingscited by 10
  7. 2023DAPR: A Benchmark on Document-Aware Passage Retrievalcited by 9
  8. 2023MegaWika: Millions of reports and their sources across 50 diverse languagescited by 9
  9. 2024Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Modelscited by 9
  10. 2024KazQAD: Kazakh Open-Domain Question Answering Datasetcited by 7
  11. 2023Alloprof: a new French question-answer education dataset and its use in an information retrieval case studycited by 6
  12. 2023AToMiC: An Image/Text Retrieval Test Collection to Support Multimedia Content Creationcited by 5
  13. 2023IndicIRSuite: Multilingual Dataset and Neural Information Models for Indian Languagescited by 5
  14. 2024Post-Hoc Answer Attribution for Grounded and Trustworthy Long Document Comprehension: Task, Insights, and Challengescited by 5
  15. 2025M3Retrieve: Benchmarking Multimodal Retrieval for Medicinecited by 5
  16. 2022Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarkingcited by 4
  17. 2024Building pre-train LLM Dataset for the INDIC Languages: a case study on Hindicited by 4
  18. 2025Improving Multilingual Retrieval-Augmented Language Models through Dialectic Reasoning Argumentationscited by 4
  19. 2025Search-R3: Unifying Reasoning and Embedding in Large Language Modelscited by 4
  20. 2025RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasetscited by 4
  21. 2023Spacerini: Plug-and-play Search Engines with Pyserini and Hugging Facecited by 3
  22. 2024IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarioscited by 3
  23. 2025MegaWika 2: A More Comprehensive Multilingual Collection of Articles and their Sourcescited by 3
  24. 2024Hindi-BEIR : A Large Scale Retrieval Benchmark in Hindicited by 2
  25. 2024Benchmarking and Building Zero-Shot Hindi Retrieval Model with Hindi-BEIR and NLLB-E5cited by 2
  26. 2024Ruri: Japanese General Text Embeddingscited by 2
  27. 2024Synergistic Approach for Simultaneous Optimization of Monolingual, Cross-lingual, and Multilingual Information Retrievalcited by 1
  28. 2023Enhancing Model Performance in Multilingual Information Retrieval with Comprehensive Data Engineering Techniquescited by 0
  29. 2024Efficient fine-tuning methodology of text embedding models for information retrieval: contrastive learning penalty (clp)cited by 0
  30. 2025A model and package for German ColBERTcited by 0
  31. 2026One Instruction Does Not Fit All: How Well Do Embeddings Align Personas and Instructions in Low-Resource Indian Languages?cited by 0
  32. 2026Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documentscited by 0
  33. 2026MMLongEmbed: Benchmarking Multimodal Embedding Models in Long-Context Scenarioscited by 0
  34. 2026HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditionscited by 0
  35. 2026LAMAR: An Open Language-Aware Multilingual Alignment Rerankercited by 0