mlsanityData to 7 Oct 2026

MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering

arXiv 2007.15207 ↗First posted 30 Jul 2020cs.CLcited by 151

Not on the map. Why?

Abstract (arXiv)

Progress in cross-lingual modeling depends on challenging, realistic, and diverse evaluation sets. We introduce Multilingual Knowledge Questions and Answers (MKQA), an open-domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). Answers are based on a heavily curated, language-independent data representation, making results comparable across languages and independent of language-specific passages. With 26 languages, this dataset supplies the widest range of languages to-date for evaluating question answering. We benchmark a variety of state-of-the-art methods and baselines for generative and extractive question answering, trained on Natural Questions, in zero shot and translation settings. Results indicate this dataset is challenging even in English, but especially in low-resource languages

Key facts. Cited by 151 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 15; 2023: 25; 2024: 31; 2025: 41; 2026: 23.

151 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  3. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  4. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  5. 2022RealTime QA: What's the Answer Right Now?cited by 239
  6. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  7. 2023Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Promptingcited by 163
  8. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  9. 2023Document Understanding Dataset and Evaluation (DUDE)cited by 132
  10. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  11. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  12. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
38 more
  1. 2024InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modelingcited by 76
  2. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  3. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  4. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  5. 2022Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languagescited by 47
  6. 2021One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrievalcited by 46
  7. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  8. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  9. 2024SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languagescited by 41
  10. 2025Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMscited by 39
  11. 2021SD-QA: Spoken Dialectal Question Answering for the Real Worldcited by 36
  12. 2024Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languagescited by 36
  13. 2024Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Modelscited by 31
  14. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  15. 2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
  16. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  17. 2023Revisiting non-English Text Simplification: A Unified Multilingual Benchmarkcited by 27
  18. 2024Retrieval-augmented generation in multilingual settingscited by 27
  19. 2021English Machine Reading Comprehension Datasets: A Surveycited by 26
  20. 2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24
  21. 2023AfriQA: Cross-lingual Open-Retrieval Question Answering for African Languagescited by 23
  22. 2024SciQAG: A Framework for Auto-Generated Science Question Answering Dataset with Fine-grained Evaluationcited by 23
  23. 2025jina-reranker-v3: Last but Not Late Interaction for Listwise Document Rerankingcited by 23
  24. 2021MTG: A Benchmark Suite for Multilingual Text Generationcited by 22
  25. 2025Investigating Language Preference of Multilingual RAG Systemscited by 22
  26. 2022TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languagescited by 21
  27. 2023Evaluating and Modeling Attribution for Cross-Lingual Question Answeringcited by 21
  28. 2021Zero-Shot Cross-lingual Semantic Parsingcited by 19
  29. 2023Sabi'a: Portuguese Large Language Modelscited by 19
  30. 2021Learning Cross-Lingual IR from an English Retrievercited by 16
  31. 2023ScandEval: A Benchmark for Scandinavian Natural Language Processingcited by 16
  32. 2023VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Modelscited by 15
  33. 2023BLUEX: A benchmark based on Brazilian Leading Universities Entrance eXamscited by 15
  34. 2024Teaching LLMs to Abstain across Languages via Multilingual Feedbackcited by 14
  35. 2022Evaluation of Transfer Learning for Polish with a Text-to-Text Modelcited by 13
  36. 2025Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Taskcited by 12
  37. 2025Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkishcited by 12
  38. 2022MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languagescited by 11

50 most cited of 151 All 151 (JSON)