mlsanityData to 7 Oct 2026

TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages

arXiv 2003.05002 ↗First posted 10 Mar 2020cs.CL cs.LGcited by 486

Not on the map. Why?

Abstract (arXiv)

Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations. We present TyDi QA---a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology---the set of linguistic features each language expresses---such that we expect models performing well on this set to generalize across a large number of the world's languages. We present a quantitative analysis of the data quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don't know the answer yet, and the data is collected directly in each language without the use of translation.

Key facts. Cited by 486 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 71; 2022: 78; 2023: 87; 2024: 113; 2025: 90; 2026: 47.

486 papers.

Sort:

By area:

  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2023A Survey of Large Language Modelscited by 2,537
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  6. 2023PaLM 2 Technical Reportcited by 1,154
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  9. 2023A Comprehensive Overview of Large Language Modelscited by 639
  10. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  11. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  12. 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
38 more
  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  2. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  3. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  4. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  5. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  6. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  7. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  8. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  9. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  10. 2022RealTime QA: What's the Answer Right Now?cited by 239
  11. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  12. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  13. 2024Generative Representational Instruction Tuningcited by 227
  14. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
  15. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  16. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  17. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  18. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  19. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  20. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  21. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
  22. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  23. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  24. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  25. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  26. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  27. 2021BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Banglacited by 123
  28. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  29. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  30. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  31. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  32. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 115
  33. 2021Sharpness-Aware Minimization Improves Language Model Generalizationcited by 108
  34. 2022Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Modelscited by 108
  35. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  36. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  37. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
  38. 2024ContextCite: Attributing Model Generation to Contextcited by 93

50 most cited of 486 All 486 (JSON)