mlsanityData to 7 Oct 2026

MIA 2022 Shared Task: Evaluating Cross-lingual Open-Retrieval Question Answering for 16 Diverse Languages

arXiv 2207.00758 ↗First posted 2 Jul 2022cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

We present the results of the Workshop on Multilingual Information Access (MIA) 2022 Shared Task, evaluating cross-lingual open-retrieval question answering (QA) systems in 16 typologically diverse languages. In this task, we adapted two large-scale cross-lingual open-retrieval QA datasets in 14 typologically diverse languages, and newly annotated open-retrieval QA data in 2 underrepresented languages: Tagalog and Tamil. Four teams submitted their systems. The best system leveraging iteratively mined diverse negative examples and larger pretrained models achieves 32.2 F1, outperforming our baseline by 4.5 points. The second best system uses entity-aware contextualized representations for document retrieval, and achieves significant improvements in Tamil (20.8 F1), whereas most of the other systems yield nearly zero scores.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 2 Jul 2022.

Builds on

37 of 47 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  7. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
  10. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  11. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  12. 2020Language-agnostic BERT Sentence Embeddingcited by 621
25 more
  1. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  2. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  3. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  4. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  5. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 339
  6. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  7. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 151
  8. 2021SituatedQA: Incorporating Extra-Linguistic Contexts into QAcited by 145
  9. 2020Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasetscited by 137
  10. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  11. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  12. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  13. 2018A Span-Extraction Dataset for Chinese Machine Reading Comprehensioncited by 84
  14. 2020FQuAD: French Question Answering Datasetcited by 58
  15. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  16. 2021One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrievalcited by 46
  17. 2020Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrievalcited by 44
  18. 2021Evidentiality-guided Generation for Knowledge-Intensive NLP Taskscited by 40
  19. 2022SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samplescited by 39
  20. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  21. 2019EvalAI: Towards Better Evaluation Systems for AI Agentscited by 34
  22. 2021mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Modelscited by 19
  23. 2022A Feasibility Study of Answer-Agnostic Question Generation for Educationcited by 6
  24. 2022Investigating Information Inconsistency in Multilingual Open-Domain Question Answeringcited by 1
  25. 2022ZusammenQA: Data Augmentation with Specialized Models for Cross-lingual Open-retrieval Question Answering Systemcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 5; 2025: 2.

11 papers.

  1. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  2. 2022RealTime QA: What's the Answer Right Now?cited by 239
  3. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  4. 2024Retrieval-augmented generation in multilingual settingscited by 27
  5. 2022Beyond Counting Datasets: A Survey of Multilingual Dataset Construction and Necessary Resourcescited by 25
  6. 2023Evaluating and Modeling Attribution for Cross-Lingual Question Answeringcited by 21
  7. 2024Multilingual Retrieval Augmented Generation for Culturally-Sensitive Tasks: A Benchmark for Cross-lingual Robustnesscited by 13
  8. 2025Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAGcited by 7
  9. 2025A survey of diversity quantification in natural language processing: The why, what, where and howcited by 4
  10. 2024What is "Typological Diversity" in NLP?cited by 3
  11. 2024sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Promptingcited by 1