mlsanityData to 7 Oct 2026

MLQA: Evaluating Cross-lingual Extractive Question Answering

arXiv 1910.07475 ↗First posted 16 Oct 2019cs.CL cs.AI cs.LGcited by 339

Not on the map. Why?

Abstract (arXiv)

Question answering (QA) models have shown rapid progress enabled by the availability of large, high-quality benchmark datasets. Such annotated datasets are difficult and costly to collect, and rarely exist in languages other than English, making training QA systems in other languages challenging. An alternative to building large monolingual training datasets is to develop cross-lingual systems which can transfer to a target language without requiring training data in that language. In order to develop such systems, it is crucial to invest in high quality multilingual evaluation benchmarks to measure progress. We present MLQA, a multi-way aligned extractive QA evaluation benchmark intended to spur research in this area. MLQA contains QA instances in 7 languages, namely English, Arabic, German, Spanish, Hindi, Vietnamese and Simplified Chinese. It consists of over 12K QA instances in English and 5K in each other language, with each QA instance being parallel between 4 languages on average. MLQA is built using a novel alignment context strategy on Wikipedia articles, and serves as a cross-lingual extension to existing extractive QA datasets. We evaluate current state-of-the-art cross-lingual representations on MLQA, and also provide machine-translation-based baselines. In all cases, transfer results are shown to be significantly behind training-language performance.

Key facts. Cited by 339 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 65; 2022: 44; 2023: 77; 2024: 62; 2025: 55; 2026: 36.

339 papers.

Sort:

By area:

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  5. 2023Tool Learning with Foundation Modelscited by 302
  6. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  7. 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
  8. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  9. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  10. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  11. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  12. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
38 more
  1. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 156
  2. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 149
  3. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  4. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  5. 2021Composable Sparse Fine-Tuning for Cross-Lingual Transfercited by 144
  6. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  7. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  8. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
  9. 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
  10. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  11. 2021QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehensioncited by 91
  12. 2023XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Modelscited by 88
  13. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  14. 2021Larger-Scale Transformers for Multilingual Masked Language Modelingcited by 86
  15. 2023Spanish Pre-trained BERT Model and Evaluation Datacited by 81
  16. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  17. 2022Mintaka: A Complex, Natural, and Multilingual Dataset for End-to-End Question Answeringcited by 71
  18. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  19. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  20. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  21. 2021AraT5: Text-to-Text Transformers for Arabic Language Generationcited by 63
  22. 2023Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinationscited by 63
  23. 2021Few-Shot Question Answering by Pretraining Span Selectioncited by 62
  24. 2022Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generationcited by 62
  25. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  26. 2021A Primer on Pretrained Multilingual Language Modelscited by 49
  27. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  28. 2021DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendationcited by 46
  29. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  30. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  31. 2023Improving Language Plasticity via Pretraining with Active Forgettingcited by 43
  32. 2022IndicNLG Benchmark: Multilingual Datasets for Diverse NLG Tasks in Indic Languagescited by 42
  33. 2023ToolTalk: Evaluating Tool-Usage in a Conversational Settingcited by 42
  34. 2021Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Modelscited by 41
  35. 2021MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairscited by 41
  36. 2021Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignmentcited by 41
  37. 2021Consistency Regularization for Cross-Lingual Fine-Tuningcited by 41
  38. 2021Multi-view Subword Regularizationcited by 40

50 most cited of 339 All 339 (JSON)