mlsanityData to 7 Oct 2026

EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answering

arXiv 2011.03080 ↗First posted 5 Nov 2020cs.CL cs.AI cs.IRcited by 60

Not on the map. Why?

Abstract (arXiv)

We propose EXAMS -- a new benchmark dataset for cross-lingual and multilingual question answering for high school examinations. We collected more than 24,000 high-quality high school exam questions in 16 languages, covering 8 language families and 24 school subjects from Natural Sciences and Social Sciences, among others. EXAMS offers a fine-grained evaluation framework across multiple languages and subjects, which allows precise analysis and comparison of various models. We perform various experiments with existing top-performing multilingual pre-trained models and we show that EXAMS offers multiple challenges that require multilingual knowledge and reasoning in multiple domains. We hope that EXAMS will enable researchers to explore challenging reasoning and knowledge transfer methods and pre-trained models for school question answering in various languages which was not possible before. The data, code, pre-trained models, and evaluation are available at https://github.com/mhardalov/exams-qa.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 11; 2024: 18; 2025: 16; 2026: 8.

60 papers.

Sort:

By area:

  1. 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
  2. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  3. 2023Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptationcited by 79
  4. 2024INCLUDE: Evaluating Multilingual Language Understanding with Regional Knowledgecited by 78
  5. 2023AceGPT, Localizing Large Language Models in Arabiccited by 75
  6. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  7. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  8. 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontierscited by 49
  9. 2021DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendationcited by 46
  10. 2024AraDiCE: Benchmarks for Dialectal and Cultural Capabilities in LLMscited by 46
  11. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Modelscited by 41
  12. 2023Revisiting Machine Translation for Cross-lingual Classificationcited by 40
38 more
  1. 2024Mitigating Catastrophic Forgetting in Language Transfer via Model Mergingcited by 36
  2. 2025Enhancing Multilingual LLM Pretraining with Model-Based Data Selectioncited by 33
  3. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  4. 2021CodeQA: A Question Answering Dataset for Source Code Comprehensioncited by 30
  5. 2023Dolphin: A Challenging and Diverse Benchmark for Arabic NLGcited by 22
  6. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 22
  7. 2025Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenizationcited by 18
  8. 2025Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLMcited by 16
  9. 2024SambaLingo: Teaching Large Language Models New Languagescited by 11
  10. 2023xPQA: Cross-Lingual Product Question Answering across 12 Languagescited by 10
  11. 2025Revisiting Multilingual Data Mixtures in Language Model Pretrainingcited by 10
  12. 2022Leaf: Multiple-Choice Question Generationcited by 9
  13. 2025Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gapscited by 9
  14. 2024BayLing 2: A Multilingual Large Language Model with Efficient Language Alignmentcited by 8
  15. 2023To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languagescited by 7
  16. 2024Alignment at Pre-training! Towards Native Alignment for Arabic LLMscited by 7
  17. 2025Cetvel: A Unified Benchmark for Evaluating Language Understanding, Generation and Cultural Capacity of LLMs for Turkishcited by 7
  18. 2021Self-Teaching Machines to Read and Comprehend with Large-Scale Multi-Subject Question-Answering Datacited by 6
  19. 2024Bilingual Adaptation of Monolingual Foundation Modelscited by 5
  20. 2025Hala Technical Report: Building Arabic-Centric Instruction & Translation Models at Scalecited by 5
  21. 2023bgGLUE: A Bulgarian General Language Understanding Evaluation Benchmarkcited by 4
  22. 2024The Invalsi Benchmarks: measuring Linguistic and Mathematical understanding of Large Language Models in Italiancited by 4
  23. 2024Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansioncited by 4
  24. 2021Generating Answer Candidates for Quizzes and Answer-Aware Question Generatorscited by 3
  25. 2024Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learningcited by 3
  26. 2024GemmAr: Enhancing LLMs Through Arabic Instruction-Tuningcited by 2
  27. 2024Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contaminationcited by 2
  28. 2025When Language Shapes Thought: Cross-Lingual Transfer of Factual Knowledge in Question Answeringcited by 2
  29. 2022Investigating Information Inconsistency in Multilingual Open-Domain Question Answeringcited by 1
  30. 2022Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matchingcited by 1
  31. 2023AMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inferencecited by 1
  32. 2024New Textual Corpora for Serbian Language Modelingcited by 1
  33. 2025Towards Multilingual LLM Evaluation for Baltic and Nordic languages: A study on Lithuanian Historycited by 1
  34. 2025Multilingual European Language Models: Benchmarking Approaches and Challengescited by 1
  35. 2025Statement-Tuning Enables Efficient Cross-lingual Generalization in Encoder-only Modelscited by 1
  36. 2025MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Modelscited by 1
  37. 2025AraLingBench A Human-Annotated Benchmark for Evaluating Arabic Linguistic Capabilities of Large Language Modelscited by 1
  38. 2026TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representationcited by 1

50 most cited of 60 All 60 (JSON)