mlsanityData to 14 Oct 2026

EuraGovExam: A Multilingual Multimodal Benchmark from Real-World Civil Service Exams

arXiv 2603.27223 ↗First posted 28 Mar 2026cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We present EuraGovExam, a multilingual and multimodal benchmark sourced from real-world civil service examinations across five representative Eurasian regions: South Korea, Japan, Taiwan, India, and the European Union. Designed to reflect the authentic complexity of public-sector assessments, the dataset contains over 8,000 high-resolution scanned multiple-choice questions covering 17 diverse academic and administrative domains. Unlike existing benchmarks, EuraGovExam embeds all question content--including problem statements, answer choices, and visual elements--within a single image, providing only a minimal standardized instruction for answer formatting. This design demands that models perform layout-aware, cross-lingual reasoning directly from visual input. All items are drawn from real exam documents, preserving rich visual structures such as tables, multilingual typography, and form-like layouts. Evaluation results show that even state-of-the-art vision-language models (VLMs) achieve only 86% accuracy, underscoring the benchmark's difficulty and its power to diagnose the limitations of current models. By emphasizing cultural realism, visual complexity, and linguistic diversity, EuraGovExam establishes a new standard for evaluating VLMs in high-stakes, multilingual, image-grounded settings. It also supports practical applications in e-governance, public-sector document analysis, and equitable exam preparation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

15 of 40 references.

References · 15

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  8. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  11. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  12. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
3 more
  1. 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
  2. 2020EXAMS: A Multi-Subject High School Examinations Dataset for Cross-Lingual and Multilingual Question Answeringcited by 60
  3. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models
  2. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models
  3. 2025PISA-Bench: The PISA Index as a Multilingual and Multimodal Metric for the Evaluation of Vision-Language Models
  4. 2023Evaluating GPT-4's Vision Capabilities on Brazilian University Admission Exams
  5. 2026Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
  6. 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding
  7. 2025Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
  8. 2024CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark