mlsanityData to 7 Oct 2026

KorMedMCQA-V: A Multimodal Benchmark for Evaluating Vision-Language Models on the Korean Medical Licensing Examination

arXiv 2602.13650 ↗First posted 14 Feb 2026cs.CV cs.AI cs.CLcited by 1

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

We introduce KorMedMCQA-V, a Korean medical licensing-exam-style multimodal multiple-choice question answering benchmark for evaluating vision-language models (VLMs). The dataset consists of 1,534 questions with 2,043 associated images from Korean Medical Licensing Examinations (2012-2023), with about 30% containing multiple images requiring cross-image evidence integration. Images cover clinical modalities including X-ray, computed tomography (CT), electrocardiography (ECG), ultrasound, endoscopy, and other medical visuals. We benchmark over 50 VLMs across proprietary and open-source categories-spanning general-purpose, medical-specialized, and Korean-specialized families-under a unified zero-shot evaluation protocol. The best proprietary model (Gemini-3.0-Pro) achieves 96.9% accuracy, the best open-source model (Qwen3-VL-32B-Thinking) 83.7%, and the best Korean-specialized model (VARCO-VISION-2.0-14B) only 43.2%. We further find that reasoning-oriented model variants gain up to +20 percentage points over instruction-tuned counterparts, medical domain specialization yields inconsistent gains over strong general-purpose baselines, all models degrade on multi-image questions, and performance varies notably across imaging modalities. By complementing the text-only KorMedMCQA benchmark, KorMedMCQA-V forms a unified evaluation suite for Korean medical reasoning across text-only and multimodal conditions. The dataset is available via Hugging Face Datasets: https://huggingface.co/datasets/seongsubae/KorMedMCQA-V.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 27 references that are in the map. First posted on arXiv 14 Feb 2026.

Builds on

25 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  2. 2024WorldMedQA-V: a multilingual, multimodal medical examination dataset for multimodal language models evaluation16 Oct

Other references · 23

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  5. 2025Gemma 3 Technical Reportcited by 1,686
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  8. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  9. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  10. 2025Kimi-VL Technical Reportcited by 381
  11. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 96
  12. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
11 more
  1. 2024KorMedMCQA: Multi-Choice Question Answering Benchmark for Korean Healthcare Professional Licensing Examinationscited by 22
  2. 2024Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaborationcited by 19
  3. 2025Kanana: Compute-efficient Bilingual Language Modelscited by 19
  4. 2024HyperCLOVA X Technical Reportcited by 18
  5. 2025KRETA: A Benchmark for Korean Reading and Reasoning in Text-Rich VQA Attuned to Diverse Visual Contextscited by 6
  6. 2025KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Languagecited by 5
  7. 2025VARCO-VISION-2.0 Technical Reportcited by 5
  8. 2025SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documentscited by 4
  9. 2025KokushiMD-10: Benchmark for Evaluating Large Language Models on Ten Japanese National Healthcare Licensing Examinationscited by 3
  10. 2026What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Modelscited by 2
  11. 2025Exploring OCR-augmented Generation for Bilingual VQAcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026JMed48k: A Multi-Profession Japanese Medical Licensing Benchmark for Vision-Language Model Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM
  2. 2026Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence
  3. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review
  4. 2024EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models
  5. 2026Does Language Shift Break Medical Vision-Language Models? Indonesian Radiology Visual Question Answering Case Study
  6. 2025Benchmarking Large Multimodal Models for Ophthalmic Visual Question Answering with OphthalWeChat
  7. 2024SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks
  8. 2026Analyzing and Mitigating Cross-Lingual Degradation in Multilingual Medical VQA