mlsanityData to 14 Oct 2026

MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning

arXiv 2505.16964 ↗First posted 22 May 2025cs.CV cs.CLcited by 34

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Real-world clinical practice demands multi-image comparative reasoning, yet current medical benchmarks remain limited to single-frame interpretation. We present MedFrameQA, the first benchmark explicitly designed to test multi-image medical VQA through educationally-validated diagnostic sequences. To construct this dataset, we develop a scalable pipeline that leverages narrative transcripts from medical education videos to align visual frames with textual concepts, automatically producing 2,851 high-quality multi-image VQA pairs with explicit, transcript-grounded reasoning chains. Our evaluation of 11 advanced MLLMs (including reasoning models) exposes severe deficiencies in multi-image synthesis, where accuracies mostly fall below 50% and exhibit instability across varying image counts. Error analysis demonstrates that models often treat images as isolated instances, failing to track pathological progression or cross-reference anatomical shifts. MedFrameQA provides a rigorous standard for evaluating the next generation of MLLMs in handling complex, temporally grounded medical narratives.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 19 of its 46 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

19 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs1 Apr
  2. 2025m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models1 Apr
  3. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
  6. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  7. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  8. 2023Towards Generalist Biomedical AIcited by 289
  9. 2024Capabilities of Gemini Models in Medicinecited by 237
  10. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  11. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  12. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
4 more
  1. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  2. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 124
  3. 2024A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?cited by 33
  4. 2024Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQAcited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 22.

34 papers.

Sort:

By area:

  1. 2025Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopycited by 11
  2. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbationscited by 9
  3. 2025MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoningcited by 8
  4. 2025OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Taskscited by 8
  5. 2025Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoningcited by 6
  6. 2026Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Modelscited by 6
  7. 2026Medical thinking with multiple imagescited by 4
  8. 2026RadThinking: A Dataset for Longitudinal Clinical Reasoning in Radiologycited by 4
  9. 2026ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoningcited by 4
  10. 2026AutoMedBench: Towards Medical AutoResearch with Agentic AI Modelscited by 4
  11. 2025Multimodal Retrieval-Augmented Generation with Large Language Models for Medical VQAcited by 3
  12. 2026ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understandingcited by 3
22 more
  1. 2025Architecting Clinical Collaboration: Multi-Agent Reasoning Systems for Multimodal Medical VQAcited by 2
  2. 2025MedRepBench: A Comprehensive Benchmark for Medical Report Interpretationcited by 2
  3. 2025TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Modelscited by 2
  4. 2026Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRIcited by 2
  5. 2026ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoningcited by 2
  6. 2026MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmarkcited by 1
  7. 2026Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Modelscited by 1
  8. 2026Unified Hallucination Fuzzing for Multimodal Large Language Modelscited by 1
  9. 2025MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQAcited by 0
  10. 2025Lightweight Structured Multimodal Reasoning for Clinical Scene Understanding in Roboticscited by 0
  11. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generationcited by 0
  12. 2026MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Modelscited by 0
  13. 2026MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicinecited by 0
  14. 2026A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scalingcited by 0
  15. 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmarkcited by 0
  16. 2026X-PCR: A Benchmark for Cross-modality Progressive Clinical Reasoning in Ophthalmic Diagnosiscited by 0
  17. 2026LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answeringcited by 0
  18. 2026Inference-Time Agentic Decision Rules Beat Longer Evolving Search for Multi-Image Medical Reasoningcited by 0
  19. 2026Objective-Aligned Direct Answer SFT for Robust Multi-Frame Medical VQAcited by 0
  20. 2026CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understandingcited by 0
  21. 2026How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning?cited by 0
  22. 2026Training-Free Speech-Centric Omni Understanding with Frozen VLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MedHorizon: Towards Long-context Medical Video Understanding in the Wild
  2. 2025Med-CMR: A Fine-Grained Benchmark Integrating Visual Evidence and Clinical Logic for Medical Complex Multimodal Reasoning
  3. 2026LoMeVQA: A Comprehensive Benchmark for Longitudinal Medical VQA
  4. 2026MedObvious: Exposing the Medical Moravec's Paradox in VLMs via Clinical Triage
  5. 2024MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis
  6. 2025MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMs
  7. 2025MedAtlas: Evaluating LLMs for Multi-Round, Multi-Task Medical Reasoning Across Diverse Imaging Modalities and Clinical Text
  8. 2025MedXplain-VQA: Multi-Component Explainable Medical Visual Question Answering