GEMeX-RMCoT: An Enhanced Med-VQA Dataset for Region-Aware Multimodal Chain-of-Thought Reasoning
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Medical visual question answering aims to support clinical decision-making by enabling models to answer natural language questions based on medical images. While recent advances in multi-modal learning have significantly improved performance, current methods still suffer from limited answer reliability and poor interpretability, impairing the ability of clinicians and patients to understand and trust model outputs. To address these limitations, this work first proposes a Region-Aware Multimodal Chain-of-Thought (RMCoT) dataset, in which the process of producing an answer is preceded by a sequence of intermediate reasoning steps that explicitly ground relevant visual regions of the medical image, thereby providing fine-grained explainability. Furthermore, we introduce a novel verifiable reward mechanism for reinforcement learning to guide post-training, improving the alignment between the model's reasoning process and its final answer. Remarkably, our method achieves comparable performance using only one-eighth of the training data, demonstrating the efficiency and effectiveness of the proposal. The dataset is available at https://www.med-vqa.com/GEMeX/.
Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 29 of its 42 references that are in the map. First posted on arXiv 22 Jun 2025.
Builds on
29 of 42 references.
Closest ancestors · 2
Same direction, newest first
- 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models18 Mar
- 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb
Other references · 27
- 2023GPT-4 Technical Reportcited by 18,132
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024OpenAI o1 System Cardcited by 1,220
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
- 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
15 more
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
- 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 310
- 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
- 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 223
- 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
- 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
- 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
- 2024Preference Optimization for Reasoning with Pseudo Feedbackcited by 38
- 2025Scientists' First Exam: Probing Cognitive Abilities of MLLM via Perception, Understanding, and Reasoningcited by 33
- 2024CodeDPO: Aligning Code Models with Self Generated and Verified Source Codecited by 32
- 2024GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosiscited by 28
- 2025Group Relative Policy Optimization for Image Captioningcited by 9
- 2025MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMscited by 9
- 2024EyeGPT: Ophthalmic Assistant with Large Language Modelscited by 4
Built on by
Citing papers per year: 2025: 4; 2026: 4.
8 papers.
- 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applicationscited by 10
- 2025OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoningcited by 10
- 2026MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learningcited by 5
- 2025Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigationcited by 4
- 2026Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidencecited by 1
- 2026CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMscited by 1
- 2025Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patientscited by 0
- 2026UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasoundcited by 0
- 2026UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound25 Sep 2026
- 2026CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs25 Jun 2026
- 2026Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence21 May 2026
- 2026MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning9 Apr 2026
- 2025Benchmark Success, Clinical Failure: When Reinforcement Learning Optimizes for Benchmarks, Not Patients28 Dec 2025
- 2025OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning28 Nov 2025
- 2025Adaptive Reinforcement for Open-ended Medical Reasoning via Semantic-Guided Reward Collapse Mitigation18 Aug 2025
- 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications1 Aug 2025
Similar, not cited
Close in wording, no citation either way
- 2026LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
- 2025CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
- 2025Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs
- 2025CAPO: Reinforcing Consistent Reasoning in Medical Decision-Making
- 2026Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning
- 2024MedThink: Explaining Medical Visual Question Answering via Multimodal Decision-Making Rationale
- 2026MedQA-MM: Shortcuts Behind Medical Visual Reasoning
- 2023Visual Question Answering in the Medical Domain