Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Multimodal large language models (MLLMs) have recently demonstrated remarkable reasoning abilities under reinforcement learning (RL) paradigm. However, most existing multimodal medical reasoning models focus on basic reasoning, which refers to shallow inference based on visual feature matching. In contrast, real-world clinical diagnosis extends beyond basic reasoning, demanding complex reasoning that integrates heterogeneous clinical information (such as chief complaints and medical history) with multimodal medical imaging data. To bridge this gap, we introduce MM-Retinal-Reason, an ophthalmic multimodal dataset covering the full spectrum of perception and reasoning. Specifically, it is the first dataset in ophthalmology to encompass both basic and complex reasoning tasks with Chain-of-Thought (CoT) trajectories, aiming to enhance visual-centric reasoning and emulate realistic clinical decision-making. Building upon MM-Retinal-Reason, we propose OphthaReason, the first RL-enhanced ophthalmic multimodal reasoning model with step-by-step reasoning traces. To enable flexible adaptation to both basic and complex reasoning tasks, we further introduce Uncertainty-Aware Dynamic Thinking (UADT), which estimates sample-level uncertainty via entropy and dynamically modulates exploration depth through a shaped advantage mechanism. Comprehensive experiments demonstrate the effectiveness of our model on both basic and complex reasoning tasks, outperforming general-purpose MLLMs, medical MLLMs, RL-based medical MLLMs, and ophthalmic MLLMs by at least 15.47%. Project Page: hrefhttps://github.com/lxirich/OphthaReasonlink.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 52 of its 74 references that are in the map. First posted on arXiv 22 Aug 2025.
Builds on
52 of 74 references.
Closest ancestors · 14
Same direction, newest first
- 2025Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning23 Jul
- 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
- 2025QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training31 May
- 2025Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning25 May
- 2025MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning22 May
- 2025MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports16 May
- 2025Disentangling Reasoning and Knowledge in Medical Large Language Models16 May
- 2025Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner16 May
- 2025ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification29 Apr
- 2025PathVLM-R1: A Reinforcement Learning-Driven Reasoning Model for Pathology Visual-Language Tasks12 Apr
- 2025GMAI-VL-R1: Harnessing Reinforcement Learning for Multimodal Medical Reasoning2 Apr
- 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models18 Mar
2 more
Other references · 38
- 2023GPT-4 Technical Reportcited by 18,396
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2022Language Models (Mostly) Know What They Knowcited by 1,856
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
26 more
- 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2025MedGemma Technical Reportcited by 436
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
- 2025Reasoning with Exploration: An Entropy Perspectivecited by 281
- 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
- 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 200
- 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
- 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 114
- 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 99
- 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 99
- 2024GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AIcited by 96
- 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
- 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
- 2019Attention Based Glaucoma Detection: A Large-scale Database and CNN Modelcited by 76
- 2021A Max-Min Entropy Framework for Reinforcement Learningcited by 28
- 2024Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicinecited by 28
- 2024MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertisecited by 24
- 2025EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Modelcited by 22
- 2024EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysiscited by 15
- 2024Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Modelcited by 12
- 2025RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Modelscited by 10
- 2024LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Modelscited by 7
- 2025MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretrainingcited by 5
- 2025A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Modelscited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2025Toward Clinically Explainable AI for Medical Diagnosis: A Foundation Model with Human-Compatible Reasoning via Reinforcement Learningcited by 5
- 2026Fundus-R1: Training a Fundus-Reading MLLM with Knowledge-Aware Reasoning on Public Datacited by 2
- 2026CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learningcited by 1
- 2026OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Modelscited by 1
- 2025RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Modelcited by 0
- 2026PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflowcited by 0
- 2026OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models27 May 2026
- 2026PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflow18 May 2026
- 2026Fundus-R1: Training a Fundus-Reading MLLM with Knowledge-Aware Reasoning on Public Data9 Apr 2026
- 2026CORE-Seg: Reasoning-Driven Segmentation for Complex Lesions via Reinforcement Learning6 Mar 2026
- 2025RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model5 Oct 2025
- 2025Toward Clinically Explainable AI for Medical Diagnosis: A Foundation Model with Human-Compatible Reasoning via Reinforcement Learning4 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2023OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue
- 2025Benchmarking Next-Generation Reasoning-Focused Large Language Models in Ophthalmology: A Head-to-Head Evaluation on 5,888 Items
- 2026VOLMO: Versatile and Open Large Models for Ophthalmology
- 2025LMOD+: A Comprehensive Multimodal Dataset and Benchmark for Developing and Evaluating Multimodal Large Language Models in Ophthalmology
- 2026EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding
- 2025EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow
- 2025EyePCR: A Comprehensive Benchmark for Fine-Grained Perception, Knowledge Comprehension and Clinical Reasoning in Ophthalmic Surgery
- 2026EyeWorld: A Generative World Model of Ocular State and Dynamics