# RetiBridge: Bridging Quantitative Retinal Biomarkers and Qualitative Diagnosis with a Knowledge-Guided Multimodal Large Language Model

arXiv: https://arxiv.org/abs/2510.04281 · HTML: https://mlsanity.com/paper/2510.04281/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 5 Oct 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning](https://mlsanity.com/paper/2506.07044/) (2025)
- [Evaluation of Retinal Image Quality Assessment Networks in Different Color-spaces](https://mlsanity.com/paper/1907.05345/) (2019)
- [OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue](https://mlsanity.com/paper/2306.12174/) (2023)
- [EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Model](https://mlsanity.com/paper/2504.13650/) (2025)
- [VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced with Clinical Knowledge](https://mlsanity.com/paper/2408.02865/) (2024)
- [RetinalGPT: A Retinal Clinical Preference Conversational Assistant Powered by Large Vision-Language Models](https://mlsanity.com/paper/2503.03987/) (2025)
- [Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning](https://mlsanity.com/paper/2507.17539/) (2025)
- [Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning](https://mlsanity.com/paper/2508.16129/) (2025)
- [Specialized curricula for training vision-language models in retinal image analysis](https://mlsanity.com/paper/2407.08410/) (2024)

## Abstract (arXiv)

Retinal biomarkers captured by color fundus photography and optical coherence tomography provide clinically valuable evidence for both ocular and systemic diseases. Multimodal large language models (MLLMs) have shown promise for retinal image interpretation, yet existing ophthalmic models rarely quantify these clinically relevant biomarkers or explicitly translate their measurements into qualitative, evidence-grounded diagnostic conclusions. To address this gap, we introduce RetiBridge, a knowledge-guided multimodal large language model that jointly analyzes color fundus photography (CFP), optical coherence tomography (OCT), and text, explicitly bridging quantitative retinal biomarkers to qualitative clinical sub-inferences and coherent diagnostic conclusions. RetiBridge combines knowledge-guided instruction generation, OCT-biomarker alignment, and supervised multimodal instruction tuning to learn a biomarker-grounded quantitative-to-qualitative diagnostic pathway. Using 15,611 paired CFP-OCT samples from UK Biobank with 31 OCT and 6 CFP biomarkers, we construct the Grounded Ophthalmic Understanding benchmark to evaluate diagnostic classification, report generation quality, and fine-grained clinical quality. Despite using only LoRA-based fine-tuning of a 7B-parameter Qwen2 backbone, RetiBridge outperforms all evaluated open-source 7B and 32B baselines, achieving the highest quantitative accuracy, evidence grounding, coverage completeness, and BERTScore, while surpassing OpenAI o3 on these key biomarker-grounded metrics. Our code and data are released in the RetiBridge repository.