# VOLMO: Versatile and Open Large Models for Ophthalmology

arXiv: https://arxiv.org/abs/2603.23953 · HTML: https://mlsanity.com/paper/2603.23953/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 112 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Instruction Tuning with GPT-4](https://mlsanity.com/paper/2304.03277/) (2023)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale](https://mlsanity.com/paper/2406.19280/) (2024)
- [Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models](https://mlsanity.com/paper/2504.04717/) (2025)
- [Do Vision-Language Models Really Understand Visual Language?](https://mlsanity.com/paper/2410.00193/) (2024)
- [Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey](https://mlsanity.com/paper/2312.16602/) (2023)
- [Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions](https://mlsanity.com/paper/2412.02621/) (2024)
- [Performance evaluation of predictive AI models to support medical decisions: Overview and guidance](https://mlsanity.com/paper/2412.10288/) (2024)
- [Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations](https://mlsanity.com/paper/2501.17860/) (2025)
- [A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models](https://mlsanity.com/paper/2508.03734/) (2025)
- [Foundation Models in Medical Image Analysis: A Systematic Review and Meta-Analysis](https://mlsanity.com/paper/2510.16973/) (2025)
- [Benchmarking Large Multimodal Models for Ophthalmic Visual Question Answering with OphthalWeChat](https://mlsanity.com/paper/2505.19624/) (2025)
- [Performance of GPT-5 Frontier Models in Ophthalmology Question Answering](https://mlsanity.com/paper/2508.09956/) (2025)
- [LMOD+: A Comprehensive Multimodal Dataset and Benchmark for Developing and Evaluating Multimodal Large Language Models in Ophthalmology](https://mlsanity.com/paper/2509.25620/) (2025)
- [From Compound Figures to Composite Understanding: Developing a Multi-Modal LLM from Biomedical Literature with Medical Multiple-Image Benchmarking and Validation](https://mlsanity.com/paper/2511.22232/) (2025)
- [Deep Learning for Ophthalmology: The State-of-the-Art and Future Trends](https://mlsanity.com/paper/2501.04073/) (2025)

## Built on by (most cited first, 2 of 2)

- [MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models](https://mlsanity.com/paper/2607.07673/) (2026)
- [Scientific Domain Knowledge Improves Vision-Language Fundus Models](https://mlsanity.com/paper/2605.02720/) (2026)

## Abstract (arXiv)

Vision impairment affects millions globally, and early detection is critical to preventing irreversible vision loss. Ophthalmology workflows require clinicians to integrate medical images, structured clinical data, and free-text notes to determine disease severity and management, which is time-consuming and burdensome. Recent multimodal large language models (MLLMs) show promise, but existing general and medical MLLMs perform poorly in ophthalmology, and few ophthalmology-specific MLLMs are openly available. We present VOLMO (Versatile and Open Large Models for Ophthalmology), a model-agnostic, data-open framework for developing ophthalmology-specific MLLMs. VOLMO includes three stages: ophthalmology knowledge pretraining on 86,965 image-text pairs from 26,569 articles across 82 journals; domain task fine-tuning on 26,929 annotated instances spanning 12 eye conditions for disease screening and severity classification; and multi-step clinical reasoning on 913 patient case reports for assessment, planning, and follow-up care. Using this framework, we trained a compact 2B-parameter MLLM and compared it with strong baselines, including InternVL-2B, LLaVA-Med-7B, MedGemma-4B, MedGemma-27B, and RETFound. We evaluated these models on image description generation, disease screening and staging classification, and assessment-and-management generation, with additional manual review by two healthcare professionals and external validation on three independent cohorts for age-related macular degeneration and diabetic retinopathy. Across settings, VOLMO-2B consistently outperformed baselines, achieving stronger image description performance, an average F1 of 87.4% across 12 eye conditions, and higher scores in external validation.