# EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imaging

arXiv: https://arxiv.org/abs/2405.11338 · HTML: https://mlsanity.com/paper/2405.11338/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 17 references that are in the map.
- First posted on arXiv 18 May 2024.

## Builds on (most cited first, 3 of 3)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [OCTID: Optical Coherence Tomography Image Database](https://mlsanity.com/paper/1812.07056/) (2018)
- [OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methods](https://mlsanity.com/paper/2312.08255/) (2023)

## Built on by (most cited first, 18 of 18)

- [Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectly](https://mlsanity.com/paper/2406.10638/) (2024)
- [Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions](https://mlsanity.com/paper/2412.02621/) (2024)
- [EyeCLIP: A visual-language foundation model for multi-modal ophthalmic image analysis](https://mlsanity.com/paper/2409.06644/) (2024)
- [ViLReF: An Expert Knowledge Enabled Vision-Language Retinal Foundation Model](https://mlsanity.com/paper/2408.10894/) (2024)
- [A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models](https://mlsanity.com/paper/2508.03734/) (2025)
- [Predicting Diabetic Macular Edema Treatment Responses Using OCT: Dataset and Methods of APTOS Competition](https://mlsanity.com/paper/2505.05768/) (2025)
- [FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis](https://mlsanity.com/paper/2508.11721/) (2025)
- [Fundus to Fluorescein Angiography Video Generation as a Retinal Generative Foundation Model](https://mlsanity.com/paper/2410.13242/) (2024)
- [Visual Question Answering in Ophthalmology: A Progressive and Practical Perspective](https://mlsanity.com/paper/2410.16662/) (2024)
- [Benchmarking Ophthalmology Foundation Models for Clinically Significant Age Macular Degeneration Detection](https://mlsanity.com/paper/2505.05291/) (2025)
- [AI-powered virtual eye: perspective, challenges and opportunities](https://mlsanity.com/paper/2505.05516/) (2025)
- [Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)](https://mlsanity.com/paper/2507.00185/) (2025)
- [Prompt Mechanisms in Medical Imaging: A Comprehensive Survey](https://mlsanity.com/paper/2507.01055/) (2025)
- [Evaluating Fundus-Specific Foundation Models for Diabetic Macular Edema Detection](https://mlsanity.com/paper/2510.07277/) (2025)
- [Representation learning from OCT images](https://mlsanity.com/paper/2605.02589/) (2026)
- [EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining](https://mlsanity.com/paper/2606.15129/) (2026)
- [Learning When to Recur: Token-Adaptive Recursion for Imbalanced Ophthalmic Domain Incremental Learning](https://mlsanity.com/paper/2609.32785/) (2026)
- [Med-RADIO: Reducing All Medical Domains Into One via Multi-Teacher Distillation](https://mlsanity.com/paper/2609.37682/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) is vital in ophthalmology, tackling tasks like diagnosis, classification, and visual question answering (VQA). However, existing AI models in this domain often require extensive annotation and are task-specific, limiting their clinical utility. While recent developments have brought about foundation models for ophthalmology, they are limited by the need to train separate weights for each imaging modality, preventing a comprehensive representation of multi-modal features. This highlights the need for versatile foundation models capable of handling various tasks and modalities in ophthalmology. To address this gap, we present EyeFound, a multimodal foundation model for ophthalmic images. Unlike existing models, EyeFound learns generalizable representations from unlabeled multimodal retinal images, enabling efficient model adaptation across multiple applications. Trained on 2.78 million images from 227 hospitals across 11 ophthalmic modalities, EyeFound facilitates generalist representations and diverse multimodal downstream tasks, even for detecting challenging rare diseases. It outperforms previous work RETFound in diagnosing eye diseases, predicting systemic disease incidents, and zero-shot multimodal VQA. EyeFound provides a generalizable solution to improve model performance and lessen the annotation burden on experts, facilitating widespread clinical AI applications for retinal imaging.