# EyeAgent: An Agentic AI System for Multimodal Clinical Decision Support in Ophthalmology

arXiv: https://arxiv.org/abs/2511.09394 · HTML: https://mlsanity.com/paper/2511.09394/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 62 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 16 of 16)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks](https://mlsanity.com/paper/2305.17100/) (2023)
- [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024)
- [Empowering Biomedical Discovery with AI Agents](https://mlsanity.com/paper/2404.02831/) (2024)
- [MMedAgent: Learning to Use Medical Tools with Multi-modal Agent](https://mlsanity.com/paper/2407.02483/) (2024)
- [OCTID: Optical Coherence Tomography Image Database](https://mlsanity.com/paper/1812.07056/) (2018)
- [A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detection](https://mlsanity.com/paper/2102.07978/) (2021)
- [AgentMD: Empowering Language Agents for Risk Prediction with Large-Scale Clinical Tool Learning](https://mlsanity.com/paper/2402.13225/) (2024)
- [OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methods](https://mlsanity.com/paper/2312.08255/) (2023)
- [EyeDiff: text-to-image diffusion model improves rare eye disease diagnosis](https://mlsanity.com/paper/2411.10004/) (2024)
- [Fundus2Video: Cross-Modal Angiography Video Generation from Static Fundus Photography with Clinical Knowledge Guidance](https://mlsanity.com/paper/2408.15217/) (2024)
- [Fundus2Globe: Generative AI-Driven 3D Digital Twins for Personalized Myopia Management](https://mlsanity.com/paper/2502.13182/) (2025)
- [Choroidal Vessel Segmentation on Indocyanine Green Angiography Images via Human-in-the-Loop Labeling](https://mlsanity.com/paper/2406.01993/) (2024)
- [APTOS-2024 challenge report: Generation of synthetic 3D OCT images from fundus photographs](https://mlsanity.com/paper/2506.07542/) (2025)
- [ChatMyopia: An AI Agent for Pre-consultation Education in Primary Eye Care Settings](https://mlsanity.com/paper/2507.19498/) (2025)

## Abstract (arXiv)

Artificial intelligence has shown promise in medical imaging, yet most existing systems lack flexibility, interpretability, and adaptability - challenges especially pronounced in ophthalmology, where diverse imaging modalities are essential. We present EyeAgent, the first agentic AI framework for comprehensive and interpretable clinical decision support in ophthalmology. Using a large language model (DeepSeek-V3) as its central reasoning engine, EyeAgent interprets user queries and dynamically orchestrates 53 validated ophthalmic tools across 23 imaging modalities for diverse tasks including classification, segmentation, detection, image/report generation, and quantitative analysis. Stepwise ablation analysis demonstrated a progressive improvement in diagnostic accuracy, rising from a baseline of 69.71% (using only 5 general tools) to 80.79% when the full suite of 53 specialized tools was integrated. In an expert rating study on 200 real-world clinical cases, EyeAgent achieved 93.7% tool selection accuracy and received expert ratings of more than 88% across accuracy, completeness, safety, reasoning, and interpretability. In human-AI collaboration, EyeAgent matched or exceeded the performance of senior ophthalmologists and, when used as an assistant, improved overall diagnostic accuracy by 18.51% and report quality scores by 19%, with the greatest benefit observed among junior ophthalmologists. These findings establish EyeAgent as a scalable and trustworthy AI framework for ophthalmology and provide a blueprint for modular, multimodal, and clinically aligned next-generation AI systems.