# DeepEyeNet: Generating Medical Report for Retinal Images

arXiv: https://arxiv.org/abs/2509.12534 · HTML: https://mlsanity.com/paper/2509.12534/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [On the Automatic Generation of Medical Imaging Reports](https://mlsanity.com/paper/1711.08195/) (2017)
- [DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation](https://mlsanity.com/paper/2011.00569/) (2020)
- [Aligning Visual Regions and Textual Concepts for Semantic-Grounded Image Representations](https://mlsanity.com/paper/1905.06139/) (2019)
- [GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization](https://mlsanity.com/paper/2104.12465/) (2021)
- [Query-controllable Video Summarization](https://mlsanity.com/paper/2004.03661/) (2020)
- [Image Captioning with Deep Bidirectional LSTMs](https://mlsanity.com/paper/1604.00790/) (2016)
- [A Novel Framework for Robustness Analysis of Visual QA Models](https://mlsanity.com/paper/1711.06232/) (2017)
- [VQABQ: Visual Question Answering by Basic Questions](https://mlsanity.com/paper/1703.06492/) (2017)
- [Robustness Analysis of Visual QA Models by Basic Questions](https://mlsanity.com/paper/1709.04625/) (2017)
- [Assessing the Robustness of Visual Question Answering Models](https://mlsanity.com/paper/1912.01452/) (2019)
- [Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics](https://mlsanity.com/paper/2406.15264/) (2024)
- [A Novel Hybrid Machine Learning Model for Auto-Classification of Retinal Diseases](https://mlsanity.com/paper/1806.06423/) (2018)
- [Contextualized Keyword Representations for Multi-modal Retinal Image Captioning](https://mlsanity.com/paper/2104.12471/) (2021)
- [Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"](https://mlsanity.com/paper/2105.14538/) (2021)
- [Synthesizing New Retinal Symptom Images by Multiple Generative Models](https://mlsanity.com/paper/1902.04147/) (2019)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [A Novel Evaluation Framework for Image2Text Generation](https://mlsanity.com/paper/2408.01723/) (2024)
- [Image2Text2Image: A Novel Framework for Label-Free Evaluation of Image-to-Text Generation with Text-to-Image Diffusion Models](https://mlsanity.com/paper/2411.05706/) (2024)
- [Auto-Classification of Retinal Diseases in the Limit of Sparse Data Using a Two-Streams Machine Learning Model](https://mlsanity.com/paper/1808.05754/) (2018)
- [QFMTS: Generating Query-Focused Summaries over Multi-Table Inputs](https://mlsanity.com/paper/2405.05109/) (2024)
- [SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation](https://mlsanity.com/paper/2502.05539/) (2025)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [MaCP: Minimal yet Mighty Adaptation via Hierarchical Cosine Projection](https://mlsanity.com/paper/2410.09103/) (2024)
- [A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation](https://mlsanity.com/paper/2408.12398/) (2024)
- [Personalized Video Summarization using Text-Based Queries and Conditional Modeling](https://mlsanity.com/paper/2408.14743/) (2024)

## Built on by (most cited first, 1 of 1)

- [Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model](https://mlsanity.com/paper/2510.02403/) (2025)

## Abstract (arXiv)

The increasing prevalence of retinal diseases poses a significant challenge to the healthcare system, as the demand for ophthalmologists surpasses the available workforce. This imbalance creates a bottleneck in diagnosis and treatment, potentially delaying critical care. Traditional methods of generating medical reports from retinal images rely on manual interpretation, which is time-consuming and prone to errors, further straining ophthalmologists' limited resources. This thesis investigates the potential of Artificial Intelligence (AI) to automate medical report generation for retinal images. AI can quickly analyze large volumes of image data, identifying subtle patterns essential for accurate diagnosis. By automating this process, AI systems can greatly enhance the efficiency of retinal disease diagnosis, reducing doctors' workloads and enabling them to focus on more complex cases. The proposed AI-based methods address key challenges in automated report generation: (1) A multi-modal deep learning approach captures interactions between textual keywords and retinal images, resulting in more comprehensive medical reports; (2) Improved methods for medical keyword representation enhance the system's ability to capture nuances in medical terminology; (3) Strategies to overcome RNN-based models' limitations, particularly in capturing long-range dependencies within medical descriptions; (4) Techniques to enhance the interpretability of the AI-based report generation system, fostering trust and acceptance in clinical practice. These methods are rigorously evaluated using various metrics and achieve state-of-the-art performance. This thesis demonstrates AI's potential to revolutionize retinal disease diagnosis by automating medical report generation, ultimately improving clinical efficiency, diagnostic accuracy, and patient care.