# DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation

arXiv: https://arxiv.org/abs/2011.00569 · HTML: https://mlsanity.com/paper/2011.00569/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 30 of 43)

- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision](https://mlsanity.com/paper/2308.07898/) (2023)
- [The Dawn of Quantum Natural Language Processing](https://mlsanity.com/paper/2110.06510/) (2021)
- [GPT2MVS: Generative Pre-trained Transformer-2 for Multi-modal Video Summarization](https://mlsanity.com/paper/2104.12465/) (2021)
- [Automated Generation of Accurate & Fluent Medical X-ray Reports](https://mlsanity.com/paper/2108.12126/) (2021)
- [Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation](https://mlsanity.com/paper/2206.01988/) (2022)
- [VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced with Clinical Knowledge](https://mlsanity.com/paper/2408.02865/) (2024)
- [Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics](https://mlsanity.com/paper/2406.15264/) (2024)
- [Contextualized Keyword Representations for Multi-modal Retinal Image Captioning](https://mlsanity.com/paper/2104.12471/) (2021)
- [Longer Version for "Deep Context-Encoding Network for Retinal Image Captioning"](https://mlsanity.com/paper/2105.14538/) (2021)
- [Query-based Video Summarization with Pseudo Label Supervision](https://mlsanity.com/paper/2307.01945/) (2023)
- [Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model](https://mlsanity.com/paper/2406.09317/) (2024)
- [A Novel Evaluation Framework for Image2Text Generation](https://mlsanity.com/paper/2408.01723/) (2024)
- [FMBench: Benchmarking Fairness in Multimodal Large Language Models on Medical Tasks](https://mlsanity.com/paper/2410.01089/) (2024)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [UrFound: Towards Universal Retinal Foundation Models via Knowledge-Guided Masked Modeling](https://mlsanity.com/paper/2408.05618/) (2024)
- [Improving Visual Question Answering Models through Robustness Analysis and In-Context Learning with a Chain of Basic Questions](https://mlsanity.com/paper/2304.03147/) (2023)
- [QFMTS: Generating Query-Focused Summaries over Multi-Table Inputs](https://mlsanity.com/paper/2405.05109/) (2024)
- [Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning](https://mlsanity.com/paper/2506.02738/) (2025)
- [LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement](https://mlsanity.com/paper/2105.01299/) (2021)
- [Causalainer: Causal Explainer for Automatic Video Summarization](https://mlsanity.com/paper/2305.00455/) (2023)
- [Advancing Medical Representation Learning Through High-Quality Data](https://mlsanity.com/paper/2503.14377/) (2025)
- [Conditional Modeling Based Automatic Video Summarization](https://mlsanity.com/paper/2311.12159/) (2023)
- [A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data](https://mlsanity.com/paper/2405.12833/) (2024)
- [Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning](https://mlsanity.com/paper/2511.20196/) (2025)
- [Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models](https://mlsanity.com/paper/2407.19041/) (2024)
- [MM-Retinal V2: Transfer an Elite Knowledge Spark into Fundus Vision-Language Pretraining](https://mlsanity.com/paper/2501.15798/) (2025)
- [Unleashing the Potential of Large Language Models for Text-to-Image Generation through Autoregressive Representation Alignment](https://mlsanity.com/paper/2503.07334/) (2025)
- [Causal Video Summarizer for Video Exploration](https://mlsanity.com/paper/2307.01947/) (2023)
- [A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models](https://mlsanity.com/paper/2508.03734/) (2025)

## Abstract (arXiv)

In this work, we propose an AI-based method that intends to improve the conventional retinal disease treatment procedure and help ophthalmologists increase diagnosis efficiency and accuracy. The proposed method is composed of a deep neural networks-based (DNN-based) module, including a retinal disease identifier and clinical description generator, and a DNN visual explanation module. To train and validate the effectiveness of our DNN-based module, we propose a large-scale retinal disease image dataset. Also, as ground truth, we provide a retinal image dataset manually labeled by ophthalmologists to qualitatively show, the proposed AI-based method is effective. With our experimental results, we show that the proposed method is quantitatively and qualitatively effective. Our method is capable of creating meaningful retinal image descriptions and visual explanations that are clinically relevant.