# A Survey on Deep Learning and Explainability for Automatic Report Generation from Medical Images

arXiv: https://arxiv.org/abs/2010.10563 · HTML: https://mlsanity.com/paper/2010.10563/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2020.

## Built on by (most cited first, 30 of 35)

- [Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities](https://mlsanity.com/paper/2111.06420/) (2021)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [Automated Radiology Report Generation: A Review of Recent Advances](https://mlsanity.com/paper/2405.10842/) (2024)
- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Has Multimodal Learning Delivered Universal Intelligence in Healthcare? A Comprehensive Survey](https://mlsanity.com/paper/2408.12880/) (2024)
- [Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals](https://mlsanity.com/paper/2201.07040/) (2022)
- [LLMCheckup: Conversational Examination of Large Language Models via Interpretability Tools and Self-Explanations](https://mlsanity.com/paper/2401.12576/) (2024)
- [Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Model](https://mlsanity.com/paper/2411.12783/) (2024)
- [A scoping review on multimodal deep learning in biomedical images and texts](https://mlsanity.com/paper/2307.07362/) (2023)
- [A Survey of Medical Vision-and-Language Applications and Their Techniques](https://mlsanity.com/paper/2411.12195/) (2024)
- [Customizing General-Purpose Foundation Models for Medical Report Generation](https://mlsanity.com/paper/2306.05642/) (2023)
- [ReXErr: Synthesizing Clinically Meaningful Errors in Diagnostic Radiology Reports](https://mlsanity.com/paper/2409.10829/) (2024)
- [Radiology-Aware Model-Based Evaluation Metric for Report Generation](https://mlsanity.com/paper/2311.16764/) (2023)
- [A Survey of Deep Learning-based Radiology Report Generation Using Multimodal Data](https://mlsanity.com/paper/2405.12833/) (2024)
- [Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation](https://mlsanity.com/paper/2407.01948/) (2024)
- [Navigating Distribution Shifts in Medical Image Analysis: A Survey](https://mlsanity.com/paper/2411.05824/) (2024)
- [UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Models](https://mlsanity.com/paper/2503.20504/) (2025)
- [Toward Relieving Clinician Burden by Automatically Generating Progress Notes using Interim Hospital Data](https://mlsanity.com/paper/2410.12845/) (2024)
- [EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge](https://mlsanity.com/paper/2411.10224/) (2024)
- [R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation](https://mlsanity.com/paper/2508.03426/) (2025)
- [A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Research](https://mlsanity.com/paper/2401.14617/) (2024)
- [Explainable Interface for Human-Autonomy Teaming: A Survey](https://mlsanity.com/paper/2405.02583/) (2024)
- [Medical Report Generation Is A Multi-label Classification Problem](https://mlsanity.com/paper/2409.00250/) (2024)
- [Anatomy-Guided Radiology Report Generation with Pathology-Aware Regional Prompts](https://mlsanity.com/paper/2411.10789/) (2024)
- [LaB-RAG: Label Boosted Retrieval Augmented Generation for Radiology Report Generation](https://mlsanity.com/paper/2411.16523/) (2024)
- [TRACE: Temporal Radiology with Anatomical Change Explanation for Grounded X-ray Report Generation](https://mlsanity.com/paper/2602.02963/) (2026)
- [Dynamic Traceback Learning for Medical Report Generation](https://mlsanity.com/paper/2401.13267/) (2024)
- [Med-R2: Perception and Reflection-driven Complex Reasoning for Medical Report Generation](https://mlsanity.com/paper/2504.02885/) (2025)
- [AI Analyst: Framework and Comprehensive Evaluation of Large Language Models for Financial Time Series Report Generation](https://mlsanity.com/paper/2507.00718/) (2025)
- [Personalized Longitudinal Medical Report Generation via Temporally-Aware Federated Adaptation](https://mlsanity.com/paper/2602.19668/) (2026)

## Abstract (arXiv)

Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to: (1) Datasets, (2) Architecture Design, (3) Explainability and (4) Evaluation Metrics. Our survey identifies interesting developments, but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on traditional Natural Language Processing (NLP) metrics, which do not accurately capture medical correctness.