# Deep reinforcement learning in medical imaging: A literature review

arXiv: https://arxiv.org/abs/2103.05115 · HTML: https://mlsanity.com/paper/2103.05115/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 160 references that are in the map.
- First posted on arXiv 5 Mar 2021.

## Builds on (most cited first, 30 of 69)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Model-Based Reinforcement Learning with Value-Targeted Regression](https://mlsanity.com/paper/2006.01107/) (2020)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Model-Ensemble Trust-Region Policy Optimization](https://mlsanity.com/paper/1802.10592/) (2018)
- [Multi-agent Reinforcement Learning in Sequential Social Dilemmas](https://mlsanity.com/paper/1702.03037/) (2017)
- [Benchmarking Model-Based Reinforcement Learning](https://mlsanity.com/paper/1907.02057/) (2019)
- [A Survey of Reinforcement Learning Informed by Natural Language](https://mlsanity.com/paper/1906.03926/) (2019)
- [Deep Spatial Autoencoders for Visuomotor Learning](https://mlsanity.com/paper/1509.06113/) (2015)

## Built on by (most cited first, 30 of 34)

- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applications](https://mlsanity.com/paper/2206.07922/) (2022)
- [Segment Anything Model (SAM) for Radiation Oncology](https://mlsanity.com/paper/2306.11730/) (2023)
- [Robust Deep Reinforcement Learning through Bootstrapped Opportunistic Curriculum](https://mlsanity.com/paper/2206.10057/) (2022)
- [Deep reinforcement learning with automated label extraction from clinical reports accurately classifies 3D MRI brain volumes](https://mlsanity.com/paper/2106.09812/) (2021)
- [DuDoTrans: Dual-Domain Transformer Provides More Attention for Sinogram Restoration in Sparse-View CT Reconstruction](https://mlsanity.com/paper/2111.10790/) (2021)
- [EmbryosFormer: Deformable Transformer and Collaborative Encoding-Decoding for Embryos Stage Development Classification](https://mlsanity.com/paper/2210.04615/) (2022)
- [SS-3DCapsNet: Self-supervised 3D Capsule Networks for Medical Segmentation on Less Labeled Data](https://mlsanity.com/paper/2201.05905/) (2022)
- [I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses](https://mlsanity.com/paper/2309.13550/) (2023)
- [RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learning](https://mlsanity.com/paper/2212.01737/) (2022)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [How to Fine-tune the Model: Unified Model Shift and Model Bias Policy Optimization](https://mlsanity.com/paper/2309.12671/) (2023)
- [Flip Learning: Erase to Segment](https://mlsanity.com/paper/2108.00752/) (2021)
- [Reinforced Inverse Scattering](https://mlsanity.com/paper/2206.04186/) (2022)
- [Dual Attention Model with Reinforcement Learning for Classification of Histology Whole-Slide Images](https://mlsanity.com/paper/2302.09682/) (2023)
- [Learning Low-Rank Feature for Thorax Disease Classification](https://mlsanity.com/paper/2404.18933/) (2024)
- [Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detection](https://mlsanity.com/paper/2504.13186/) (2025)
- [Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised Learning](https://mlsanity.com/paper/2210.06297/) (2022)
- [Towards Saner Deep Image Registration](https://mlsanity.com/paper/2307.09696/) (2023)
- [Flip Learning: Weakly Supervised Erase to Segment Nodules in Breast Ultrasound](https://mlsanity.com/paper/2503.20685/) (2025)
- [Strategising template-guided needle placement for MR-targeted prostate biopsy](https://mlsanity.com/paper/2207.10784/) (2022)
- [Multi-Target Landmark Detection with Incomplete Images via Reinforcement Learning and Shape Prior](https://mlsanity.com/paper/2301.05392/) (2023)
- [Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging](https://mlsanity.com/paper/2503.16543/) (2025)
- [Towards deep observation: A systematic survey on artificial intelligence techniques to monitor fetus via Ultrasound Images](https://mlsanity.com/paper/2201.07935/) (2022)
- [Deep reinforcement learning for fMRI prediction of Autism Spectrum Disorder](https://mlsanity.com/paper/2206.11224/) (2022)
- [Automated Gadget Discovery in Science](https://mlsanity.com/paper/2212.12743/) (2022)
- [SolarFormer: Multi-scale Transformer for Solar PV Profiling](https://mlsanity.com/paper/2310.20057/) (2023)
- [CarcassFormer: An End-to-end Transformer-based Framework for Simultaneous Localization, Segmentation and Classification of Poultry Carcass Defect](https://mlsanity.com/paper/2404.11429/) (2024)
- [EMAD: Evidence-Centric Grounded Multimodal Diagnosis for Alzheimer's Disease](https://mlsanity.com/paper/2602.19178/) (2026)
- [Automated Diagnosis of Cardiovascular Diseases from Cardiac Magnetic Resonance Imaging Using Deep Learning Models: A Review](https://mlsanity.com/paper/2210.14909/) (2022)

## Abstract (arXiv)

Deep reinforcement learning (DRL) augments the reinforcement learning framework, which learns a sequence of actions that maximizes the expected reward, with the representative power of deep neural networks. Recent works have demonstrated the great potential of DRL in medicine and healthcare. This paper presents a literature review of DRL in medical imaging. We start with a comprehensive tutorial of DRL, including the latest model-free and model-based algorithms. We then cover existing DRL applications for medical imaging, which are roughly divided into three main categories: (I) parametric medical image analysis tasks including landmark detection, object/lesion detection, registration, and view plane localization; (ii) solving optimization tasks including hyperparameter tuning, selecting augmentation strategies, and neural architecture search; and (iii) miscellaneous applications including surgical gesture segmentation, personalized mobile health intervention, and computational model personalization. The paper concludes with discussions of future perspectives.