# Deep Reinforcement Learning in Computer Vision: A Comprehensive Survey

arXiv: https://arxiv.org/abs/2108.11510 · HTML: https://mlsanity.com/paper/2108.11510/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 140 references that are in the map.
- First posted on arXiv 25 Aug 2021.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1407.1808/) (2014)
- [Deep Spatial Autoencoders for Visuomotor Learning](https://mlsanity.com/paper/1509.06113/) (2015)
- [Video Summarization with Long Short-term Memory](https://mlsanity.com/paper/1605.08110/) (2016)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation](https://mlsanity.com/paper/1707.00652/) (2017)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [Segmental Spatiotemporal CNNs for Fine-grained Action Segmentation](https://mlsanity.com/paper/1602.02995/) (2016)
- [3D Deep Learning on Medical Images: A Review](https://mlsanity.com/paper/2004.00218/) (2020)
- [Video Summarization Using Fully Convolutional Sequence Networks](https://mlsanity.com/paper/1805.10538/) (2018)
- [Experience Replay Optimization](https://mlsanity.com/paper/1906.08387/) (2019)
- [Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video](https://mlsanity.com/paper/1709.05943/) (2017)
- [Weighing Counts: Sequential Crowd Counting by Reinforcement Learning](https://mlsanity.com/paper/2007.08260/) (2020)
- [Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking](https://mlsanity.com/paper/1807.11348/) (2018)
- [Multiple Landmark Detection using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1907.00318/) (2019)
- [Recognizing Surgical Activities with Recurrent Neural Networks](https://mlsanity.com/paper/1606.06329/) (2016)
- [Crowd Counting using Deep Recurrent Spatial-Aware Network](https://mlsanity.com/paper/1807.00601/) (2018)
- [Reinforced Axial Refinement Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2008.13748/) (2020)
- [Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation](https://mlsanity.com/paper/1704.03593/) (2017)
- [Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification](https://mlsanity.com/paper/1806.08089/) (2018)
- [RGB-D-based Action Recognition Datasets: A Survey](https://mlsanity.com/paper/1601.05511/) (2016)
- [Iterative Interaction Training for Segmentation Editing Networks](https://mlsanity.com/paper/1807.08555/) (2018)
- [Longitudinal Face Modeling via Temporal Deep Restricted Boltzmann Machines](https://mlsanity.com/paper/1606.02254/) (2016)

## Built on by (most cited first, 30 of 43)

- [Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2305.18010/) (2023)
- [Tuning computer vision models with task rewards](https://mlsanity.com/paper/2302.08242/) (2023)
- [Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving](https://mlsanity.com/paper/2404.04804/) (2024)
- [Task-Oriented Image Transmission for Scene Classification in Unmanned Aerial Systems](https://mlsanity.com/paper/2112.10948/) (2021)
- [A Comprehensive Overview and Comparative Analysis on Deep Learning Models: CNN, RNN, LSTM, GRU](https://mlsanity.com/paper/2305.17473/) (2023)
- [PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learning](https://mlsanity.com/paper/2202.03609/) (2022)
- [Reinforced UI Instruction Grounding: Towards a Generic UI Task Automation API](https://mlsanity.com/paper/2310.04716/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [Genes in Intelligent Agents](https://mlsanity.com/paper/2306.10225/) (2023)
- [Susceptibility of Continual Learning Against Adversarial Attacks](https://mlsanity.com/paper/2207.05225/) (2022)
- [Uncertainty Estimation for Safety-critical Scene Segmentation via Fine-grained Reward Maximization](https://mlsanity.com/paper/2311.02719/) (2023)
- [AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation](https://mlsanity.com/paper/2405.11467/) (2024)
- [Cauchy activation function and XNet](https://mlsanity.com/paper/2409.19221/) (2024)
- [A Game of Bundle Adjustment -- Learning Efficient Convergence](https://mlsanity.com/paper/2308.13270/) (2023)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised Learning](https://mlsanity.com/paper/2210.06297/) (2022)
- [Test-Time Adaptation with Binary Feedback](https://mlsanity.com/paper/2505.18514/) (2025)
- [Subequivariant Reinforcement Learning Framework for Coordinated Motion Control](https://mlsanity.com/paper/2403.15100/) (2024)
- [Comprehensive Review of Reinforcement Learning for Medical Ultrasound Imaging](https://mlsanity.com/paper/2503.16543/) (2025)
- [Fast and Highly Expressive Policy Learning for Offline Reinforcement Learning via Bootstrapped Flow Q-Learning](https://mlsanity.com/paper/2606.10613/) (2026)
- [CryoRL: Reinforcement Learning Enables Efficient Cryo-EM Data Collection](https://mlsanity.com/paper/2204.07543/) (2022)
- [Online SuBmodular + SuPermodular (BP) Maximization with Bandit Feedback](https://mlsanity.com/paper/2207.03091/) (2022)
- [Mastering Spatial Graph Prediction of Road Networks](https://mlsanity.com/paper/2210.00828/) (2022)
- [Constrained Proximal Policy Optimization](https://mlsanity.com/paper/2305.14216/) (2023)
- [Reinforcement Learning-based Mixture of Vision Transformers for Video Violence Recognition](https://mlsanity.com/paper/2310.03108/) (2023)
- [SolarFormer: Multi-scale Transformer for Solar PV Profiling](https://mlsanity.com/paper/2310.20057/) (2023)
- [A Survey of Blockchain, Artificial Intelligence, and Edge Computing for Web 3.0](https://mlsanity.com/paper/2311.13731/) (2023)
- [CarcassFormer: An End-to-end Transformer-based Framework for Simultaneous Localization, Segmentation and Classification of Poultry Carcass Defect](https://mlsanity.com/paper/2404.11429/) (2024)
- [Coding for Intelligence from the Perspective of Category](https://mlsanity.com/paper/2407.01017/) (2024)

## Abstract (arXiv)

Deep reinforcement learning augments the reinforcement learning framework and utilizes the powerful representation of deep neural networks. Recent works have demonstrated the remarkable successes of deep reinforcement learning in various domains including finance, medicine, healthcare, video games, robotics, and computer vision. In this work, we provide a detailed review of recent and state-of-the-art research advances of deep reinforcement learning in computer vision. We start with comprehending the theories of deep learning, reinforcement learning, and deep reinforcement learning. We then propose a categorization of deep reinforcement learning methodologies and discuss their advantages and limitations. In particular, we divide deep reinforcement learning into seven main categories according to their applications in computer vision, i.e. (i)landmark localization (ii) object detection; (iii) object tracking; (iv) registration on both 2D image and 3D image volumetric data (v) image segmentation; (vi) videos analysis; and (vii) other applications. Each of these categories is further analyzed with reinforcement learning techniques, network design, and performance. Moreover, we provide a comprehensive analysis of the existing publicly available datasets and examine source code availability. Finally, we present some open issues and discuss future research directions on deep reinforcement learning in computer vision