# An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs

arXiv: https://arxiv.org/abs/2105.01342 · HTML: https://mlsanity.com/paper/2105.01342/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 199 references that are in the map.
- First posted on arXiv 4 May 2021.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges](https://mlsanity.com/paper/1902.07830/) (2019)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [Video Representation Learning by Dense Predictive Coding](https://mlsanity.com/paper/1909.04656/) (2019)
- [Vision Meets Drones: A Challenge](https://mlsanity.com/paper/1804.07437/) (2018)
- [Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice](https://mlsanity.com/paper/1805.11773/) (2018)
- [Fully Convolutional Adaptation Networks for Semantic Segmentation](https://mlsanity.com/paper/1804.08286/) (2018)
- [Video Processing from Electro-optical Sensors for Object Detection and Tracking in Maritime Environment: A Survey](https://mlsanity.com/paper/1611.05842/) (2016)
- [Learning Multi-scale Features for Foreground Segmentation](https://mlsanity.com/paper/1808.01477/) (2018)
- [Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation](https://mlsanity.com/paper/1811.05255/) (2018)
- [Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Tracking](https://mlsanity.com/paper/1807.11348/) (2018)
- [Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection](https://mlsanity.com/paper/1810.09111/) (2018)
- [BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos](https://mlsanity.com/paper/1907.11371/) (2019)
- [MOR-UAV: A Benchmark Dataset and Baselines for Moving Object Recognition in UAV Videos](https://mlsanity.com/paper/2008.01699/) (2020)
- [Towards Benchmarking Scene Background Initialization](https://mlsanity.com/paper/1506.04051/) (2015)
- [Quadruplet Network with One-Shot Learning for Fast Visual Object Tracking](https://mlsanity.com/paper/1705.07222/) (2017)
- [A General Framework for Edited Video and Raw Video Summarization](https://mlsanity.com/paper/1904.10669/) (2019)
- [Moving Objects Detection with a Moving Camera: A Comprehensive Review](https://mlsanity.com/paper/2001.05238/) (2020)
- [Unsupervised Deep Context Prediction for Background Foreground Separation](https://mlsanity.com/paper/1805.07903/) (2018)
- [Combining Background Subtraction Algorithms with Convolutional Neural Network](https://mlsanity.com/paper/1807.02080/) (2018)
- [Detection of Collision-Prone Vehicle Behavior at Intersections using Siamese Interaction LSTM](https://mlsanity.com/paper/1912.04801/) (2019)
- [Move to See Better: Self-Improving Embodied Object Detection](https://mlsanity.com/paper/2012.00057/) (2020)
- [Challenges in Time-Stamp Aware Anomaly Detection in Traffic Videos](https://mlsanity.com/paper/1906.04574/) (2019)

## Built on by (most cited first, 9 of 9)

- [Autoencoder-based background reconstruction and foreground segmentation with background noise estimation](https://mlsanity.com/paper/2112.08001/) (2021)
- [Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentation](https://mlsanity.com/paper/2304.09949/) (2023)
- [rSVDdpd: A Robust Scalable Video Surveillance Background Modelling Algorithm](https://mlsanity.com/paper/2109.10680/) (2021)
- [SIDAR: Synthetic Image Dataset for Alignment & Restoration](https://mlsanity.com/paper/2305.12036/) (2023)
- [An exploration of the performances achievable by combining unsupervised background subtraction algorithms](https://mlsanity.com/paper/2202.12563/) (2022)
- [SoftMatch Distance: A Novel Distance for Weakly-Supervised Trend Change Detection in Bi-Temporal Images](https://mlsanity.com/paper/2303.04737/) (2023)
- [Deep Neural Networks in Video Human Action Recognition: A Review](https://mlsanity.com/paper/2305.15692/) (2023)
- [Multimodal Feature Fusion Network with Text Difference Enhancement for Remote Sensing Change Detection](https://mlsanity.com/paper/2509.03961/) (2025)
- [DeepATLAS: One-Shot Localization for Biomedical Data](https://mlsanity.com/paper/2402.09587/) (2024)

## Abstract (arXiv)

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions.