# BSUV-Net 2.0: Spatio-Temporal Data Augmentations for Video-Agnostic Supervised Background Subtraction

arXiv: https://arxiv.org/abs/2101.09585 · HTML: https://mlsanity.com/paper/2101.09585/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 23 Jan 2021.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/1905.00737/) (2019)
- [Improving Deep Learning using Generic Data Augmentation](https://mlsanity.com/paper/1708.06020/) (2017)
- [Background Subtraction in Real Applications: Challenges, Current Models and Future Directions](https://mlsanity.com/paper/1901.03577/) (2019)
- [Learning Multi-scale Features for Foreground Segmentation](https://mlsanity.com/paper/1808.01477/) (2018)
- [Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation](https://mlsanity.com/paper/1811.05255/) (2018)
- [BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos](https://mlsanity.com/paper/1907.11371/) (2019)
- [Foreground Segmentation Using a Triplet Convolutional Neural Network for Multiscale Feature Encoding](https://mlsanity.com/paper/1801.02225/) (2018)
- [Towards Benchmarking Scene Background Initialization](https://mlsanity.com/paper/1506.04051/) (2015)
- [GraphBGS: Background Subtraction via Recovery of Graph Signals](https://mlsanity.com/paper/2001.06404/) (2020)
- [WisenetMD: Motion Detection Using Dynamic Background Region Analysis](https://mlsanity.com/paper/1805.09277/) (2018)
- [How Far Can You Get By Combining Change Detection Algorithms?](https://mlsanity.com/paper/1505.02921/) (2015)
- [Background Subtraction with Real-time Semantic Segmentation](https://mlsanity.com/paper/1811.10020/) (2018)
- [3DFR: A Swift 3D Feature Reductionist Framework for Scene Independent Change Detection](https://mlsanity.com/paper/1912.11891/) (2019)
- [Real-Time Semantic Background Subtraction](https://mlsanity.com/paper/2002.04993/) (2020)

## Built on by (most cited first, 15 of 15)

- [Universal Background Subtraction based on Arithmetic Distribution Neural Network](https://mlsanity.com/paper/2104.08390/) (2021)
- [ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection](https://mlsanity.com/paper/2303.14679/) (2023)
- [Autoencoder-based background reconstruction and foreground segmentation with background noise estimation](https://mlsanity.com/paper/2112.08001/) (2021)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [LangGas: Introducing Language in Selective Zero-Shot Background Subtraction for Semi-Transparent Gas Leak Detection with a New Dataset](https://mlsanity.com/paper/2503.02910/) (2025)
- [Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentation](https://mlsanity.com/paper/2304.09949/) (2023)
- [Dynamic Background Subtraction by Generative Neural Networks](https://mlsanity.com/paper/2202.05336/) (2022)
- [Inductive Graph Neural Networks for Moving Object Segmentation](https://mlsanity.com/paper/2305.09585/) (2023)
- [SAEN-BGS: Energy-Efficient Spiking AutoEncoder Network for Background Subtraction](https://mlsanity.com/paper/2505.07336/) (2025)
- [DeePaste -- Inpainting for Pasting](https://mlsanity.com/paper/2112.10600/) (2021)
- [Reducing False Alarms in Video Surveillance by Deep Feature Statistical Modeling](https://mlsanity.com/paper/2307.04159/) (2023)
- [Saliency detection with moving camera via background model completion](https://mlsanity.com/paper/2111.01681/) (2021)
- [Weakly Supervised Realtime Dynamic Background Subtraction](https://mlsanity.com/paper/2303.02857/) (2023)
- [TSOM: Small Object Motion Detection Neural Network Inspired by Avian Visual Circuit](https://mlsanity.com/paper/2404.00855/) (2024)
- [Event Detection in Videos: A Framework for the Development of New Methods](https://mlsanity.com/paper/2607.04372/) (2026)

## Abstract (arXiv)

Background subtraction (BGS) is a fundamental video processing task which is a key component of many applications. Deep learning-based supervised algorithms achieve very good perforamnce in BGS, however, most of these algorithms are optimized for either a specific video or a group of videos, and their performance decreases dramatically when applied to unseen videos. Recently, several papers addressed this problem and proposed video-agnostic supervised BGS algorithms. However, nearly all of the data augmentations used in these algorithms are limited to the spatial domain and do not account for temporal variations that naturally occur in video data. In this work, we introduce spatio-temporal data augmentations and apply them to one of the leading video-agnostic BGS algorithms, BSUV-Net. We also introduce a new cross-validation training and evaluation strategy for the CDNet-2014 dataset that makes it possible to fairly and easily compare the performance of various video-agnostic supervised BGS algorithms. Our new model trained using the proposed data augmentations, named BSUV-Net 2.0, significantly outperforms state-of-the-art algorithms evaluated on unseen videos of CDNet-2014. We also evaluate the cross-dataset generalization capacity of BSUV-Net 2.0 by training it solely on CDNet-2014 videos and evaluating its performance on LASIESTA dataset. Overall, BSUV-Net 2.0 provides a ~5% improvement in the F-score over state-of-the-art methods on unseen videos of CDNet-2014 and LASIESTA datasets. Furthermore, we develop a real-time variant of our model, that we call Fast BSUV-Net 2.0, whose performance is close to the state of the art.