# BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos

arXiv: https://arxiv.org/abs/1907.11371 · HTML: https://mlsanity.com/paper/1907.11371/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2019.

## Built on by (most cited first, 22 of 22)

- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing](https://mlsanity.com/paper/2110.15128/) (2021)
- [BSUV-Net 2.0: Spatio-Temporal Data Augmentations for Video-Agnostic Supervised Background Subtraction](https://mlsanity.com/paper/2101.09585/) (2021)
- [SOAR: Scene-debiasing Open-set Action Recognition](https://mlsanity.com/paper/2309.01265/) (2023)
- [Universal Background Subtraction based on Arithmetic Distribution Neural Network](https://mlsanity.com/paper/2104.08390/) (2021)
- [An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs](https://mlsanity.com/paper/2105.01342/) (2021)
- [ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection](https://mlsanity.com/paper/2303.14679/) (2023)
- [FactorMatte: Redefining Video Matting for Re-Composition Tasks](https://mlsanity.com/paper/2211.02145/) (2022)
- [Instance-Level Moving Object Segmentation from a Single Image with Events](https://mlsanity.com/paper/2502.12975/) (2025)
- [LangGas: Introducing Language in Selective Zero-Shot Background Subtraction for Semi-Transparent Gas Leak Detection with a New Dataset](https://mlsanity.com/paper/2503.02910/) (2025)
- [Deep Learning for Robust Motion Segmentation with Non-Static Cameras](https://mlsanity.com/paper/2102.10929/) (2021)
- [Graph CNN for Moving Object Detection in Complex Environments from Unseen Videos](https://mlsanity.com/paper/2207.06440/) (2022)
- [Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentation](https://mlsanity.com/paper/2304.09949/) (2023)
- [Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtraction](https://mlsanity.com/paper/2309.15576/) (2023)
- [Dynamic Background Subtraction by Generative Neural Networks](https://mlsanity.com/paper/2202.05336/) (2022)
- [SAEN-BGS: Energy-Efficient Spiking AutoEncoder Network for Background Subtraction](https://mlsanity.com/paper/2505.07336/) (2025)
- [Reducing False Alarms in Video Surveillance by Deep Feature Statistical Modeling](https://mlsanity.com/paper/2307.04159/) (2023)
- [Open Set Recognition for Endoscopic Image Classification: A Deep Learning Approach on the Kvasir Dataset](https://mlsanity.com/paper/2506.18284/) (2025)
- [Saliency detection with moving camera via background model completion](https://mlsanity.com/paper/2111.01681/) (2021)
- [Weakly Supervised Realtime Dynamic Background Subtraction](https://mlsanity.com/paper/2303.02857/) (2023)
- [Compound and Parallel Modes of Tropical Convolutional Neural Networks](https://mlsanity.com/paper/2504.06881/) (2025)
- [Flow-CDNet: A Novel Network for Detecting Both Slow and Fast Changes in Bitemporal Images](https://mlsanity.com/paper/2507.02307/) (2025)

## Abstract (arXiv)

Background subtraction is a basic task in computer vision and video processing often applied as a pre-processing step for object tracking, people recognition, etc. Recently, a number of successful background-subtraction algorithms have been proposed, however nearly all of the top-performing ones are supervised. Crucially, their success relies upon the availability of some annotated frames of the test video during training. Consequently, their performance on completely "unseen" videos is undocumented in the literature. In this work, we propose a new, supervised, background-subtraction algorithm for unseen videos (BSUV-Net) based on a fully-convolutional neural network. The input to our network consists of the current frame and two background frames captured at different time scales along with their semantic segmentation maps. In order to reduce the chance of overfitting, we also introduce a new data-augmentation technique which mitigates the impact of illumination difference between the background frames and the current frame. On the CDNet-2014 dataset, BSUV-Net outperforms state-of-the-art algorithms evaluated on unseen videos in terms of several metrics including F-measure, recall and precision.