# Spatio-temporal Data Augmentation for Visual Surveillance

arXiv: https://arxiv.org/abs/2101.09895 · HTML: https://mlsanity.com/paper/2101.09895/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [Learning Multi-scale Features for Foreground Segmentation](https://mlsanity.com/paper/1808.01477/) (2018)
- [Foreground Segmentation Using a Triplet Convolutional Neural Network for Multiscale Feature Encoding](https://mlsanity.com/paper/1801.02225/) (2018)
- [Towards Benchmarking Scene Background Initialization](https://mlsanity.com/paper/1506.04051/) (2015)
- [Background Subtraction with Real-time Semantic Segmentation](https://mlsanity.com/paper/1811.10020/) (2018)
- [Joint Background Reconstruction and Foreground Segmentation via A Two-stage Convolutional Neural Network](https://mlsanity.com/paper/1707.07584/) (2017)

## Built on by (most cited first, 1 of 1)

- [Extending Temporal Data Augmentation for Video Action Recognition](https://mlsanity.com/paper/2211.04888/) (2022)

## Abstract (arXiv)

Visual surveillance aims to stably detect a foreground object using a continuous image acquired from a fixed camera. Recent deep learning methods based on supervised learning show superior performance compared to classical background subtraction algorithms. However, there is still a room for improvement in static foreground, dynamic background, hard shadow, illumination changes, camouflage, etc. In addition, most of the deep learning-based methods operates well on environments similar to training. If the testing environments are different from training ones, their performance degrades. As a result, additional training on those operating environments is required to ensure a good performance. Our previous work which uses spatio-temporal input data consisted of a number of past images, background images and current image showed promising results in different environments from training, although it uses a simple U-NET structure. In this paper, we propose a data augmentation technique suitable for visual surveillance for additional performance improvement using the same network used in our previous work. In deep learning, most data augmentation techniques deal with spatial-level data augmentation techniques for use in image classification and object detection. In this paper, we propose a new method of data augmentation in the spatio-temporal dimension suitable for our previous work. Two data augmentation methods of adjusting background model images and past images are proposed. Through this, it is shown that performance can be improved in difficult areas such as static foreground and ghost objects, compared to previous studies. Through quantitative and qualitative evaluation using SBI, LASIESTA, and our own dataset, we show that it gives superior performance compared to deep learning-based algorithms and background subtraction algorithms.