mlsanityData to 14 Oct 2026

Autoencoder-based background reconstruction and foreground segmentation with background noise estimation

arXiv 2112.08001 ↗First posted 15 Dec 2021cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Even after decades of research, dynamic scene background reconstruction and foreground object segmentation are still considered as open problems due various challenges such as illumination changes, camera movements, or background noise caused by air turbulence or moving trees. We propose in this paper to model the background of a frame sequence as a low dimensional manifold using an autoencoder and compare the reconstructed background provided by this autoencoder with the original image to compute the foreground/background segmentation masks. The main novelty of the proposed model is that the autoencoder is also trained to predict the background noise, which allows to compute for each frame a pixel-dependent threshold to perform the foreground segmentation. Although the proposed model does not use any temporal or motion information, it exceeds the state of the art for unsupervised background subtraction on the CDnet 2014 and LASIESTA datasets, with a significant improvement on videos where the camera is moving. It is also able to perform background reconstruction on some non-video image datasets.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 25 of its 81 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

25 of 81 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  3. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  4. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  5. 2021Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimationcited by 116
  6. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  7. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  8. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  9. 2018ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stackingcited by 55
  10. 2021CoordiNet: uncertainty-aware pose regressor for reliable vehicle localizationcited by 34
  11. 2020Unsupervised object-centric video generation and decomposition in 3Dcited by 31
  12. 2019Background Subtraction in Real Applications: Challenges, Current Models and Future Directionscited by 29
13 more
  1. 2018Learning Multi-scale Features for Foreground Segmentationcited by 27
  2. 2018Foreground Segmentation Using a Triplet Convolutional Neural Network for Multiscale Feature Encodingcited by 16
  3. 2021APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulationcited by 16
  4. 2021BSUV-Net 2.0: Spatio-Temporal Data Augmentations for Video-Agnostic Supervised Background Subtractioncited by 15
  5. 2020GraphBGS: Background Subtraction via Recovery of Graph Signalscited by 9
  6. 2021An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needscited by 9
  7. 2018Unsupervised Deep Context Prediction for Background Foreground Separationcited by 7
  8. 2018WisenetMD: Motion Detection Using Dynamic Background Region Analysiscited by 7
  9. 2015How Far Can You Get By Combining Change Detection Algorithms?cited by 6
  10. 2018Background Subtraction with Real-time Semantic Segmentationcited by 6
  11. 2015Background Subtraction via Generalized Fused Lasso Foreground Modelingcited by 4
  12. 2020G-LBM:Generative Low-dimensional Background Model Estimation from Video Sequencescited by 3
  13. 2019DeepPBM: Deep Probabilistic Background Model Estimation from Video Sequencescited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 1.

7 papers.

  1. 2022Unsupervised Multi-object Segmentation Using Attention and Soft-argmaxcited by 11
  2. 2023ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selectioncited by 9
  3. 2023Benchmarks for Physical Reasoning AIcited by 8
  4. 2023Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentationcited by 4
  5. 2022Application Of ADNN For Background Subtraction In Smart Surveillance Systemcited by 0
  6. 2024Improving Prediction Accuracy of Semantic Segmentation Methods Using Convolutional Autoencoder Based Pre-processing Layerscited by 0
  7. 2025Quaternion-Based Robust PCA for Efficient Moving Target Detection and Background Recovery in Color Videoscited by 0