# Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentation

arXiv: https://arxiv.org/abs/2304.09949 · HTML: https://mlsanity.com/paper/2304.09949/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 100 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 19 of 19)

- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Learning Multi-scale Features for Foreground Segmentation](https://mlsanity.com/paper/1808.01477/) (2018)
- [Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation](https://mlsanity.com/paper/1811.05255/) (2018)
- [Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery](https://mlsanity.com/paper/1711.09492/) (2017)
- [BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos](https://mlsanity.com/paper/1907.11371/) (2019)
- [Compressed Dynamic Mode Decomposition for Background Modeling](https://mlsanity.com/paper/1512.04205/) (2015)
- [BSUV-Net 2.0: Spatio-Temporal Data Augmentations for Video-Agnostic Supervised Background Subtraction](https://mlsanity.com/paper/2101.09585/) (2021)
- [Towards Benchmarking Scene Background Initialization](https://mlsanity.com/paper/1506.04051/) (2015)
- [Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset](https://mlsanity.com/paper/1511.01245/) (2015)
- [Robust PCA via Nonconvex Rank Approximation](https://mlsanity.com/paper/1511.05261/) (2015)
- [Universal Background Subtraction based on Arithmetic Distribution Neural Network](https://mlsanity.com/paper/2104.08390/) (2021)
- [An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs](https://mlsanity.com/paper/2105.01342/) (2021)
- [ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection](https://mlsanity.com/paper/2303.14679/) (2023)
- [Unsupervised Deep Context Prediction for Background Foreground Separation](https://mlsanity.com/paper/1805.07903/) (2018)
- [Combining Background Subtraction Algorithms with Convolutional Neural Network](https://mlsanity.com/paper/1807.02080/) (2018)
- [Autoencoder-based background reconstruction and foreground segmentation with background noise estimation](https://mlsanity.com/paper/2112.08001/) (2021)
- [How Far Can You Get By Combining Change Detection Algorithms?](https://mlsanity.com/paper/1505.02921/) (2015)
- [Online Illumination Invariant Moving Object Detection by Generative Neural Network](https://mlsanity.com/paper/1808.01066/) (2018)
- [Graph CNN for Moving Object Detection in Complex Environments from Unseen Videos](https://mlsanity.com/paper/2207.06440/) (2022)

## Built on by (most cited first, 4 of 4)

- [Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtraction](https://mlsanity.com/paper/2309.15576/) (2023)
- [Affine-Transformation-Invariant Image Classification by Differentiable Arithmetic Distribution Module](https://mlsanity.com/paper/2309.00752/) (2023)
- [Quaternion-Based Robust PCA for Efficient Moving Target Detection and Background Recovery in Color Videos](https://mlsanity.com/paper/2507.19730/) (2025)
- [Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection](https://mlsanity.com/paper/2606.24092/) (2026)

## Abstract (arXiv)

The goal of moving object segmentation is separating moving objects from stationary backgrounds in videos. One major challenge in this problem is how to develop a universal model for videos from various natural scenes since previous methods are often effective only in specific scenes. In this paper, we propose a method called Learning Temporal Distribution and Spatial Correlation (LTS) that has the potential to be a general solution for universal moving object segmentation. In the proposed approach, the distribution from temporal pixels is first learned by our Defect Iterative Distribution Learning (DIDL) network for a scene-independent segmentation. Notably, the DIDL network incorporates the use of an improved product distribution layer that we have newly derived. Then, the Stochastic Bayesian Refinement (SBR) Network, which learns the spatial correlation, is proposed to improve the binary mask generated by the DIDL network. Benefiting from the scene independence of the temporal distribution and the accuracy improvement resulting from the spatial correlation, the proposed approach performs well for almost all videos from diverse and complex natural scenes with fixed parameters. Comprehensive experiments on standard datasets including LASIESTA, CDNet2014, BMC, SBMI2015 and 128 real world videos demonstrate the superiority of proposed approach compared to state-of-the-art methods with or without the use of deep learning networks. To the best of our knowledge, this work has high potential to be a general solution for moving object segmentation in real world environments. The code and real-world videos can be found on GitHub https://github.com/guanfangdong/LTS-UniverisalMOS.