# Learning Spatial-Temporal Regularized Tensor Sparse RPCA for Background Subtraction

arXiv: https://arxiv.org/abs/2309.15576 · HTML: https://mlsanity.com/paper/2309.15576/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 90 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 22 of 22)

- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Tensor Robust Principal Component Analysis with A New Tensor Nuclear Norm](https://mlsanity.com/paper/1804.03728/) (2018)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGB](https://mlsanity.com/paper/2107.05287/) (2021)
- [Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization](https://mlsanity.com/paper/1708.04181/) (2017)
- [Robust Low-rank Tensor Recovery: Models and Algorithms](https://mlsanity.com/paper/1311.6182/) (2013)
- [Background Subtraction in Real Applications: Challenges, Current Models and Future Directions](https://mlsanity.com/paper/1901.03577/) (2019)
- [Deep Neural Network Concepts for Background Subtraction: A Systematic Review and Comparative Evaluation](https://mlsanity.com/paper/1811.05255/) (2018)
- [Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery](https://mlsanity.com/paper/1711.09492/) (2017)
- [BSUV-Net: A Fully-Convolutional Neural Network for Background Subtraction of Unseen Videos](https://mlsanity.com/paper/1907.11371/) (2019)
- [Subspace Learning and Imputation for Streaming Big Data Matrices and Tensors](https://mlsanity.com/paper/1404.4667/) (2014)
- [Total Variation Regularized Tensor RPCA for Background Subtraction from Compressive Measurements](https://mlsanity.com/paper/1503.01868/) (2015)
- [Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset](https://mlsanity.com/paper/1511.01245/) (2015)
- [Universal Background Subtraction based on Arithmetic Distribution Neural Network](https://mlsanity.com/paper/2104.08390/) (2021)
- [Fast and Provable Tensor Robust Principal Component Analysis via Scaled Gradient Descent](https://mlsanity.com/paper/2206.09109/) (2022)
- [Robust Principal Component Analysis on Graphs](https://mlsanity.com/paper/1504.06151/) (2015)
- [Robust Online Matrix Factorization for Dynamic Background Subtraction](https://mlsanity.com/paper/1705.10000/) (2017)
- [ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection](https://mlsanity.com/paper/2303.14679/) (2023)
- [Moving Object Detection under Discontinuous Change in Illumination Using Tensor Low-Rank and Invariant Sparse Decomposition](https://mlsanity.com/paper/1904.03175/) (2019)
- [Learning Temporal Distribution and Spatial Correlation Towards Universal Moving Object Segmentation](https://mlsanity.com/paper/2304.09949/) (2023)
- [COROLA: A Sequential Solution to Moving Object Detection Using Low-rank Approximation](https://mlsanity.com/paper/1505.03566/) (2015)

## Built on by (most cited first, 4 of 4)

- [FreePCA: Integrating Consistency Information across Long-short Frames in Training-free Long Video Generation via Principal Component Analysis](https://mlsanity.com/paper/2505.01172/) (2025)
- [Outlier-aware Tensor Robust Principal Component Analysis with Self-guided Data Augmentation](https://mlsanity.com/paper/2504.18323/) (2025)
- [Quaternion-Based Robust PCA for Efficient Moving Target Detection and Background Recovery in Color Videos](https://mlsanity.com/paper/2507.19730/) (2025)
- [Robust Principal Component Completion](https://mlsanity.com/paper/2603.25132/) (2026)

## Abstract (arXiv)

Video background subtraction is one of the fundamental problems in computer vision that aims to segment all moving objects. Robust principal component analysis has been identified as a promising unsupervised paradigm for background subtraction tasks in the last decade thanks to its competitive performance in a number of benchmark datasets. Tensor robust principal component analysis variations have improved background subtraction performance further. However, because moving object pixels in the sparse component are treated independently and do not have to adhere to spatial-temporal structured-sparsity constraints, performance is reduced for sequences with dynamic backgrounds, camouflaged, and camera jitter problems. In this work, we present a spatial-temporal regularized tensor sparse RPCA algorithm for precise background subtraction. Within the sparse component, we impose spatial-temporal regularizations in the form of normalized graph-Laplacian matrices. To do this, we build two graphs, one across the input tensor spatial locations and the other across its frontal slices in the time domain. While maximizing the objective function, we compel the tensor sparse component to serve as the spatiotemporal eigenvectors of the graph-Laplacian matrices. The disconnected moving object pixels in the sparse component are preserved by the proposed graph-based regularizations since they both comprise of spatiotemporal subspace-based structure. Additionally, we propose a unique objective function that employs batch and online-based optimization methods to jointly maximize the background-foreground and spatial-temporal regularization components. Experiments are performed on six publicly available background subtraction datasets that demonstrate the superior performance of the proposed algorithm compared to several existing methods. Our source code will be available very soon.