# VADMamba: Exploring State Space Models for Fast Video Anomaly Detection

arXiv: https://arxiv.org/abs/2503.21169 · HTML: https://mlsanity.com/paper/2503.21169/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Time series forecasting with SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Time series forecasting with SSMs](https://mlsanity.com/ssm/time-series-forecasting-with-ssms/)

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [VM-UNet: Vision Mamba UNet for Medical Image Segmentation](https://mlsanity.com/paper/2402.02491/) (2024)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection](https://mlsanity.com/paper/2404.06564/) (2024)
- [Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video Events](https://mlsanity.com/paper/2008.11988/) (2020)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Video Event Restoration Based on Keyframes for Video Anomaly Detection](https://mlsanity.com/paper/2304.05112/) (2023)
- [Real-Time Anomaly Detection and Localization in Crowded Scenes](https://mlsanity.com/paper/1511.06936/) (2015)
- [Learning Appearance-motion Normality for Video Anomaly Detection](https://mlsanity.com/paper/2207.13361/) (2022)
- [Bidirectional skip-frame prediction for video anomaly detection with intra-domain disparity-driven attention](https://mlsanity.com/paper/2407.15424/) (2024)
- [Appearance Blur-driven AutoEncoder and Motion-guided Memory Module for Video Anomaly Detection](https://mlsanity.com/paper/2409.17608/) (2024)

## Built on by (most cited first, 8 of 8)

- [Subject Information Extraction for Novelty Detection with Domain Shifts](https://mlsanity.com/paper/2504.21247/) (2025)
- [The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM](https://mlsanity.com/paper/2507.21649/) (2025)
- [Video Anomaly Detection with Semantics-Aware Information Bottleneck](https://mlsanity.com/paper/2506.02535/) (2025)
- [M2S2L: Mamba-based Multi-Scale Spatial-temporal Learning for Video Anomaly Detection](https://mlsanity.com/paper/2511.05564/) (2025)
- [VADMamba++: Efficient Video Anomaly Detection via Hybrid Modeling in Grayscale Space](https://mlsanity.com/paper/2604.00360/) (2026)
- [FTDMamba: Frequency-Assisted Temporal Dilation Mamba for Unmanned Aerial Vehicle Video Anomaly Detection](https://mlsanity.com/paper/2601.11254/) (2026)
- [Forward Consistency Learning with Gated Context Aggregation for Video Anomaly Detection](https://mlsanity.com/paper/2601.18135/) (2026)
- [Strictly Causal Streaming Video Anomaly Detection with a Theoretically-Grounded State-Space Core](https://mlsanity.com/paper/2608.24810/) (2026)

## Abstract (arXiv)

Video anomaly detection (VAD) methods are mostly CNN-based or Transformer-based, achieving impressive results, but the focus on detection accuracy often comes at the expense of inference speed. The emergence of state space models in computer vision, exemplified by the Mamba model, demonstrates improved computational efficiency through selective scans and showcases the great potential for long-range modeling. Our study pioneers the application of Mamba to VAD, dubbed VADMamba, which is based on multi-task learning for frame prediction and optical flow reconstruction. Specifically, we propose the VQ-Mamba Unet (VQ-MaU) framework, which incorporates a Vector Quantization (VQ) layer and Mamba-based Non-negative Visual State Space (NVSS) block. Furthermore, two individual VQ-MaU networks separately predict frames and reconstruct corresponding optical flows, further boosting accuracy through a clip-level fusion evaluation strategy. Experimental results validate the efficacy of the proposed VADMamba across three benchmark datasets, demonstrating superior performance in inference speed compared to previous work. Code is available at https://github.com/jLooo/VADMamba.