# A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video

arXiv: https://arxiv.org/abs/2008.12328 · HTML: https://mlsanity.com/paper/2008.12328/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2020.

## Built on by (most cited first, 30 of 86)

- [Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection](https://mlsanity.com/paper/2111.09099/) (2021)
- [UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection](https://mlsanity.com/paper/2111.08644/) (2021)
- [Video Anomaly Detection by Solving Decoupled Spatio-Temporal Jigsaw Puzzles](https://mlsanity.com/paper/2207.10172/) (2022)
- [Normalizing Flows for Human Pose Anomaly Detection](https://mlsanity.com/paper/2211.10946/) (2022)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [SSMTL++: Revisiting Self-Supervised Multi-Task Learning for Video Anomaly Detection](https://mlsanity.com/paper/2207.08003/) (2022)
- [Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts](https://mlsanity.com/paper/2408.05905/) (2024)
- [Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detection](https://mlsanity.com/paper/2303.13051/) (2023)
- [Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors](https://mlsanity.com/paper/2306.12041/) (2023)
- [Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection](https://mlsanity.com/paper/2209.12148/) (2022)
- [Learning Not to Reconstruct Anomalies](https://mlsanity.com/paper/2110.09742/) (2021)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection](https://mlsanity.com/paper/2403.14497/) (2024)
- [Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinement](https://mlsanity.com/paper/2108.01975/) (2021)
- [Networking Systems for Video Anomaly Detection: A Tutorial and Survey](https://mlsanity.com/paper/2405.10347/) (2024)
- [Cross-Domain Video Anomaly Detection without Target Domain Adaptation](https://mlsanity.com/paper/2212.07010/) (2022)
- [EVAL: Explainable Video Anomaly Localization](https://mlsanity.com/paper/2212.07900/) (2022)
- [Context Recovery and Knowledge Retrieval: A Novel Two-Stream Framework for Video Anomaly Detection](https://mlsanity.com/paper/2209.02899/) (2022)
- [Learnable Locality-Sensitive Hashing for Video Anomaly Detection](https://mlsanity.com/paper/2111.07839/) (2021)
- [FastAno: Fast Anomaly Detection via Spatio-temporal Patch Transformation](https://mlsanity.com/paper/2106.08613/) (2021)
- [Contracting Skeletal Kinematics for Human-Related Video Anomaly Detection](https://mlsanity.com/paper/2301.09489/) (2023)
- [A Survey of Visual Sensory Anomaly Detection](https://mlsanity.com/paper/2202.07006/) (2022)
- [Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach](https://mlsanity.com/paper/2311.16514/) (2023)
- [Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videos](https://mlsanity.com/paper/2401.03522/) (2024)
- [Graph-Jigsaw Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection](https://mlsanity.com/paper/2403.12172/) (2024)
- [PseudoBound: Limiting the anomaly reconstruction capability of one-class classifiers using pseudo anomalies](https://mlsanity.com/paper/2303.10704/) (2023)
- [ComplexVAD: Detecting Interaction Anomalies in Video](https://mlsanity.com/paper/2501.09733/) (2025)
- [Track Any Anomalous Object: A Granular Video Anomaly Detection Pipeline](https://mlsanity.com/paper/2506.05175/) (2025)

## Abstract (arXiv)

Abnormal event detection in video is a complex computer vision problem that has attracted significant attention in recent years. The complexity of the task arises from the commonly-adopted definition of an abnormal event, that is, a rarely occurring event that typically depends on the surrounding context. Following the standard formulation of abnormal event detection as outlier detection, we propose a background-agnostic framework that learns from training videos containing only normal events. Our framework is composed of an object detector, a set of appearance and motion auto-encoders, and a set of classifiers. Since our framework only looks at object detections, it can be applied to different scenes, provided that normal events are defined identically across scenes and that the single main factor of variation is the background. To overcome the lack of abnormal data during training, we propose an adversarial learning strategy for the auto-encoders. We create a scene-agnostic set of out-of-domain pseudo-abnormal examples, which are correctly reconstructed by the auto-encoders before applying gradient ascent on the pseudo-abnormal examples. We further utilize the pseudo-abnormal examples to serve as abnormal examples when training appearance-based and motion-based binary classifiers to discriminate between normal and abnormal latent features and reconstructions. We compare our framework with the state-of-the-art methods on four benchmark data sets, using various evaluation metrics. Compared to existing methods, the empirical results indicate that our approach achieves favorable performance on all data sets. In addition, we provide region-based and track-based annotations for two large-scale abnormal event detection data sets from the literature, namely ShanghaiTech and Subway.