# A Scalable and Generalized Deep Learning Framework for Anomaly Detection in Surveillance Videos

arXiv: https://arxiv.org/abs/2408.00792 · HTML: https://mlsanity.com/paper/2408.00792/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Model](https://mlsanity.com/paper/2202.02212/) (2022)
- [Suspicious Behavior Detection on Shoplifting Cases for Crime Prevention by Using 3D Convolutional Neural Networks](https://mlsanity.com/paper/2005.02142/) (2020)

## Built on by (most cited first, 2 of 2)

- [Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervision](https://mlsanity.com/paper/2603.13306/) (2026)
- [Human-Centric Anomaly Detection in Surveillance Videos Using YOLO-World and Spatio-Temporal Deep Learning](https://mlsanity.com/paper/2510.22056/) (2025)

## Abstract (arXiv)

Anomaly detection in videos is challenging due to the complexity, noise, and diverse nature of activities such as violence, shoplifting, and vandalism. While deep learning (DL) has shown excellent performance in this area, existing approaches have struggled to apply DL models across different anomaly tasks without extensive retraining. This repeated retraining is time-consuming, computationally intensive, and unfair. To address this limitation, a new DL framework is introduced in this study, consisting of three key components: transfer learning to enhance feature generalization, model fusion to improve feature representation, and multi-task classification to generalize the classifier across multiple tasks without training from scratch when new task is introduced. The framework's main advantage is its ability to generalize without requiring retraining from scratch for each new task. Empirical evaluations demonstrate the framework's effectiveness, achieving an accuracy of 97.99% on the RLVS dataset (violence detection), 83.59% on the UCF dataset (shoplifting detection), and 88.37% across both datasets using a single classifier without retraining. Additionally, when tested on an unseen dataset, the framework achieved an accuracy of 87.25%. The study also utilizes two explainability tools to identify potential biases, ensuring robustness and fairness. This research represents the first successful resolution of the generalization issue in anomaly detection, marking a significant advancement in the field.