# A Multi-Task Deep Learning Framework for Real-Time Intelligent Video Surveillance with Temporal Event Validation

arXiv: https://arxiv.org/abs/2607.03131 · HTML: https://mlsanity.com/paper/2607.03131/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 48 references that are in the map.
- First posted on arXiv 3 Jul 2026.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devices](https://mlsanity.com/paper/1804.07573/) (2018)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [RWF-2000: An Open Large Scale Video Database for Violence Detection](https://mlsanity.com/paper/1911.05913/) (2019)
- [Real-time Convolutional Neural Networks for Emotion and Gender Classification](https://mlsanity.com/paper/1710.07557/) (2017)
- [The Security of IP-based Video Surveillance Systems](https://mlsanity.com/paper/1910.10749/) (2019)
- [Real-Time Threat Detection from Surveillance Cameras using Machine Learning](https://mlsanity.com/paper/2606.05708/) (2026)

## Built on by (most cited first, 3 of 3)

- [ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding](https://mlsanity.com/paper/2607.28312/) (2026)
- [Beyond Retrieval: Progressive Latent Memory Evolution for Streaming Video Understanding](https://mlsanity.com/paper/2609.04131/) (2026)
- [trACT: temporal revelation Airborne Camera Trap](https://mlsanity.com/paper/2610.09417/) (2026)

## Abstract (arXiv)

Modern video surveillance systems generate far more video streams than human operators can effectively monitor, making automated analysis essential for timely detection of security events. This paper presents a unified multi-task deep learning framework that simultaneously performs face recognition with zone-based authorization, automatic license plate recognition, weapon detection, fire and smoke detection, and human action recognition on a shared GPU platform. Among the integrated modules, two task-specific deep-learning models are proposed in this work to address scenarios that are insufficiently represented in publicly available datasets: a single-class weapon detector fine-tuned on a merged and relabeled dataset, achieving a mean average precision (mAP@0.5) of 0.947, and a SlowFast-R50 action recognition model trained on a purpose-built vandalism dataset comprising 614 video clips, achieving 94.33% classification accuracy. To improve robustness in continuous video, all detection modules are integrated into a temporal event-validation architecture based on multi-frame confirmation, confidence-weighted voting, and cascaded filtering, transforming frame-level predictions into reliable security events. Each module is evaluated independently on established public datasets (LFW, D-Fire, FIRESENSE, and UCF-Crime), followed by integrated end-to-end system evaluation. The proposed temporal validation strategy reduces the fire and smoke false-alarm rate from 52% to 4% and improves video license plate exact-match accuracy from 66.7% to 81.8%, while the complete framework maintains real-time operation with a per-frame latency below 100 ms on commodity hardware. These results demonstrate that combining specialized deep-learning models with temporal event validation provides an effective and practical solution for reliable real-time intelligent video surveillance.