# Multi-Task Learning based Video Anomaly Detection with Attention

arXiv: https://arxiv.org/abs/2210.07697 · HTML: https://mlsanity.com/paper/2210.07697/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 63 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 30 of 35)

- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Uninformed Students: Student-Teacher Anomaly Detection with Discriminative Latent Embeddings](https://mlsanity.com/paper/1911.02357/) (2019)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Anomaly Detection in Video via Self-Supervised and Multi-Task Learning](https://mlsanity.com/paper/2011.07491/) (2020)
- [Detecting the Unexpected via Image Resynthesis](https://mlsanity.com/paper/1904.07595/) (2019)
- [A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction](https://mlsanity.com/paper/2108.06852/) (2021)
- [Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection](https://mlsanity.com/paper/2111.09099/) (2021)
- [Anomaly Detection in Video Sequence with Appearance-Motion Correspondence](https://mlsanity.com/paper/1908.06351/) (2019)
- [Adversarially Learned Anomaly Detection](https://mlsanity.com/paper/1812.02288/) (2018)
- [Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Video](https://mlsanity.com/paper/1812.04960/) (2018)
- [Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos](https://mlsanity.com/paper/1903.03295/) (2019)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [Unsupervised Learning of Object Structure and Dynamics from Videos](https://mlsanity.com/paper/1906.07889/) (2019)
- [A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video](https://mlsanity.com/paper/2008.12328/) (2020)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [Abnormal Event Detection in Videos using Spatiotemporal Autoencoder](https://mlsanity.com/paper/1701.01546/) (2017)
- [Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge](https://mlsanity.com/paper/1709.09121/) (2017)
- [Any-Shot Sequential Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/2004.02072/) (2020)
- [Future Frame Prediction Using Convolutional VRNN for Anomaly Detection](https://mlsanity.com/paper/1909.02168/) (2019)
- [Anomaly Detection in Video Using Predictive Convolutional Long Short-Term Memory Networks](https://mlsanity.com/paper/1612.00390/) (2016)
- [Multi-Contextual Predictions with Vision Transformer for Video Anomaly Detection](https://mlsanity.com/paper/2206.08568/) (2022)
- [Local Anomaly Detection in Videos using Object-Centric Adversarial Learning](https://mlsanity.com/paper/2011.06722/) (2020)

## Built on by (most cited first, 6 of 6)

- [Foundation Models and Transformers for Anomaly Detection: A Survey](https://mlsanity.com/paper/2507.15905/) (2025)
- [ALFred: An Active Learning Framework for Real-world Semi-supervised Anomaly Detection with Adaptive Thresholds](https://mlsanity.com/paper/2508.09058/) (2025)
- [From Offline to Periodic Adaptation for Pose-Based Shoplifting Detection in Real-world Retail Security](https://mlsanity.com/paper/2603.04723/) (2026)
- [Future Video Prediction from a Single Frame for Video Anomaly Detection](https://mlsanity.com/paper/2308.07783/) (2023)
- [VQ-VAD: Vector-quantized Motion Representation Learning for Human-centric Video Anomaly Detection](https://mlsanity.com/paper/2608.05069/) (2026)
- [Look Inside Each Video: Rethinking How Video Anomaly Detection Is Evaluated](https://mlsanity.com/paper/2608.21854/) (2026)

## Abstract (arXiv)

Multi-task learning based video anomaly detection methods combine multiple proxy tasks in different branches to detect video anomalies in different situations. Most existing methods either do not combine complementary tasks to effectively cover all motion patterns, or the class of the objects is not explicitly considered. To address the aforementioned shortcomings, we propose a novel multi-task learning based method that combines complementary proxy tasks to better consider the motion and appearance features. We combine the semantic segmentation and future frame prediction tasks in a single branch to learn the object class and consistent motion patterns, and to detect respective anomalies simultaneously. In the second branch, we added several attention mechanisms to detect motion anomalies with attention to object parts, the direction of motion, and the distance of the objects from the camera. Our qualitative results show that the proposed method considers the object class effectively and learns motion with attention to the aforementioned important factors which results in a precise motion modeling and a better motion anomaly detection. Additionally, quantitative results show the superiority of our method compared with state-of-the-art methods.