# Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

arXiv: https://arxiv.org/abs/2301.00114 · HTML: https://mlsanity.com/paper/2301.00114/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 110 references that are in the map.
- First posted on arXiv 31 Dec 2022.

## Builds on (most cited first, 30 of 33)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [ActionFormer: Localizing Moments of Actions with Transformers](https://mlsanity.com/paper/2202.07925/) (2022)
- [3D Human Pose Estimation with Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.10455/) (2021)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos](https://mlsanity.com/paper/1903.03295/) (2019)
- [Graph Embedded Pose Clustering for Anomaly Detection](https://mlsanity.com/paper/1912.11850/) (2019)
- [One-Class Classification: Taxonomy of Study and Review of Techniques](https://mlsanity.com/paper/1312.0049/) (2013)
- [Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection](https://mlsanity.com/paper/1908.04321/) (2019)
- [Monocular Human Pose Estimation: A Survey of Deep Learning-based Methods](https://mlsanity.com/paper/2006.01423/) (2020)
- [Driver Anomaly Detection: A Dataset and Contrastive Learning Approach](https://mlsanity.com/paper/2009.14660/) (2020)
- [RWF-2000: An Open Large Scale Video Database for Violence Detection](https://mlsanity.com/paper/1911.05913/) (2019)
- [A Hierarchical Spatio-Temporal Graph Convolutional Neural Network for Anomaly Detection in Videos](https://mlsanity.com/paper/2112.04294/) (2021)
- [FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks](https://mlsanity.com/paper/2011.03180/) (2020)
- [Multi-Person Pose Estimation with Local Joint-to-Person Associations](https://mlsanity.com/paper/1608.08526/) (2016)
- [Privacy-Protecting Behaviours of Risk Detection in People with Dementia using Videos](https://mlsanity.com/paper/2212.10682/) (2022)
- [Motion and Region Aware Adversarial Learning for Fall Detection with Thermal Imaging](https://mlsanity.com/paper/2004.08352/) (2020)
- [Multi Visual Modality Fall Detection Dataset](https://mlsanity.com/paper/2206.12740/) (2022)
- [Unsupervised Anomaly Detection from Time-of-Flight Depth Images](https://mlsanity.com/paper/2203.01052/) (2022)

## Built on by (most cited first, 14 of 14)

- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [Privacy-Preserving Video Anomaly Detection: A Survey](https://mlsanity.com/paper/2411.14565/) (2024)
- [Adaptive Hyper-Graph Convolution Network for Skeleton-based Human Action Recognition with Virtual Connections](https://mlsanity.com/paper/2411.14796/) (2024)
- [Graph-Jigsaw Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection](https://mlsanity.com/paper/2403.12172/) (2024)
- [Frequency-Guided Diffusion Model with Perturbation Training for Skeleton-Based Video Anomaly Detection](https://mlsanity.com/paper/2412.03044/) (2024)
- [A Survey on Video Anomaly Detection via Deep Learning: Human, Vehicle, and Environment](https://mlsanity.com/paper/2508.14203/) (2025)
- [Unveiling Context-Related Anomalies: Knowledge Graph Empowered Decoupling of Scene and Action for Human-Related Video Anomaly Detection](https://mlsanity.com/paper/2409.03236/) (2024)
- [Learning by Neighbor-Aware Semantics, Deciding by Open-form Flows: Towards Robust Zero-Shot Skeleton Action Recognition](https://mlsanity.com/paper/2511.09388/) (2025)
- [Emotion Recognition from Skeleton Data: A Comprehensive Survey](https://mlsanity.com/paper/2507.18026/) (2025)
- [MemoVAD: Resource-Efficient Video Anomaly Detection via Dynamic Semantic Memory in Edge Computing Scenarios](https://mlsanity.com/paper/2606.07669/) (2026)
- [Point Cloud Video Anomaly Detection Based on Point Spatio-Temporal Auto-Encoder](https://mlsanity.com/paper/2306.04466/) (2023)
- [From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection](https://mlsanity.com/paper/2604.09327/) (2026)
- [Depth-Weighted Detection of Behaviours of Risk in People with Dementia using Cameras](https://mlsanity.com/paper/2408.15519/) (2024)
- [BAAF: Universal Transformation of One-Class Classifiers for Unsupervised Image Anomaly Detection](https://mlsanity.com/paper/2602.13091/) (2026)

## Abstract (arXiv)

The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacy-protecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them.