# Collaborative Learning of Anomalies with Privacy (CLAP) for Unsupervised Video Anomaly Detection: A New Baseline

arXiv: https://arxiv.org/abs/2404.00847 · HTML: https://mlsanity.com/paper/2404.00847/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Adversarially Learned One-Class Classifier for Novelty Detection](https://mlsanity.com/paper/1802.09088/) (2018)
- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [FedScale: Benchmarking Model and System Performance of Federated Learning at Scale](https://mlsanity.com/paper/2105.11367/) (2021)
- [FedVision: An Online Visual Object Detection Platform Powered by Federated Learning](https://mlsanity.com/paper/2001.06202/) (2020)
- [Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm](https://mlsanity.com/paper/2004.07657/) (2020)
- [Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Video](https://mlsanity.com/paper/1812.04960/) (2018)
- [Generative Cooperative Learning for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2203.03962/) (2022)
- [CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection](https://mlsanity.com/paper/2011.12077/) (2020)
- [Deep Federated Learning for Autonomous Driving](https://mlsanity.com/paper/2110.05754/) (2021)
- [Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.04090/) (2022)
- [FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2202.13670/) (2022)
- [A Secure Federated Learning Framework for 5G Networks](https://mlsanity.com/paper/2005.05752/) (2020)
- [A Self-Reasoning Framework for Anomaly Detection Using Video-Level Labels](https://mlsanity.com/paper/2008.11887/) (2020)
- [A Coarse-to-Fine Pseudo-Labeling (C2FPL) Framework for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2310.17650/) (2023)
- [Clustering Aided Weakly Supervised Training to Detect Anomalous Events in Surveillance Videos](https://mlsanity.com/paper/2203.13704/) (2022)
- [FeSViBS: Federated Split Learning of Vision Transformer with Block Sampling](https://mlsanity.com/paper/2306.14638/) (2023)

## Built on by (most cited first, 17 of 17)

- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [Networking Systems for Video Anomaly Detection: A Tutorial and Survey](https://mlsanity.com/paper/2405.10347/) (2024)
- [Video Anomaly Detection in 10 Years: A Survey and Outlook](https://mlsanity.com/paper/2405.19387/) (2024)
- [SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Models](https://mlsanity.com/paper/2505.12589/) (2025)
- [Privacy-Preserving Video Anomaly Detection: A Survey](https://mlsanity.com/paper/2411.14565/) (2024)
- [Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Prompting](https://mlsanity.com/paper/2510.02155/) (2025)
- [ACCIDENT: A Benchmark Dataset for Vehicle Accident Detection from Traffic Surveillance Videos](https://mlsanity.com/paper/2604.09819/) (2026)
- [The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM](https://mlsanity.com/paper/2507.21649/) (2025)
- [A Framework for Double-Blind Federated Adaptation of Foundation Models](https://mlsanity.com/paper/2502.01289/) (2025)
- [MLE-UVAD: Minimal Latent Entropy Autoencoder for Fully Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2603.23868/) (2026)
- [QVAD: A Question-Centric Agentic Framework for Efficient and Training-Free Video Anomaly Detection](https://mlsanity.com/paper/2604.03040/) (2026)
- [Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models](https://mlsanity.com/paper/2605.12725/) (2026)
- [Images in Motion?: A First Look into Video Leakage in Collaborative Deep Learning](https://mlsanity.com/paper/2509.09742/) (2025)
- [RobustA: Robust Anomaly Detection in Multimodal Data](https://mlsanity.com/paper/2511.07276/) (2025)
- [SENSE-VAD: Sentient and Semantic Video Anomaly Detection for Autonomous Driving](https://mlsanity.com/paper/2606.31875/) (2026)
- [FedVAR: Prototype-Aligned Federated Framework for Video Anomaly Recognition](https://mlsanity.com/paper/2608.06876/) (2026)
- [STAIN-FL: Stealthy Targeted Attack Injection with Contextual Triggers in Federated Learning](https://mlsanity.com/paper/2608.23952/) (2026)

## Abstract (arXiv)

Unsupervised (US) video anomaly detection (VAD) in surveillance applications is gaining more popularity recently due to its practical real-world applications. As surveillance videos are privacy sensitive and the availability of large-scale video data may enable better US-VAD systems, collaborative learning can be highly rewarding in this setting. However, due to the extremely challenging nature of the US-VAD task, where learning is carried out without any annotations, privacy-preserving collaborative learning of US-VAD systems has not been studied yet. In this paper, we propose a new baseline for anomaly detection capable of localizing anomalous events in complex surveillance videos in a fully unsupervised fashion without any labels on a privacy-preserving participant-based distributed training configuration. Additionally, we propose three new evaluation protocols to benchmark anomaly detection approaches on various scenarios of collaborations and data availability. Based on these protocols, we modify existing VAD datasets to extensively evaluate our approach as well as existing US SOTA methods on two large-scale datasets including UCF-Crime and XD-Violence. All proposed evaluation protocols, dataset splits, and codes are available here: https://github.com/AnasEmad11/CLAP