mlsanityData to 7 Oct 2026

Vision-Language Models Assisted Unsupervised Video Anomaly Detection

arXiv 2409.14109 ↗First posted 21 Sep 2024cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Video anomaly detection is a subject of great interest across industrial and academic domains due to its crucial role in computer vision applications. However, the inherent unpredictability of anomalies and the scarcity of anomaly samples present significant challenges for unsupervised learning methods. To overcome the limitations of unsupervised learning, which stem from a lack of comprehensive prior knowledge about anomalies, we propose VLAVAD (Video-Language Models Assisted Anomaly Detection). Our method employs a cross-modal pre-trained model that leverages the inferential capabilities of large language models (LLMs) in conjunction with a Selective-Prompt Adapter (SPA) for selecting semantic space. Additionally, we introduce a Sequence State Space Module (S3M) that detects temporal inconsistencies in semantic features. By mapping high-dimensional visual features to low-dimensional semantic ones, our method significantly enhance the interpretability of unsupervised anomaly detection. Our proposed approach effectively tackles the challenge of detecting elusive anomalies that are hard to discern over periods, achieving SOTA on the challenging ShanghaiTech dataset.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 76 of its 122 references that are in the map. First posted on arXiv 21 Sep 2024.

Builds on

76 of 122 references.

References · 76

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  7. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  8. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  9. 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
  10. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
  11. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  12. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
38 more
  1. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
  2. 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 235
  3. 2021Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learningcited by 226
  4. 2018Adversarially Learned One-Class Classifier for Novelty Detectioncited by 202
  5. 2019Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detectioncited by 171
  6. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  7. 2020Anomaly Detection in Video via Self-Supervised and Multi-Task Learningcited by 147
  8. 2021MIST: Multiple Instance Self-Training Framework for Video Anomaly Detectioncited by 137
  9. 2021A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Predictioncited by 129
  10. 2021Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detectioncited by 119
  11. 2019Anomaly Detection in Video Sequence with Appearance-Motion Correspondencecited by 118
  12. 2020Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigmcited by 115
  13. 2018Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Videocited by 110
  14. 2019Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videoscited by 103
  15. 2022Generative Cooperative Learning for Unsupervised Video Anomaly Detectioncited by 102
  16. 2019Graph Embedded Pose Clustering for Anomaly Detectioncited by 95
  17. 2021Towards a Rigorous Evaluation of Time-series Anomaly Detectioncited by 95
  18. 2021UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detectioncited by 91
  19. 2020A Survey of Single-Scene Video Anomaly Detectioncited by 88
  20. 2020A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Videocited by 85
  21. 2020Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detectioncited by 83
  22. 2020CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detectioncited by 79
  23. 2021Learning Normal Dynamics in Videos with Meta Prototype Networkcited by 71
  24. 2023Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detectioncited by 64
  25. 2020Few-shot Scene-adaptive Anomaly Detectioncited by 62
  26. 2022Video Anomaly Detection by Solving Decoupled Spatio-Temporal Jigsaw Puzzlescited by 62
  27. 2022Normalizing Flows for Human Pose Anomaly Detectioncited by 59
  28. 2023Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Modelscited by 59
  29. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  30. 2023Video Event Restoration Based on Keyframes for Video Anomaly Detectioncited by 54
  31. 2020Continual Learning for Anomaly Detection in Surveillance Videoscited by 49
  32. 2022Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detectioncited by 47
  33. 2020Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Predictioncited by 42
  34. 2023A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipationcited by 42
  35. 2023Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detectioncited by 42
  36. 2023Diversity-Measurable Anomaly Detectioncited by 41
  37. 2021Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspectivecited by 40
  38. 2023Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detectioncited by 39

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 4.

6 papers.

  1. 2024Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlightcited by 8
  2. 2025The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLMcited by 4
  3. 2026Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervisioncited by 1
  4. 2026Cascading multi-agent anomaly detection in surveillance systems via vision-language models and embedding-based classificationcited by 0
  5. 2026Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detectioncited by 0
  6. 2026Look Inside Each Video: Rethinking How Video Anomaly Detection Is Evaluatedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Harnessing Large Language Models for Training-free Video Anomaly Detection
  2. 2025SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model
  3. 2024Video Anomaly Detection in 10 Years: A Survey and Outlook
  4. 2026No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection
  5. 2025From Benchmarks to Reality: Advancing Visual Anomaly Detection by the VAND 3.0 Challenge
  6. 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models
  7. 2026Context-structured Video Anomaly Detection with Large Vision-Language Models
  8. 2026SphereVAD: Training-Free Video Anomaly Detection via Geodesic Inference on the Unit Hypersphere