mlsanityData to 7 Oct 2026

Harnessing Large Language Models for Training-free Video Anomaly Detection

arXiv 2404.01014 ↗First posted 1 Apr 2024cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

Video anomaly detection (VAD) aims to temporally locate abnormal events in a video. Existing works mostly rely on training deep models to learn the distribution of normality with either video-level supervision, one-class supervision, or in an unsupervised setting. Training-based methods are prone to be domain-specific, thus being costly for practical deployment as any domain change will involve data collection and model training. In this paper, we radically depart from previous efforts and propose LAnguage-based VAD (LAVAD), a method tackling VAD in a novel, training-free paradigm, exploiting the capabilities of pre-trained large language models (LLMs) and existing vision-language models (VLMs). We leverage VLM-based captioning models to generate textual descriptions for each frame of any test video. With the textual scene description, we then devise a prompting mechanism to unlock the capability of LLMs in terms of temporal aggregation and anomaly score estimation, turning LLMs into an effective video anomaly detector. We further leverage modality-aligned VLMs and propose effective techniques based on cross-modal similarity for cleaning noisy captions and refining the LLM-based anomaly scores. We evaluate LAVAD on two large datasets featuring real-world surveillance scenarios (UCF-Crime and XD-Violence), showing that it outperforms both unsupervised and one-class methods without requiring any training or data collection.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

29 of 41 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  7. 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
  8. 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
  9. 2016Learning Temporal Regularity in Video Sequencescited by 254
  10. 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 235
  11. 2021Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learningcited by 226
  12. 2019Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detectioncited by 171
17 more
  1. 2023AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Modelscited by 169
  2. 2021MIST: Multiple Instance Self-Training Framework for Video Anomaly Detectioncited by 137
  3. 2021A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Predictioncited by 129
  4. 2020Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigmcited by 115
  5. 2022Generative Cooperative Learning for Unsupervised Video Anomaly Detectioncited by 102
  6. 2022MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detectioncited by 87
  7. 2020CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detectioncited by 79
  8. 2021Learning Normal Dynamics in Videos with Meta Prototype Networkcited by 71
  9. 2022CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detectioncited by 67
  10. 2019GODS: Generalized One-class Discriminative Subspaces for Anomaly Detectioncited by 64
  11. 2023Semantic Anomaly Detection with Large Language Modelscited by 53
  12. 2023Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detectioncited by 39
  13. 2018Subspace Support Vector Data Descriptioncited by 31
  14. 2022DyAnNet: A Scene Dynamicity Guided Self-Trained Video Anomaly Detection Networkcited by 26
  15. 2023Exploring Diffusion Models for Unsupervised Video Anomaly Detectioncited by 23
  16. 2023Unsupervised Video Anomaly Detection with Diffusion Models Conditioned on Compact Motion Representationscited by 18
  17. 2023Survey on video anomaly detection in dynamic scenes with moving camerascited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 34; 2026: 42.

85 papers.

Sort:

By area:

  1. 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 66
  2. 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
  3. 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanationscited by 37
  4. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  5. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 35
  6. 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
  7. 2024VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Modelscited by 24
  8. 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuningcited by 24
  9. 2025Video Summarization with Large Language Modelscited by 16
  10. 2025PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineercited by 15
  11. 2025VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Treecited by 15
  12. 2025Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detectioncited by 12
38 more
  1. 2024Privacy-Preserving Video Anomaly Detection: A Surveycited by 11
  2. 2025MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detectioncited by 10
  3. 2024Exploring What Why and How: A Multifaceted Benchmark for Causation Understanding of Video Anomalycited by 9
  4. 2025Towards Visual Discrimination and Reasoning of Real-World Physical Dynamics: Physics-Grounded Anomaly Detectioncited by 9
  5. 2025A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysiscited by 9
  6. 2025Language-guided Open-world Video Anomaly Detection under Weak Supervisioncited by 8
  7. 2025Track Any Anomalous Object: A Granular Video Anomaly Detection Pipelinecited by 8
  8. 2025Foundation Models and Transformers for Anomaly Detection: A Surveycited by 8
  9. 2025HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMscited by 8
  10. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  11. 2025Player-Centric Multimodal Prompt Generation for Large Language Model Based Identity-Aware Basketball Video Captioningcited by 7
  12. 2025AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detectioncited by 6
  13. 2026No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detectioncited by 6
  14. 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understandingcited by 5
  15. 2025Pistachio: Towards Synthetic, Balanced, and Long-Form Video Anomaly Benchmarkscited by 5
  16. 2025SmartHome-Bench: A Comprehensive Benchmark for Video Anomaly Detection in Smart Homes Using Multi-Modal Large Language Modelscited by 4
  17. 2025The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLMcited by 4
  18. 2025Leveraging Multimodal LLM Descriptions of Activity for Explainable Semi-Supervised Video Anomaly Detectioncited by 4
  19. 2026Instance-Aligned Captions for Explainable Video Anomaly Detectioncited by 4
  20. 2025Foundation Models for Anomaly Detection: Vision and Challengescited by 3
  21. 2025Rethinking Metrics and Benchmarks of Video Anomaly Detectioncited by 3
  22. 2025Evaluation of Vision-LLMs in Surveillance Videocited by 3
  23. 2025VLMDiff: Leveraging Vision-Language Models for Multi-Class Anomaly Detection with Diffusioncited by 3
  24. 2026TAU-R1: Visual Language Model for Traffic Anomaly Understandingcited by 3
  25. 2024Advancing Pre-trained Teacher: Towards Robust Feature Discrepancy for Anomaly Detectioncited by 2
  26. 2025FrameShield: Adversarially Robust Video Anomaly Detectioncited by 2
  27. 2025Text-guided Fine-Grained Video Anomaly Understandingcited by 2
  28. 2025VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Modelscited by 2
  29. 2026Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detectioncited by 2
  30. 2026Multi-turn Physics-informed Vision-language Model for Physics-grounded Anomaly Detectioncited by 2
  31. 2026SphereVAD: Training-Free Video Anomaly Detection via Geodesic Inference on the Unit Hyperspherecited by 2
  32. 2026MemoVAD: Resource-Efficient Video Anomaly Detection via Dynamic Semantic Memory in Edge Computing Scenarioscited by 2
  33. 2026The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detectioncited by 2
  34. 2026Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindnesscited by 2
  35. 2026TD-VAD: Breaking Visual Dependence in Video Anomaly Detection with Text-Driven Learningcited by 2
  36. 2025Can Text-to-Video Generation help Video-Language Alignment?cited by 1
  37. 2025Action Hints: Semantic Typicality and Context Uniqueness for Generalizable Skeleton-based Video Anomaly Detectioncited by 1
  38. 2025AVAR-Net: A Lightweight Audio-Visual Anomaly Recognition Framework with a Benchmark Datasetcited by 1

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight
  2. 2024Video Anomaly Detection in 10 Years: A Survey and Outlook
  3. 2024Vision-Language Models Assisted Unsupervised Video Anomaly Detection
  4. 2025SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model
  5. 2026Context-structured Video Anomaly Detection with Large Vision-Language Models
  6. 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models
  7. 2023VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection
  8. 2026Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection