mlsanityData to 7 Oct 2026

Video Anomaly Detection and Explanation via Large Language Models

arXiv 2401.05702 ↗First posted 11 Jan 2024cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Video Anomaly Detection (VAD) aims to localize abnormal events on the timeline of long-range surveillance videos. Anomaly-scoring-based methods have been prevailing for years but suffer from the high complexity of thresholding and low explanability of detection results. In this paper, we conduct pioneer research on equipping video-based large language models (VLLMs) in the framework of VAD, making the VAD model free from thresholds and able to explain the reasons for the detected anomalies. We introduce a novel network module Long-Term Context (LTC) to mitigate the incapability of VLLMs in long-range context modeling. We design a three-phase training method to improve the efficiency of fine-tuning VLLMs by substantially minimizing the requirements for VAD data and lowering the costs of annotating instruction-tuning data. Our trained model achieves the top performance on the anomaly videos of the UCF-Crime and TAD benchmarks, with the AUC improvements of +3.86% and +4.96%, respectively. More impressively, our approach can provide textual explanations for detected anomalies.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

20 of 30 references.

References · 20

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  4. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  5. 2023VideoChat: Chat-Centric Video Understandingcited by 851
  6. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  7. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  8. 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
  9. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 330
  10. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  11. 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 235
  12. 2021Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learningcited by 226
8 more
  1. 2019Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detectioncited by 171
  2. 2023Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detectioncited by 88
  3. 2020CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detectioncited by 79
  4. 2023Unbiased Multiple Instance Learning for Weakly Supervised Video Anomaly Detectioncited by 64
  5. 2020Localizing Anomalies from Weakly-Labeled Videoscited by 52
  6. 2019Motion-Aware Feature for Improved Video Anomaly Detectioncited by 48
  7. 2022Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detectioncited by 47
  8. 2021Global Information Guided Video Anomaly Detectioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 21; 2026: 12.

45 papers.

Sort:

By area:

  1. 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 66
  2. 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
  3. 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Modelscited by 55
  4. 2025Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Modelscited by 54
  5. 2024Advancing Video Anomaly Detection: A Concise Review and a New Datasetcited by 48
  6. 2024Hawk: Learning to Understand Open-World Video Anomaliescited by 47
  7. 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanationscited by 37
  8. 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
  9. 2024VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Modelscited by 24
  10. 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuningcited by 24
  11. 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 23
  12. 2024GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computingcited by 22
33 more
  1. 2025VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluationcited by 21
  2. 2025VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Treecited by 15
  3. 2025Reasoning Segmentation for Images and Videos: A Surveycited by 12
  4. 2024Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlightcited by 8
  5. 2025HiProbe-VAD: Video Anomaly Detection via Hidden States Probing in Tuning-Free Multimodal LLMscited by 8
  6. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  7. 2025VAGU & GtS: LLM-Based Benchmark and Framework for Joint Video Anomaly Grounding and Understandingcited by 6
  8. 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understandingcited by 5
  9. 2024Evaluation and Comparison of Visual Language Models for Transportation Engineering Problemscited by 4
  10. 2025SmartHome-Bench: A Comprehensive Benchmark for Video Anomaly Detection in Smart Homes Using Multi-Modal Large Language Modelscited by 4
  11. 2025The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLMcited by 4
  12. 2025Leveraging Multimodal LLM Descriptions of Activity for Explainable Semi-Supervised Video Anomaly Detectioncited by 4
  13. 2026Instance-Aligned Captions for Explainable Video Anomaly Detectioncited by 4
  14. 2025Foundation Models for Anomaly Detection: Vision and Challengescited by 3
  15. 2026TAU-R1: Visual Language Model for Traffic Anomaly Understandingcited by 3
  16. 2025Sage Deer: A Super-Aligned Driving Generalist Is Your Copilotcited by 2
  17. 2025VADER: Towards Causal Video Anomaly Understanding with Relation-Aware Large Language Modelscited by 2
  18. 2025ALARM: Automated MLLM-Based Anomaly Detection in Complex-EnviRonment Monitoring with Uncertainty Quantificationcited by 2
  19. 2024INVARLLM: LLM-assisted Physical Invariant Extraction for Cyber-Physical Systems Anomaly Detectioncited by 1
  20. 2025CADE: Continual Weakly-supervised Video Anomaly Detection with Ensemblescited by 1
  21. 2026PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understandingcited by 1
  22. 2026COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detectioncited by 1
  23. 2026CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detectioncited by 1
  24. 2025Explaining Low Perception Model Competency with High-Competency Counterfactualscited by 0
  25. 2025PiercingEye: Dual-Space Video Violence Detection with Hyperbolic Vision-Language Guidancecited by 0
  26. 2025Period-LLM: Extending the Periodic Capability of Multimodal Large Language Modelcited by 0
  27. 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Methodcited by 0
  28. 2026Spatiotemporal Semantic V2X Framework for Cooperative Collision Predictioncited by 0
  29. 2026SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learningcited by 0
  30. 2026LATERN: Test-Time Context-Aware Explainable Video Anomaly Detectioncited by 0
  31. 2026Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoningcited by 0
  32. 2026TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understandingcited by 0
  33. 2026Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understandingcited by 0