# Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight

arXiv: https://arxiv.org/abs/2412.18298 · HTML: https://mlsanity.com/paper/2412.18298/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 24 Dec 2024.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity](https://mlsanity.com/paper/2412.06171/) (2024)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Hawk: Learning to Understand Open-World Video Anomalies](https://mlsanity.com/paper/2405.16886/) (2024)
- [Video Anomaly Detection and Explanation via Large Language Models](https://mlsanity.com/paper/2401.05702/) (2024)
- [Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts](https://mlsanity.com/paper/2408.05905/) (2024)
- [VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models](https://mlsanity.com/paper/2412.01095/) (2024)
- [Learnable Expansion of Graph Operators for Multi-Modal Feature Fusion](https://mlsanity.com/paper/2410.01506/) (2024)
- [Do Language Models Understand Time?](https://mlsanity.com/paper/2412.13845/) (2024)
- [Taylor Videos for Action Recognition](https://mlsanity.com/paper/2402.03019/) (2024)
- [Motion meets Attention: Video Motion Prompts](https://mlsanity.com/paper/2407.03179/) (2024)
- [TrackNetV4: Enhancing Fast Sports Object Tracking with Motion Attention Maps](https://mlsanity.com/paper/2409.14543/) (2024)
- [Vision-Language Models Assisted Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2409.14109/) (2024)
- [When Spatial meets Temporal in Action Recognition](https://mlsanity.com/paper/2411.15284/) (2024)

## Built on by (most cited first, 8 of 8)

- [A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysis](https://mlsanity.com/paper/2511.00962/) (2025)
- [The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM](https://mlsanity.com/paper/2507.21649/) (2025)
- [Graph Your Own Prompt](https://mlsanity.com/paper/2509.23373/) (2025)
- [Learning Time in Static Classifiers](https://mlsanity.com/paper/2511.12321/) (2025)
- [Evolving Skeletons: Motion Dynamics in Action Recognition](https://mlsanity.com/paper/2501.02593/) (2025)
- [Benchmarking Compact VLMs for Clip-Level Surveillance Anomaly Detection Under Weak Supervision](https://mlsanity.com/paper/2603.13306/) (2026)
- [Video Understanding by Design: How Datasets Shape Video Models](https://mlsanity.com/paper/2509.09151/) (2025)
- [Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection](https://mlsanity.com/paper/2605.08651/) (2026)

## Abstract (arXiv)

Video anomaly detection (VAD) has witnessed significant advancements through the integration of large language models (LLMs) and vision-language models (VLMs), addressing critical challenges such as interpretability, temporal reasoning, and generalization in dynamic, open-world scenarios. This paper presents an in-depth review of cutting-edge LLM-/VLM-based methods in 2024, focusing on four key aspects: (i) enhancing interpretability through semantic insights and textual explanations, making visual anomalies more understandable; (ii) capturing intricate temporal relationships to detect and localize dynamic anomalies across video frames; (iii) enabling few-shot and zero-shot detection to minimize reliance on large, annotated datasets; and (iv) addressing open-world and class-agnostic anomalies by using semantic understanding and motion features for spatiotemporal coherence. We highlight their potential to redefine the landscape of VAD. Additionally, we explore the synergy between visual and textual modalities offered by LLMs and VLMs, highlighting their combined strengths and proposing future directions to fully exploit the potential in enhancing video anomaly detection.