# Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Prompting

arXiv: https://arxiv.org/abs/2510.02155 · HTML: https://mlsanity.com/paper/2510.02155/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 44 of its 55 references that are in the map.
- First posted on arXiv 2 Oct 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 44)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Video-R1: Reinforcing Video Reasoning in MLLMs](https://mlsanity.com/paper/2503.21776/) (2025)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)
- [PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model](https://mlsanity.com/paper/2208.11625/) (2022)
- [VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2308.11681/) (2023)
- [Generative Cooperative Learning for Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2203.03962/) (2022)
- [Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2302.05160/) (2023)
- [MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2211.15098/) (2022)
- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.05136/) (2022)
- [GODS: Generalized One-class Discriminative Subspaces for Anomaly Detection](https://mlsanity.com/paper/1908.05884/) (2019)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Open-Vocabulary Video Anomaly Detection](https://mlsanity.com/paper/2311.07042/) (2023)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2404.08531/) (2024)
- [Video Anomaly Detection and Explanation via Large Language Models](https://mlsanity.com/paper/2401.05702/) (2024)
- [ArGue: Attribute-Guided Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2311.16494/) (2023)
- [EventVAD: Training-Free Event-Aware Video Anomaly Detection](https://mlsanity.com/paper/2504.13092/) (2025)
- [More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models](https://mlsanity.com/paper/2509.25848/) (2025)

## Built on by (most cited first, 8 of 8)

- [Understanding Multimodal Complementarity for Single-Frame Action Anticipation](https://mlsanity.com/paper/2601.22039/) (2026)
- [Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models](https://mlsanity.com/paper/2605.02912/) (2026)
- [High-Entropy Tokens as Multimodal Failure Points in Vision-Language Models](https://mlsanity.com/paper/2512.21815/) (2025)
- [COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection](https://mlsanity.com/paper/2605.15325/) (2026)
- [Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximization](https://mlsanity.com/paper/2605.10764/) (2026)
- [O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning](https://mlsanity.com/paper/2607.18142/) (2026)
- [NOVA: Normal-Side Modeling for Training-Free Zero-Shot Video Anomaly Detection](https://mlsanity.com/paper/2609.06360/) (2026)
- [AnomalyCraft-700K: Component-Level Controllable and Verifiable Synthetic Anomalies for Fine-Grained Video Anomaly Understanding](https://mlsanity.com/paper/2609.06978/) (2026)

## Abstract (arXiv)

Prompting has emerged as a practical way to adapt frozen vision-language models (VLMs) for video anomaly detection (VAD). Yet, existing prompts are often overly abstract, overlooking the fine-grained human-object interactions or action semantics that define complex anomalies in surveillance videos. We propose ASK-Hint, a structured prompting framework that leverages action-centric knowledge to elicit more accurate and interpretable reasoning from frozen VLMs. Our approach organizes prompts into semantically coherent groups (e.g. violence, property crimes, public safety) and formulates fine-grained guiding questions that align model predictions with discriminative visual cues. Extensive experiments on UCF-Crime and XD-Violence show that ASK-Hint consistently improves AUC over prior baselines, achieving state-of-the-art performance compared to both fine-tuned and training-free methods. Beyond accuracy, our framework provides interpretable reasoning traces towards anomaly and demonstrates strong generalization across datasets and VLM backbones. These results highlight the critical role of prompt granularity and establish ASK-Hint as a new training-free and generalizable solution for explainable video anomaly detection.