# Are Multimodal LLMs Ready for Surveillance? A Reality Check on Zero-Shot Anomaly Detection in the Wild

arXiv: https://arxiv.org/abs/2603.04727 · HTML: https://mlsanity.com/paper/2603.04727/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 38 of its 53 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [LVBench: An Extreme Long Video Understanding Benchmark](https://mlsanity.com/paper/2406.08035/) (2024)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [TempCompass: Do Video LLMs Really Understand Videos?](https://mlsanity.com/paper/2403.00476/) (2024)
- [VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness](https://mlsanity.com/paper/2503.21755/) (2025)
- [MM-LLMs: Recent Advances in MultiModal Large Language Models](https://mlsanity.com/paper/2401.13601/) (2024)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding](https://mlsanity.com/paper/2406.14515/) (2024)
- [VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2308.11681/) (2023)
- [Graph Embedded Pose Clustering for Anomaly Detection](https://mlsanity.com/paper/1912.11850/) (2019)
- [A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4](https://mlsanity.com/paper/2310.12321/) (2023)
- [Normalizing Flows for Human Pose Anomaly Detection](https://mlsanity.com/paper/2211.10946/) (2022)
- [Video-Bench: Human-Aligned Video Generation Benchmark](https://mlsanity.com/paper/2504.04907/) (2025)
- [Pre-Trained Video Generative Models as World Simulators](https://mlsanity.com/paper/2502.07825/) (2025)
- [Delving into CLIP latent space for Video Anomaly Recognition](https://mlsanity.com/paper/2310.02835/) (2023)
- [Regularity Learning via Explicit Distribution Modeling for Skeletal Video Anomaly Detection](https://mlsanity.com/paper/2112.03649/) (2021)
- [An Exploratory Study on Human-Centric Video Anomaly Detection through Variational Autoencoders and Trajectory Prediction](https://mlsanity.com/paper/2406.15395/) (2024)
- [CHAD: Charlotte Anomaly Dataset](https://mlsanity.com/paper/2212.09258/) (2022)
- [Ancilia: Scalable Intelligent Video Surveillance for the Artificial Intelligence of Things](https://mlsanity.com/paper/2301.03561/) (2023)
- [Evaluating the Effectiveness of Video Anomaly Detection in the Wild: Online Learning and Inference for Real-world Deployment](https://mlsanity.com/paper/2404.18747/) (2024)
- [Quantifying Influencer Impact on Affective Polarization](https://mlsanity.com/paper/2405.15893/) (2024)
- [Towards Adaptive Human-centric Video Anomaly Detection: A Comprehensive Framework and A New Benchmark](https://mlsanity.com/paper/2408.14329/) (2024)

## Built on by (most cited first, 2 of 2)

- [From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection](https://mlsanity.com/paper/2604.09327/) (2026)
- [trACT: temporal revelation Airborne Camera Trap](https://mlsanity.com/paper/2610.09417/) (2026)

## Abstract (arXiv)

Multimodal large language models (MLLMs) have demonstrated impressive general competence in video understanding, yet their reliability for real-world Video Anomaly Detection (VAD) remains largely unexplored. Unlike conventional pipelines relying on reconstruction or pose-based cues, MLLMs enable a paradigm shift: treating anomaly detection as a language-guided reasoning task. In this work, we systematically evaluate state-of-the-art MLLMs on the ShanghaiTech and CHAD benchmarks by reformulating VAD as a binary classification task under weak temporal supervision. We investigate how prompt specificity and temporal window lengths (1s--3s) influence performance, focusing on the precision--recall trade-off. Our findings reveal a pronounced conservative bias in zero-shot settings; while models exhibit high confidence, they disproportionately favor the 'normal' class, resulting in high precision but a recall collapse that limits practical utility. We demonstrate that class-specific instructions can significantly shift this decision boundary, improving the peak F1-score on ShanghaiTech from 0.09 to 0.64, yet recall remains a critical bottleneck. These results highlight a significant performance gap for MLLMs in noisy environments and provide a foundation for future work in recall-oriented prompting and model calibration for open-world surveillance, which demands complex video understanding and reasoning.