# Language-guided Open-world Video Anomaly Detection under Weak Supervision

arXiv: https://arxiv.org/abs/2503.13160 · HTML: https://mlsanity.com/paper/2503.13160/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 56 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Builds on (most cited first, 30 of 39)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [VATEX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research](https://mlsanity.com/paper/1904.03493/) (2019)
- [LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment](https://mlsanity.com/paper/2310.01852/) (2023)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Fine-tuned CLIP Models are Efficient Video Learners](https://mlsanity.com/paper/2212.03640/) (2022)
- [Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection](https://mlsanity.com/paper/1903.07256/) (2019)
- [VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset](https://mlsanity.com/paper/2304.08345/) (2023)
- [VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2308.11681/) (2023)
- [UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection](https://mlsanity.com/paper/2111.08644/) (2021)
- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity](https://mlsanity.com/paper/2412.06171/) (2024)
- [CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.05136/) (2022)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Open-Vocabulary Video Anomaly Detection](https://mlsanity.com/paper/2311.07042/) (2023)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2404.08531/) (2024)
- [Hawk: Learning to Understand Open-World Video Anomalies](https://mlsanity.com/paper/2405.16886/) (2024)
- [Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly](https://mlsanity.com/paper/2405.00181/) (2024)
- [Learning Prompt-Enhanced Context Features for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2306.14451/) (2023)
- [A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipation](https://mlsanity.com/paper/2305.13611/) (2023)

## Built on by (most cited first, 8 of 8)

- [The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM](https://mlsanity.com/paper/2507.21649/) (2025)
- [Rethinking Metrics and Benchmarks of Video Anomaly Detection](https://mlsanity.com/paper/2505.19022/) (2025)
- [Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindness](https://mlsanity.com/paper/2607.20780/) (2026)
- [ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions](https://mlsanity.com/paper/2604.07772/) (2026)
- [NOVA: Normal-Side Modeling for Training-Free Zero-Shot Video Anomaly Detection](https://mlsanity.com/paper/2609.06360/) (2026)
- [ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding](https://mlsanity.com/paper/2609.07941/) (2026)
- [RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos](https://mlsanity.com/paper/2609.17843/) (2026)
- [Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues](https://mlsanity.com/paper/2609.40014/) (2026)

## Abstract (arXiv)

Video anomaly detection (VAD) aims to detect anomalies that deviate from what is expected. In open-world scenarios, the expected events may change as requirements change. For example, not wearing a mask may be considered abnormal during a flu outbreak but normal otherwise. However, existing methods assume that the definition of anomalies is invariable, and thus are not applicable to the open world. To address this, we propose a novel open-world VAD paradigm with variable definitions, allowing guided detection through user-provided natural language at inference time. This paradigm necessitates establishing a robust mapping from video and textual definition to anomaly scores. Therefore, we propose LaGoVAD (Language-guided Open-world Video Anomaly Detector), a model that dynamically adapts anomaly definitions under weak supervision with two regularization strategies: diversifying the relative durations of anomalies via dynamic video synthesis, and enhancing feature robustness through contrastive learning with negative mining. Training such adaptable models requires diverse anomaly definitions, but existing datasets typically provide labels without semantic descriptions. To bridge this gap, we collect PreVAD (Pre-training Video Anomaly Dataset), the largest and most diverse video anomaly dataset to date, featuring 35,279 annotated videos with multi-level category labels and descriptions that explicitly define anomalies. Zero-shot experiments on seven datasets demonstrate LaGoVAD's SOTA performance. Our dataset and code are released at https://github.com/Kamino666/LaGoVAD-PreVAD.