Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought
On the map: LLM reasoning → Chain-of-thought reasoning for driving
Abstract (arXiv)
Recent advancements in reasoning capability of Multimodal Large Language Models (MLLMs) demonstrate its effectiveness in tackling complex visual tasks. However, existing MLLM-based Video Anomaly Detection (VAD) methods remain limited to shallow anomaly descriptions without deep reasoning. In this paper, we propose a new task named Video Anomaly Reasoning (VAR), which aims to enable deep analysis and understanding of anomalies in the video by requiring MLLMs to think explicitly before answering. To this end, we propose Vad-R1, an end-to-end MLLM-based framework for VAR. Specifically, we design a Perception-to-Cognition Chain-of-Thought (P2C-CoT) that simulates the human process of recognizing anomalies, guiding the MLLM to reason anomaly step-by-step. Based on the structured P2C-CoT, we construct Vad-Reasoning, a dedicated dataset for VAR. Furthermore, we propose an improved reinforcement learning algorithm AVA-GRPO, which explicitly incentivizes the anomaly reasoning capability of MLLMs through a self-verification mechanism with limited annotations. Experimental results demonstrate that Vad-R1 achieves superior performance, outperforming both open-source and proprietary models on VAD and VAR tasks. Codes and datasets will be released at https://github.com/wbfwonderful/Vad-R1.
Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 38 of its 92 references that are in the map. First posted on arXiv 26 May 2025.
Builds on
38 of 92 references.
References · 38
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2024GPT-4o System Cardcited by 4,588
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2024OpenAI o1 System Cardcited by 1,244
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2023VideoChat: Chat-Centric Video Understandingcited by 857
26 more
- 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 518
- 2024Long Context Transfer from Language to Visioncited by 515
- 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 322
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
- 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modelingcited by 220
- 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
- 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMscited by 147
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
- 2025Streaming Video Question-Answering with In-context Video KV-Cache Retrievalcited by 117
- 2025Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledgecited by 73
- 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 70
- 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learningcited by 69
- 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 64
- 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 62
- 2025SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understandingcited by 57
- 2024Video Anomaly Detection and Explanation via Large Language Modelscited by 47
- 2025VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoningcited by 43
- 2025Unhackable Temporal Rewarding for Scalable Video MLLMscited by 33
- 2024VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Modelscited by 24
- 2024Exploring What Why and How: A Multifaceted Benchmark for Causation Understanding of Video Anomalycited by 9
- 2025SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Modelcited by 6
Built on by
Citing papers per year: 2025: 5; 2026: 19.
24 papers.
Sort:
By area:
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Modelscited by 10
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
- 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understandingcited by 5
- 2025CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-Worldcited by 4
- 2026Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimizationcited by 4
- 2026TAU-R1: Visual Language Model for Traffic Anomaly Understandingcited by 3
- 2026Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Modelscited by 2
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Taskscited by 2
- 2026ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitionscited by 1
- 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Modelscited by 1
- 2026TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understandingcited by 1
- 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoningcited by 1
12 more
- 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluationscited by 1
- 2025DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detectioncited by 0
- 2026SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learningcited by 0
- 2026LATERN: Test-Time Context-Aware Explainable Video Anomaly Detectioncited by 0
- 2026O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoningcited by 0
- 2026TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understandingcited by 0
- 2026Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detectioncited by 0
- 2026ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understandingcited by 0
- 2026Test-time Reinforcement Learning for Anomalous Video Understandingcited by 0
- 2026Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understandingcited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimizationcited by 0
- 2026trACT: temporal revelation Airborne Camera Trapcited by 0
- 2026trACT: temporal revelation Airborne Camera Trap7 Oct 2026
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization1 Oct 2026
- 2026Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding1 Oct 2026
- 2026ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding7 Sep 2026
- 2026Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection7 Sep 2026
- 2026TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding26 Aug 2026
- 2026Test-time Reinforcement Learning for Anomalous Video Understanding18 Aug 2026
- 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations13 Aug 2026
- 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning10 Aug 2026
- 2026TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding6 Aug 2026
- 2026O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning20 Jul 2026
- 2026Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization1 Jul 2026
12 more
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks21 May 2026
- 2026LATERN: Test-Time Context-Aware Explainable Video Anomaly Detection14 May 2026
- 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models12 May 2026
- 2026ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions9 Apr 2026
- 2026Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models7 Apr 2026
- 2026TAU-R1: Visual Language Model for Traffic Anomaly Understanding19 Mar 2026
- 2026SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learning1 Feb 2026
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models28 Nov 2025
- 2025CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World1 Nov 2025
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models6 Oct 2025
- 2025DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detection28 Jul 2025
- 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understanding28 May 2025
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Modelscited by 10
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
- 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understandingcited by 5
- 2026Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Modelscited by 2
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Taskscited by 2
- 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoningcited by 1
- 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluationscited by 1
- 2026SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learningcited by 0
- 2026LATERN: Test-Time Context-Aware Explainable Video Anomaly Detectioncited by 0
- 2026Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understandingcited by 0
- 2026Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding1 Oct 2026
- 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations13 Aug 2026
- 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning10 Aug 2026
- 2026MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks21 May 2026
- 2026LATERN: Test-Time Context-Aware Explainable Video Anomaly Detection14 May 2026
- 2026Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models7 Apr 2026
- 2026SRVAU-R1: Enhancing Video Anomaly Understanding via Reflection-Aware Learning1 Feb 2026
- 2025Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models28 Nov 2025
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models6 Oct 2025
- 2025A2Seek: Towards Reasoning-Centric Benchmark for Aerial Anomaly Understanding28 May 2025
- 2025CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-Worldcited by 4
- 2026Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimizationcited by 4
- 2026ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitionscited by 1
- 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Modelscited by 1
- 2026TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understandingcited by 1
- 2025DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detectioncited by 0
- 2026O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoningcited by 0
- 2026ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understandingcited by 0
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimizationcited by 0
- 2026trACT: temporal revelation Airborne Camera Trapcited by 0
- 2026trACT: temporal revelation Airborne Camera Trap7 Oct 2026
- 2026Controllable Multi-label Video Safety Detection via Adaptive Tversky Policy Optimization1 Oct 2026
- 2026ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding7 Sep 2026
- 2026TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding6 Aug 2026
- 2026O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning20 Jul 2026
- 2026Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization1 Jul 2026
- 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models12 May 2026
- 2026ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions9 Apr 2026
- 2025CueBench: Advancing Unified Understanding of Context-Aware Video Anomalies in Real-World1 Nov 2025
- 2025DAMS:Dual-Branch Adaptive Multiscale Spatiotemporal Framework for Video Anomaly Detection28 Jul 2025
Similar, not cited
Close in wording, no citation either way
- 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method
- 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning
- 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models
- 2026Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis
- 2024Quo Vadis, Anomaly Detection? LLMs and VLMs in the Spotlight
- 2026PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding
- 2024Harnessing Large Language Models for Training-free Video Anomaly Detection
- 2026CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection