Text-guided Fine-Grained Video Anomaly Understanding
Not on the map. Why?
Abstract (arXiv)
Subtle abnormal events in videos often manifest as weak spatio-temporal cues that are easily overlooked by conventional anomaly detection systems. Existing video anomaly detection approaches typically provide coarse binary anomaly decisions without interpretable evidence, while large vision-language models (LVLMs) can produce textual judgments but lack precise localization of subtle visual signals. To address this gap, we propose Text-guided Fine-Grained Video Anomaly Understanding T-VAU, a framework that grounds subtle anomaly evidence into multimodal reasoning. Specifically, we introduce an Anomaly Heatmap Decoder (AHD) that performs visual-textual feature alignment to extract pixel-level spatio-temporal anomaly heatmaps from intermediate visual representations. We further design a Region-aware Anomaly Encoder (RAE) that converts these heatmaps into structured prompt embeddings, enabling the LVLM to perform anomaly detection, localization, and semantic explanation in a unified reasoning pipeline. To support fine-grained supervision, we construct a target-level fine-grained video-text anomaly dataset derived from ShanghaiTech and UBnormal with detailed annotations of object appearance, localization, and motion trajectories. Extensive experiments demonstrate that T-VAU significantly improves anomaly localization and textual reasoning performance on both benchmarks, achieving strong results in BLEU-4 metrics and Yes/No decision accuracy while providing interpretable pixel-level spatio-temporal evidence for anomaly understanding. The code will be available at https://github.com/momiji-bit/T-VAU.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 1 Nov 2025.
Builds on
45 of 54 references.
References · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
- 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
- 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
- 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
- 2020Learning Memory-guided Normality for Anomaly Detectioncited by 278
33 more
- 2019Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detectioncited by 171
- 2023VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detectioncited by 121
- 2021PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformercited by 118
- 2019Street Scene: A new dataset and evaluation protocol for video anomaly detectioncited by 89
- 2020A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Videocited by 85
- 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
- 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 66
- 2023Open-Vocabulary Video Anomaly Detectioncited by 57
- 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Modelscited by 55
- 2024FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localizationcited by 51
- 2024Benchmarking Micro-action Recognition: Dataset, Methods, and Applicationscited by 48
- 2024Hawk: Learning to Understand Open-World Video Anomaliescited by 47
- 2024Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomalycited by 44
- 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
- 2024Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Promptscited by 40
- 2025EventVAD: Training-Free Event-Aware Video Anomaly Detectioncited by 32
- 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
- 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classificationcited by 24
- 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuningcited by 24
- 2024Prototypical Calibrating Ambiguous Samples for Micro-Action Recognitioncited by 22
- 2024MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detectioncited by 21
- 2024VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMscited by 20
- 2024MMAD: Multi-label Micro-Action Detection in Videoscited by 20
- 2025SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Modelscited by 18
- 2024Micro-gesture Online Recognition using Learnable Query Pointscited by 17
- 2023Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videoscited by 15
- 2025AnyAnomaly: Zero-Shot Customizable Video Anomaly Detection with LVLMcited by 15
- 2023Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learningcited by 14
- 2025Anomize: Better Open Vocabulary Video Anomaly Detectioncited by 13
- 2025MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusioncited by 13
- 2025Motion Matters: Motion-guided Modulation Network for Skeleton-based Micro-Action Recognitioncited by 11
- 2024Benchmarking Cross-Domain Audio-Visual Deception Detectioncited by 6
- 2025SVC 2025: the First Multimodal Deception Detection Challengecited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026PrismVAU: Prompt-Refined Inference System for Multimodal Video Anomaly Understanding
- 2026Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method
- 2026Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models
- 2026AnomalyCraft-700K: Component-Level Controllable and Verifiable Synthetic Anomalies for Fine-Grained Video Anomaly Understanding
- 2025Knowledge-Guided Textual Reasoning for Explainable Video Anomaly Detection via LLMs
- 2026Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning
- 2025SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model
- 2026Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models