mlsanityData to 7 Oct 2026

TAU-R1: Visual Language Model for Traffic Anomaly Understanding

arXiv 2603.19098 ↗First posted 19 Mar 2026cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

Abstract (arXiv)

Traffic Anomaly Understanding (TAU) is important for traffic safety in Intelligent Transportation Systems. Recent vision-language models (VLMs) have shown strong capabilities in video understanding. However, progress on TAU remains limited due to the lack of benchmarks and task-specific methodologies. To address this limitation, we introduce Roundabout-TAU, a dataset constructed from real-world roundabout videos collected in collaboration with the City of Carmel, Indiana. The dataset contains 342 clips and is annotated with more than 2,000 question-answer pairs covering multiple aspects of traffic anomaly understanding. Building on this benchmark, we propose TAU-R1, a two-layer vision-language framework for TAU. The first layer is a lightweight anomaly classifier that performs coarse anomaly categorisation, while the second layer is a larger anomaly reasoner that generates detailed event summaries. To improve task-specific reasoning, we introduce a two-stage training strategy consisting of decomposed-QA-enhanced supervised fine-tuning followed by TAU-GRPO, a GRPO-based post-training method with TAU-specific reward functions. Experimental results show that TAU-R1 achieves strong performance on both anomaly classification and reasoning tasks while maintaining deployment efficiency. The dataset and code are available at: https://github.com/siri-rouser/TAU-R1

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 59 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

41 of 59 references.

References · 41

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2025Qwen3-VL Technical Reportcited by 2,534
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  9. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  10. 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
  11. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 464
  12. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 403
29 more
  1. 2020Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervisioncited by 235
  2. 2024VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understandingcited by 113
  3. 2020Uncertainty-based Traffic Accident Anticipation with Spatio-Temporal Relational Learningcited by 91
  4. 2021UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detectioncited by 91
  5. 2024Harnessing Large Language Models for Training-free Video Anomaly Detectioncited by 85
  6. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  7. 2019Unsupervised Traffic Accident Detection in First-Person Videoscited by 69
  8. 2024Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularitycited by 66
  9. 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
  10. 2024Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Modelscited by 55
  11. 2024Advancing Video Anomaly Detection: A Concise Review and a New Datasetcited by 48
  12. 2024Hawk: Learning to Understand Open-World Video Anomaliescited by 47
  13. 2024Video Anomaly Detection and Explanation via Large Language Modelscited by 45
  14. 2024Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomalycited by 44
  15. 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
  16. 2025EventVAD: Training-Free Event-Aware Video Anomaly Detectioncited by 32
  17. 2024VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Modelscited by 24
  18. 2025Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thoughtcited by 23
  19. 2022TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillancecited by 20
  20. 2025AnyAnomaly: Zero-Shot Customizable Video Anomaly Detection with LVLMcited by 15
  21. 2025VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Treecited by 15
  22. 2025MoniTor: Exploiting Large Language Models with Instruction for Online Video Anomaly Detectioncited by 10
  23. 2025LAD-Reasoner: Tiny Multimodal Models are Good Reasoners for Logical Anomaly Detectioncited by 9
  24. 2025A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysiscited by 9
  25. 2025Real-time Traffic Accident Anticipation with Feature Reusecited by 8
  26. 2023MM-AU:Towards Multimodal Understanding of Advertisement Videoscited by 7
  27. 2025Aligning Effective Tokens with Video Anomaly in Large Language Modelscited by 7
  28. 2025SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Modelcited by 6
  29. 2025RoadSceneVQA: Benchmarking Visual Question Answering in Roadside Perception Systems for Intelligent Transportation Systemcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning
  2. 2024GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Events
  3. 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations
  4. 2025When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis
  5. 2025TUMTraffic-VideoQA: A Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenes
  6. 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning
  7. 2025CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution
  8. 2025Structured Prompting and Multi-Agent Knowledge Distillation for Traffic Video Interpretation and Risk Inference