# From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning

arXiv: https://arxiv.org/abs/2608.10317 · HTML: https://mlsanity.com/paper/2608.10317/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)

## Builds on (most cited first, 20 of 20)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models](https://mlsanity.com/paper/2407.11691/) (2024)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity](https://mlsanity.com/paper/2412.06171/) (2024)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges](https://mlsanity.com/paper/2309.13925/) (2023)
- [CADP: A Novel Dataset for CCTV Traffic Camera based Accident Analysis](https://mlsanity.com/paper/1809.05782/) (2018)
- [Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought](https://mlsanity.com/paper/2505.19877/) (2025)
- [FishEye8K: A Benchmark and Dataset for Fisheye Camera Object Detection](https://mlsanity.com/paper/2305.17449/) (2023)
- [TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillance](https://mlsanity.com/paper/2209.12386/) (2022)
- [SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Models](https://mlsanity.com/paper/2505.12589/) (2025)
- [PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions](https://mlsanity.com/paper/2112.02604/) (2021)
- [The 9th AI City Challenge](https://mlsanity.com/paper/2508.13564/) (2025)
- [AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond](https://mlsanity.com/paper/2509.26636/) (2025)
- [MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks](https://mlsanity.com/paper/2605.21917/) (2026)

## Built on by (most cited first, 1 of 1)

- [The 10th AI City Challenge](https://mlsanity.com/paper/2608.17044/) (2026)

## Abstract (arXiv)

Detecting a traffic anomaly does not establish whether a video-language model can explain what happened, localize it in time, or identify its causes. We introduce TAR (Traffic Anomaly Reasoning) and TAR-Bench, paired resources for training and evaluating these complementary capabilities across 10 tasks spanning question answering, temporal reasoning, and scene understanding. TAR contains 44,040 automatically generated annotations with chain-of-thought traces for 3,670 CCTV videos from eight public datasets. TAR-Bench provides 960 human-curated annotations for 80 held-out clips from 17 public YouTube videos. Evaluation of eleven vision-language models reveals a gap between question-answering performance and temporal or scene reasoning. Progressively adding task groups during supervised fine-tuning improves aggregate performance on both Cosmos-Reason2-8B and Qwen3-VL-8B-Instruct. Training on all 10 tasks raises their mean benchmark scores from 34.3 to 55.7 and from 30.9 to 53.9, respectively. These results support joint training across complementary tasks as a promising approach to traffic anomaly understanding, while highlighting persistent limitations in temporal precision and causal attribution. TAR and TAR-Bench serve as the official training and in-domain evaluation resources for AI City Challenge 2026 Track 3.The dataset is available at https://huggingface.co/datasets/nvidia/PhysicalAI-Traffic-Anomaly-Reasoning