# UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations

arXiv: https://arxiv.org/abs/2608.13031 · HTML: https://mlsanity.com/paper/2608.13031/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 20 of 20)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023)
- [VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners](https://mlsanity.com/paper/2212.04979/) (2022)
- [ReconBoost: Boosting Can Achieve Modality Reconcilement](https://mlsanity.com/paper/2405.09321/) (2024)
- [Localizing Anomalies from Weakly-Labeled Videos](https://mlsanity.com/paper/2008.08944/) (2020)
- [Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought](https://mlsanity.com/paper/2505.19877/) (2025)
- [The 6th AI City Challenge](https://mlsanity.com/paper/2204.10380/) (2022)
- [FishEye8K: A Benchmark and Dataset for Fisheye Camera Object Detection](https://mlsanity.com/paper/2305.17449/) (2023)
- [TAD: A Large-Scale Benchmark for Traffic Accidents Detection from Video Surveillance](https://mlsanity.com/paper/2209.12386/) (2022)
- [TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioning](https://mlsanity.com/paper/2404.09275/) (2024)
- [Rethinking Collaborative Metric Learning: Toward an Efficient Alternative without Negative Sampling](https://mlsanity.com/paper/2206.11549/) (2022)
- [OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning](https://mlsanity.com/paper/2505.05180/) (2025)
- [Improved Diversity-Promoting Collaborative Metric Learning for Recommendation](https://mlsanity.com/paper/2409.01012/) (2024)
- [STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Models](https://mlsanity.com/paper/2508.13470/) (2025)
- [AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond](https://mlsanity.com/paper/2509.26636/) (2025)
- [Warehouse Spatial Question Answering with LLM Agent](https://mlsanity.com/paper/2507.10778/) (2025)

## Built on by (most cited first, 1 of 1)

- [The 10th AI City Challenge](https://mlsanity.com/paper/2608.17044/) (2026)

## Abstract (arXiv)

Traffic video understanding has become an important problem in intelligent transportation, as road videos provide direct evidence for accidents, violations, and interactions between vehicles and vulnerable road users. A useful system should explain how a traffic event develops, why it happens, and when the relevant interaction occurs, yet this remains difficult for multimodal large language models (MLLMs) because traffic videos contain sparse events and varied viewpoints. We introduce UniTraffic-Agent, the MR-CAS solution for Track~3 of the 10th AI City Challenge, which includes Traffic Anomaly Reasoning (TAR) and two out-of-domain evaluations: FETV for fisheye traffic events and PSI-VQA for pedestrian intention reasoning. UniTraffic-Agent follows an observe--reason--act--verify workflow that samples timestamped visual evidence, reasons over all questions from the same clip in one request, and converts responses through task-specific action adapters. On the official Public leaderboards, MR-CAS ranks 16th on TAR with a score of 0.5780, 2nd on FETV with 0.4884, and 4th on PSI-VQA with 64.4161. The code is available at https://github.com/Roclp/UniTraffic-Agent.