SGTA: Scene-Graph Based Multi-Modal Traffic Agent for Video Understanding
Not on the map. Why?
Abstract (arXiv)
We present Scene-Graph Based Multi-Modal Traffic Agent (SGTA), a modular framework for traffic video understanding that combines structured scene graphs with multi-modal reasoning. It constructs a traffic scene graph from roadside videos using detection, tracking, and lane extraction, followed by tool-based reasoning over both symbolic graph queries and visual inputs. SGTA adopts ReAct to process interleaved reasoning traces from large language models with tool invocations, enabling interpretable decision-making for complex video questions. Experiments on selected TUMTraffic VideoQA dataset sample demonstrate that SGTA achieves competitive accuracy across multiple question types while providing transparent reasoning steps. These results highlight the potential of integrating structured scene representations with multi-modal agents for traffic video understanding.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 18 references that are in the map. First posted on arXiv 4 Apr 2026.
Builds on
17 of 18 references.
References · 17
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 425
- 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
- 2022DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detectioncited by 273
- 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 229
- 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
- 2024Large Multimodal Agents: A Surveycited by 105
- 2024RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformercited by 91
- 2021YOLOP: You Only Look Once for Panoptic Driving Perceptioncited by 82
- 2023BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detectioncited by 75
- 2024TUMTraf V2X Cooperative Perception Datasetcited by 67
- 2025TUMTraffic-VideoQA: A Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenescited by 23
5 more
- 2025When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysiscited by 21
- 2019Providentia -- A Large-Scale Sensor System for the Assistance of Autonomous Vehicles and Its Evaluationcited by 17
- 2024GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Eventscited by 16
- 2024WARM-3D: A Weakly-Supervised Sim2Real Domain Adaptation Framework for Roadside Monocular 3D Object Detectioncited by 9
- 2025Scenario Understanding of Traffic Scenes Through Large Visual Language Modelscited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026UniTraffic-Agent: Unified Traffic Video Reasoning for AI City Challenge 2026 Track 3 with Two Out-of-Domain Evaluations
- 2025Multi-Agent Visual-Language Reasoning for Comprehensive Highway Scene Understanding
- 2026UDVideoQA: A Traffic Video Question Answering Dataset for Multi-Object Spatio-Temporal Reasoning in Urban Dynamics
- 2026TAU-Agent: An Agentic Retrieval-Augmented Framework for Traffic Anomaly Understanding
- 2026Beyond Scene Priors: Fine-Grained Traffic Scene Reasoning with Benchmarking and Query-Guided Small-Object Focus
- 2026Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understanding
- 2026OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning
- 2025NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models