V2X-QA: A Comprehensive Reasoning Dataset and Benchmark for Multimodal Large Language Models in Autonomous Driving Across Ego, Infrastructure, and Cooperative Views
On the map: LLM reasoning → Chain-of-thought reasoning for driving
also in 3 more
Abstract (arXiv)
Multimodal large language models (MLLMs) have shown strong potential for autonomous driving, yet existing benchmarks remain largely ego-centric and therefore cannot systematically assess model performance in infrastructure-centric and cooperative driving conditions. In this work, we introduce V2X-QA, a real-world dataset and benchmark for evaluating MLLMs across vehicle-side, infrastructure-side, and cooperative viewpoints. V2X-QA is built around a view-decoupled evaluation protocol that enables controlled comparison under vehicle-only, infrastructure-only, and cooperative driving conditions within a unified multiple-choice question answering (MCQA) framework. The benchmark is organized into a twelve-task taxonomy spanning perception, prediction, and reasoning and planning, and is constructed through expert-verified MCQA annotation to enable fine-grained diagnosis of viewpoint-dependent capabilities. Benchmark results across ten representative state-of-the-art proprietary and open-source models show that viewpoint accessibility substantially affects performance, and infrastructure-side reasoning supports meaningful macroscopic traffic understanding. Results also indicate that cooperative reasoning remains challenging since it requires cross-view alignment and evidence integration rather than simply additional visual input. To address these challenges, we introduce V2X-MoE, a benchmark-aligned baseline with explicit view routing and viewpoint-specific LoRA experts. The strong performance of V2X-MoE further suggests that explicit viewpoint specialization is a promising direction for multi-view reasoning in autonomous driving. Overall, V2X-QA provides a foundation for studying multi-perspective reasoning, reliability, and cooperative physical intelligence in connected autonomous driving. The dataset and V2X-MoE resources are publicly available at: https://github.com/junwei0001/V2X-QA.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 15 of its 44 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
15 of 44 references.
Closest ancestors · 3
Same direction, newest first
- 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving25 Nov
- 2025NuRisk: A Visual Question Answering Dataset for Agent-Level Risk Assessment in Autonomous Driving30 Sep
- 2025V2V-GoT: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models and Graph-of-Thoughts22 Sep
Other references · 12
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 49
- 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Modelscited by 27
- 2025UrbanIng-V2X: A Large-Scale Multi-Vehicle, Multi-Infrastructure Dataset Across Multiple Intersections for Cooperative Perceptioncited by 15
- 2025Planning Safety Trajectories with Dual-Phase, Physics-Informed, and Transportation Knowledge-Driven Large Language Modelscited by 5
- 2026HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Drivingcited by 5
- 2025SEAL: Vision-Language Model-Based Safe End-to-End Cooperative Autonomous Driving with Adaptive Long-Tail Modelingcited by 3
- 2025CATS-V2V: A Real-World Vehicle-to-Vehicle Cooperative Perception Dataset with Complex Adverse Traffic Scenarioscited by 2
- 2025LLM-Driven Kernel Evolution: Automating Driver Updates in Linuxcited by 1
- 2026ScenePilot-4K: A Large-Scale First-Person Dataset and Benchmark for Vision-Language Models in Autonomous Drivingcited by 1
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systemscited by 4
- 2026D2-V2X: Depth-Driven Cooperative V2X Reasoning for Autonomous Drivingcited by 1
- 2026RISE: Roadside Infrastructure Sequence Understanding across 3D Tracking and Structured Vision-Language Reasoningcited by 0
- 2026Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answeringcited by 0
- 2026LANTERN: A Closed-Loop Benchmark for VLM-Based Cooperative Driving with Temporally Grounded Warningscited by 0
- 2026CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planningcited by 0
- 2026CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning22 Sep 2026
- 2026LANTERN: A Closed-Loop Benchmark for VLM-Based Cooperative Driving with Temporally Grounded Warnings6 Sep 2026
- 2026Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering28 Aug 2026
- 2026RISE: Roadside Infrastructure Sequence Understanding across 3D Tracking and Structured Vision-Language Reasoning17 Aug 2026
- 2026D2-V2X: Depth-Driven Cooperative V2X Reasoning for Autonomous Driving22 May 2026
- 2026MDrive: Benchmarking Closed-Loop Cooperative Driving for End-to-End Multi-agent Systems11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Driving
- 2025V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Driving
- 2025A Survey on Vision-Language-Action Models for Autonomous Driving
- 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving
- 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
- 2026GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving
- 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving
- 2024Application of Multimodal Large Language Models in Autonomous Driving