# V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Driving

arXiv: https://arxiv.org/abs/2506.02580 · HTML: https://mlsanity.com/paper/2506.02580/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning.
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- End-to-end autonomous driving / [Reasoning VLA and VLM driving policies](https://mlsanity.com/driving/reasoning-vla-and-vlm-driving-policies/)

## Builds on (most cited first, 20 of 20)

- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Explain Yourself! Leveraging Language Models for Commonsense Reasoning](https://mlsanity.com/paper/1906.02361/) (2019)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model](https://mlsanity.com/paper/2402.10828/) (2024)
- [V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Models](https://mlsanity.com/paper/2408.09251/) (2024)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models](https://mlsanity.com/paper/2411.13112/) (2024)
- [V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models](https://mlsanity.com/paper/2502.09980/) (2025)
- [LangCoop: Collaborative Driving with Language](https://mlsanity.com/paper/2504.13406/) (2025)
- [Self-Driving like a Human driver instead of a Robocar: Personalized comfortable driving experience for autonomous vehicles](https://mlsanity.com/paper/2001.03908/) (2020)
- [Probing Multimodal LLMs as World Models for Driving](https://mlsanity.com/paper/2405.05956/) (2024)
- [PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Driving](https://mlsanity.com/paper/2412.02025/) (2024)
- [Vehicle-to-Everything Cooperative Perception for Autonomous Driving](https://mlsanity.com/paper/2310.03525/) (2023)
- [CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perception](https://mlsanity.com/paper/2503.13504/) (2025)
- [Cross-Domain Transfer Learning using Attention Latent Features for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2411.06087/) (2024)
- [A Survey on Large Language Model-empowered Autonomous Driving](https://mlsanity.com/paper/2409.14165/) (2024)
- [LiDAR-based End-to-end Temporal Perception for Vehicle-Infrastructure Cooperation](https://mlsanity.com/paper/2411.14927/) (2024)

## Built on by (most cited first, 14 of 14)

- [AirV2X: Unified Air-Ground Vehicle-to-Everything Collaboration](https://mlsanity.com/paper/2506.19283/) (2025)
- [Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition](https://mlsanity.com/paper/2507.21610/) (2025)
- [DRAMA-X: A Fine-grained Intent Prediction and Risk Reasoning Benchmark For Driving](https://mlsanity.com/paper/2506.17590/) (2025)
- [Automated Vehicles Should be Connected with Natural Language](https://mlsanity.com/paper/2507.01059/) (2025)
- [Demystifying the Visual Quality Paradox in Multimodal Large Language Models](https://mlsanity.com/paper/2506.15645/) (2025)
- [Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance](https://mlsanity.com/paper/2508.01057/) (2025)
- [Large Foundation Models for Trajectory Prediction in Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2509.10570/) (2025)
- [SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving](https://mlsanity.com/paper/2510.18123/) (2025)
- [NexusFlow: Unifying Disparate Tasks under Partial Supervision via Invertible Flow Networks](https://mlsanity.com/paper/2512.06251/) (2025)
- [Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models](https://mlsanity.com/paper/2606.13840/) (2026)
- [Large Multimodal Agents for Intelligent Transportation Systems: Architectures, Evidence, and Deployment Challenges](https://mlsanity.com/paper/2608.08184/) (2026)
- [Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving](https://mlsanity.com/paper/2609.01659/) (2026)
- [LANTERN: A Closed-Loop Benchmark for VLM-Based Cooperative Driving with Temporally Grounded Warnings](https://mlsanity.com/paper/2609.06368/) (2026)
- [CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning](https://mlsanity.com/paper/2609.35823/) (2026)

## Abstract (arXiv)

Autonomous driving (AD) has achieved significant progress, yet single-vehicle perception remains constrained by sensing range and occlusions. Vehicle-to-Everything (V2X) communication addresses these limits by enabling collaboration across vehicles and infrastructure, but it also faces heterogeneity, synchronization, and latency constraints. Language models offer strong knowledge-driven reasoning and decision-making capabilities, but they are not inherently designed to process raw sensor streams and are prone to hallucination. We propose V2X-UniPool, the first framework that unifies V2X perception with language-based reasoning for knowledge-driven AD. It transforms multimodal V2X data into structured, language-based knowledge, organizes it in a time-indexed knowledge pool for temporally consistent reasoning, and employs Retrieval-Augmented Generation (RAG) to ground decisions in real-time context. Experiments on the real-world DAIR-V2X dataset show that V2X-UniPool achieves state-of-the-art planning accuracy and safety while reducing communication cost by more than 80%, achieving the lowest overhead among evaluated methods. These results highlight the promise of bridging V2X perception and language reasoning to advance scalable and trustworthy driving. Our code is available at: https://github.com/Xuewen2025/V2X-UniPool