# Automated Vehicles Should be Connected with Natural Language

arXiv: https://arxiv.org/abs/2507.01059 · HTML: https://mlsanity.com/paper/2507.01059/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 102 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 30 of 69)

- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2304.14365/) (2023)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication](https://mlsanity.com/paper/2109.07644/) (2021)
- [Deep Learning Enabled Semantic Communication Systems](https://mlsanity.com/paper/2006.10685/) (2020)
- [V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction](https://mlsanity.com/paper/2008.07519/) (2020)
- [Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps](https://mlsanity.com/paper/2209.12836/) (2022)
- [CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers](https://mlsanity.com/paper/2207.02202/) (2022)
- [Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Clouds](https://mlsanity.com/paper/1905.05265/) (2019)
- [When2com: Multi-Agent Perception via Communication Graph Grouping](https://mlsanity.com/paper/2006.00176/) (2020)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigs](https://mlsanity.com/paper/2203.04050/) (2022)
- [Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving](https://mlsanity.com/paper/2312.03661/) (2023)
- [Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors](https://mlsanity.com/paper/1912.12147/) (2019)
- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [An Extensible Framework for Open Heterogeneous Collaborative Perception](https://mlsanity.com/paper/2401.13964/) (2024)
- [SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving](https://mlsanity.com/paper/2306.09001/) (2023)
- [V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception](https://mlsanity.com/paper/2403.16034/) (2024)
- [DSDNet: Deep Structured self-Driving Network](https://mlsanity.com/paper/2008.06041/) (2020)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Game-theoretic LLM: Agent Workflow for Negotiation Games](https://mlsanity.com/paper/2411.05990/) (2024)
- [Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Driving](https://mlsanity.com/paper/2403.19838/) (2024)
- [Model-Agnostic Multi-Agent Perception Framework](https://mlsanity.com/paper/2203.13168/) (2022)
- [V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Models](https://mlsanity.com/paper/2408.09251/) (2024)
- [Uncertainty Quantification of Collaborative Detection for Self-Driving](https://mlsanity.com/paper/2209.08162/) (2022)
- [STAMP: Scalable Task And Model-agnostic Collaborative Perception](https://mlsanity.com/paper/2501.18616/) (2025)
- [Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perception](https://mlsanity.com/paper/2308.16714/) (2023)

## Built on by (most cited first, 5 of 5)

- [Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competition](https://mlsanity.com/paper/2507.21610/) (2025)
- [SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Driving](https://mlsanity.com/paper/2510.18123/) (2025)
- [Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs](https://mlsanity.com/paper/2606.23938/) (2026)
- [Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentation](https://mlsanity.com/paper/2509.05751/) (2025)
- [AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation](https://mlsanity.com/paper/2609.19527/) (2026)

## Abstract (arXiv)

Multi-agent collaborative driving promises improvements in traffic safety and efficiency through collective perception and decision making. However, existing communication media -- including raw sensor data, neural network features, and perception results -- suffer limitations in bandwidth efficiency, information completeness, and agent interoperability. Moreover, traditional approaches have largely ignored decision-level fusion, neglecting critical dimensions of collaborative driving. In this paper we argue that addressing these challenges requires a transition from purely perception-oriented data exchanges to explicit intent and reasoning communication using natural language. Natural language balances semantic density and communication bandwidth, adapts flexibly to real-time conditions, and bridges heterogeneous agent platforms. By enabling the direct communication of intentions, rationales, and decisions, it transforms collaborative driving from reactive perception-data sharing into proactive coordination, advancing safety, efficiency, and transparency in intelligent transportation systems.