mlsanityData to 7 Oct 2026

Automated Vehicles Should be Connected with Natural Language

arXiv 2507.01059 ↗First posted 29 Jun 2025cs.MA cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Multi-agent collaborative driving promises improvements in traffic safety and efficiency through collective perception and decision making. However, existing communication media -- including raw sensor data, neural network features, and perception results -- suffer limitations in bandwidth efficiency, information completeness, and agent interoperability. Moreover, traditional approaches have largely ignored decision-level fusion, neglecting critical dimensions of collaborative driving. In this paper we argue that addressing these challenges requires a transition from purely perception-oriented data exchanges to explicit intent and reasoning communication using natural language. Natural language balances semantic density and communication bandwidth, adapts flexibly to real-time conditions, and bridges heterogeneous agent platforms. By enabling the direct communication of intentions, rationales, and decisions, it transforms collaborative driving from reactive perception-data sharing into proactive coordination, advancing safety, efficiency, and transparency in intelligent transportation systems.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 69 of its 102 references that are in the map. First posted on arXiv 29 Jun 2025.

Builds on

69 of 102 references.

References · 69

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  3. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  4. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  5. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  6. 2021OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communicationcited by 304
  7. 2020V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predictioncited by 292
  8. 2020Deep Learning Enabled Semantic Communication Systemscited by 282
  9. 2022Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Mapscited by 255
  10. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  11. 2019Cooper: Cooperative Perception for Connected Autonomous Vehicles based on 3D Point Cloudscited by 165
  12. 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
38 more
  1. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  2. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 124
  3. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  4. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Drivingcited by 122
  5. 2019Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensorscited by 110
  6. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  7. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  8. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  9. 2024V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perceptioncited by 65
  10. 2020DSDNet: Deep Structured self-Driving Networkcited by 61
  11. 2022Bridging the Domain Gap for Multi-Agent Perceptioncited by 60
  12. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 55
  13. 2024Multi-Frame, Lightweight & Efficient Vision-Language Models for Question Answering in Autonomous Drivingcited by 52
  14. 2022Model-Agnostic Multi-Agent Perception Frameworkcited by 51
  15. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  16. 2022Uncertainty Quantification of Collaborative Detection for Self-Drivingcited by 46
  17. 2025STAMP: Scalable Task And Model-agnostic Collaborative Perceptioncited by 44
  18. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  19. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  20. 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Drivingcited by 34
  21. 2024Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunitiescited by 32
  22. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  23. 2023Collaborative Multi-Object Tracking with Conformal Uncertainty Propagationcited by 27
  24. 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Modelscited by 27
  25. 2025LangCoop: Collaborative Driving with Languagecited by 25
  26. 2020Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Cameracited by 22
  27. 2020Cooperative LIDAR Object Detection via Feature Sharing in Deep Networkscited by 19
  28. 2024On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validationcited by 15
  29. 2025V2X-UniPool: Unifying Multimodal Perception and Knowledge Reasoning for Autonomous Drivingcited by 14
  30. 2024Large Language Models Powered Context-aware Motion Prediction in Autonomous Drivingcited by 13
  31. 2025V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridorscited by 13
  32. 2025CoLMDriver: LLM-based Negotiation Benefits Cooperative Autonomous Drivingcited by 13
  33. 2025CoCMT: Communication-Efficient Cross-Modal Transformer for Collaborative Perceptioncited by 13
  34. 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMscited by 12
  35. 2025AirV2X: Unified Air-Ground Vehicle-to-Everything Collaborationcited by 11
  36. 2024Pragmatic Communication in Multi-Agent Collaborative Perceptioncited by 10
  37. 2024A Digital Twin Framework for Physical-Virtual Integration in V2X-Enabled Connected Vehicle Corridorscited by 9
  38. 2025SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Drivingcited by 9

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Research Challenges and Progress in the End-to-End V2X Cooperative Autonomous Driving Competitioncited by 9
  2. 2025SafeCoop: Unravelling Full Stack Safety in Agentic Collaborative Drivingcited by 2
  3. 2026Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAscited by 1
  4. 2025Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentationcited by 0
  5. 2026AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulationcited by 0