# CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems arXiv: https://arxiv.org/abs/2501.06132 ยท HTML: https://mlsanity.com/paper/2501.06132/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 56 references that are in the map. - First posted on arXiv 10 Jan 2025. ## Builds on (most cited first, 28 of 28) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023) - [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017) - [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022) - [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023) - [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024) - [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023) - [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023) - [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023) - [PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning](https://mlsanity.com/paper/1809.03531/) (2018) - [Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving](https://mlsanity.com/paper/2310.02251/) (2023) - [PRIMAL2: Pathfinding via Reinforcement and Imitation Multi-Agent Learning -- Lifelong](https://mlsanity.com/paper/2010.08184/) (2020) - [Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systems](https://mlsanity.com/paper/1709.07032/) (2017) - [Hybrid Multi-agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2212.07313/) (2022) - [Towards Socially Responsive Autonomous Vehicles: A Reinforcement Learning Framework with Driving Priors and Coordination Awareness](https://mlsanity.com/paper/2309.09726/) (2023) - [Edge-enabled V2X Service Placement for Intelligent Transportation Systems](https://mlsanity.com/paper/2001.06288/) (2020) - [Alternating Direction Method of Multipliers for Constrained Iterative LQR in Autonomous Driving](https://mlsanity.com/paper/2011.00462/) (2020) - [A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systems](https://mlsanity.com/paper/2209.08230/) (2022) - [Cooperative Behavior Planning for Automated Driving using Graph Neural Networks](https://mlsanity.com/paper/2202.11376/) (2022) - [Decentralized iLQR for Cooperative Trajectory Planning of Connected Autonomous Vehicles via Dual Consensus ADMM](https://mlsanity.com/paper/2301.04386/) (2023) - [Improved Consensus ADMM for Cooperative Motion Planning of Large-Scale Connected Autonomous Vehicles with Limited Communication](https://mlsanity.com/paper/2401.09032/) (2024) - [Regulating Mobility-on-Demand Services: Tri-level Model and Bayesian Optimization Solution Approach](https://mlsanity.com/paper/2102.10382/) (2021) - [Distributed and Constrained H2 Control Design via System Level Synthesis and Dual Consensus ADMM](https://mlsanity.com/paper/2207.06947/) (2022) - [Integrated Behavior Planning and Motion Control for Autonomous Vehicles with Traffic Rules Compliance](https://mlsanity.com/paper/2304.01041/) (2023) - [Recent Surge in Public Interest in Transportation: Sentiment Analysis of Baidu Apollo Go Using Weibo Data](https://mlsanity.com/paper/2408.10088/) (2024) - [Parallel Optimization with Hard Safety Constraints for Cooperative Planning of Connected Autonomous Vehicles](https://mlsanity.com/paper/2303.03090/) (2023) - [Electric Autonomous Mobility-on-Demand: Jointly Optimal Vehicle Design and Fleet Operation](https://mlsanity.com/paper/2309.13012/) (2023) ## Built on by (most cited first, 4 of 4) - [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025) - [DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planning](https://mlsanity.com/paper/2505.05360/) (2025) - [CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution](https://mlsanity.com/paper/2503.01632/) (2025) - [VLM-UDMC: VLM-Enhanced Unified Decision-Making and Motion Control for Urban Autonomous Driving](https://mlsanity.com/paper/2507.15266/) (2025) ## Abstract (arXiv) The increasing demand for flexible and efficient urban transportation solutions has spotlighted the limitations of traditional Demand Responsive Transport (DRT) systems, particularly in accommodating diverse passenger needs and dynamic urban environments. Autonomous Mobility-on-Demand (AMoD) systems have emerged as a promising alternative, leveraging connected and autonomous vehicles (CAVs) to provide responsive and adaptable services. However, existing methods primarily focus on either vehicle scheduling or path planning, which often simplify complex urban layouts and neglect the necessity for simultaneous coordination and mutual avoidance among CAVs. This oversimplification poses significant challenges to the deployment of AMoD systems in real-world scenarios. To address these gaps, we propose CoDriveVLM, a novel framework that integrates high-fidelity simultaneous dispatching and cooperative motion planning for future AMoD systems. Our method harnesses Vision-Language Models (VLMs) to enhance multi-modality information processing, and this enables comprehensive dispatching and collision risk evaluation. The VLM-enhanced CAV dispatching coordinator is introduced to effectively manage complex and unforeseen AMoD conditions, thus supporting efficient scheduling decision-making. Furthermore, we propose a scalable decentralized cooperative motion planning method via consensus alternating direction method of multipliers (ADMM) focusing on collision risk evaluation and decentralized trajectory optimization. Simulation results demonstrate the feasibility and robustness of CoDriveVLM in various traffic conditions, showcasing its potential to significantly improve the fidelity and effectiveness of AMoD systems in future urban transportation networks. The code is available at https://github.com/henryhcliu/CoDriveVLM.git.