# Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances

arXiv: https://arxiv.org/abs/2502.16804 · HTML: https://mlsanity.com/paper/2502.16804/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 49 of its 83 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Placed in

- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)
- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 30 of 49)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model](https://mlsanity.com/paper/2304.15010/) (2023)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [GPT-Driver: Learning to Drive with GPT](https://mlsanity.com/paper/2310.01415/) (2023)
- [V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer](https://mlsanity.com/paper/2203.10638/) (2022)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario](https://mlsanity.com/paper/2305.14836/) (2023)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Textual Explanations for Self-Driving Vehicles](https://mlsanity.com/paper/1807.11546/) (2018)
- [LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving](https://mlsanity.com/paper/2310.03026/) (2023)
- [A Language Agent for Autonomous Driving](https://mlsanity.com/paper/2311.10813/) (2023)
- [V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception](https://mlsanity.com/paper/2303.07601/) (2023)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles](https://mlsanity.com/paper/2309.10228/) (2023)
- [Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents](https://mlsanity.com/paper/2402.05746/) (2024)
- [Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges](https://mlsanity.com/paper/2301.06262/) (2023)
- [Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles](https://mlsanity.com/paper/2310.08034/) (2023)
- [Conditional Affordance Learning for Driving in Urban Environments](https://mlsanity.com/paper/1806.06498/) (2018)

## Built on by (most cited first, 21 of 21)

- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [Internet of Agents: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2505.07176/) (2025)
- [LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey](https://mlsanity.com/paper/2505.00753/) (2025)
- [When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigation](https://mlsanity.com/paper/2604.00892/) (2026)
- [PSG-Agent: Personality-Aware Safety Guardrail for LLM-based Agents](https://mlsanity.com/paper/2509.23614/) (2025)
- [A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy](https://mlsanity.com/paper/2506.09420/) (2025)
- [Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologies](https://mlsanity.com/paper/2510.22095/) (2025)
- [Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Framework](https://mlsanity.com/paper/2507.08459/) (2025)
- [When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditing](https://mlsanity.com/paper/2603.17445/) (2026)
- [FuncPoison: Poisoning Function Library to Hijack Multi-agent Autonomous Driving Systems](https://mlsanity.com/paper/2509.24408/) (2025)
- [MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extraction](https://mlsanity.com/paper/2510.05611/) (2025)
- [Weak-Link Optimization for Multi-Agent Reasoning and Collaboration](https://mlsanity.com/paper/2604.15972/) (2026)
- [What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Models](https://mlsanity.com/paper/2607.00283/) (2026)
- [From Human Intention to Action Prediction: Intention-Driven End-to-End Autonomous Driving](https://mlsanity.com/paper/2512.12302/) (2025)
- [Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science](https://mlsanity.com/paper/2602.05289/) (2026)
- [WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agents](https://mlsanity.com/paper/2602.22923/) (2026)
- [Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic](https://mlsanity.com/paper/2604.12857/) (2026)
- [SwarmDrive: Semantic V2V Coordination for Latency-Constrained Cooperative Autonomous Driving](https://mlsanity.com/paper/2604.22852/) (2026)
- [LLM-enabled Social Agents](https://mlsanity.com/paper/2605.02335/) (2026)
- [Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems](https://mlsanity.com/paper/2609.16599/) (2026)
- [REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Models](https://mlsanity.com/paper/2609.19903/) (2026)

## Abstract (arXiv)

Autonomous Driving Systems (ADSs) are revolutionizing transportation by reducing human intervention, improving operational efficiency, and enhancing safety. Large Language Models (LLMs) have been integrated into ADSs to support high-level decision-making through their powerful reasoning, instruction-following, and communication abilities. However, LLM-based single-agent ADSs face three major challenges: limited perception, insufficient collaboration, and high computational demands. To address these issues, recent advances in LLM-based multi-agent ADSs leverage language-driven communication and coordination to enhance inter-agent collaboration. This paper provides a frontier survey of this emerging intersection between NLP and multi-agent ADSs. We begin with a background introduction to related concepts, followed by a categorization of existing LLM-based methods based on different agent interaction modes. We then discuss agent-human interactions in scenarios where LLM-based agents engage with humans. Finally, we summarize key applications, datasets, and challenges to support future research.