mlsanityData to 14 Oct 2026

Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances

arXiv 2502.16804 ↗First posted 24 Feb 2025cs.MA cs.AIcited by 21

On the map: End-to-end autonomous driving → Large language models for driving

also in 1 more
Abstract (arXiv)

Autonomous Driving Systems (ADSs) are revolutionizing transportation by reducing human intervention, improving operational efficiency, and enhancing safety. Large Language Models (LLMs) have been integrated into ADSs to support high-level decision-making through their powerful reasoning, instruction-following, and communication abilities. However, LLM-based single-agent ADSs face three major challenges: limited perception, insufficient collaboration, and high computational demands. To address these issues, recent advances in LLM-based multi-agent ADSs leverage language-driven communication and coordination to enhance inter-agent collaboration. This paper provides a frontier survey of this emerging intersection between NLP and multi-agent ADSs. We begin with a background introduction to related concepts, followed by a categorization of existing LLM-based methods based on different agent interaction modes. We then discuss agent-human interactions in scenarios where LLM-based agents engage with humans. Finally, we summarize key applications, datasets, and challenges to support future research.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 49 of its 83 references that are in the map. First posted on arXiv 24 Feb 2025.

Builds on

49 of 83 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Hybrid LLM-DDQN based Joint Optimization of V2I Communication and Autonomous Driving11 Oct
  2. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF6 Jun
  3. 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agents5 Jun
  4. 2024DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences5 Jun
  5. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning9 Apr
  6. 2023Personalized Autonomous Driving with Large Language Models: Field Experiments14 Dec
  7. 2023A Survey on Multimodal Large Language Models for Autonomous Driving21 Nov
  8. 2023A Language Agent for Autonomous Driving17 Nov
  9. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving2 Nov
  10. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles12 Oct
  11. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving4 Oct
  12. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles19 Sep

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  6. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  8. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  9. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  10. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  11. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  12. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
25 more
  1. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  2. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  3. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  4. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  5. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Modelscited by 246
  6. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  7. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  8. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  9. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  10. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  11. 2023Collaborative Perception in Autonomous Driving: Methods, Datasets and Challengescited by 87
  12. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  13. 2016Dialogue Learning With Human-In-The-Loopcited by 46
  14. 2024Large Language Model-based Human-Agent Collaboration for Complex Task Solvingcited by 38
  15. 2024LC-LLM: Explainable Lane-Change Intention and Trajectory Predictions with Large Language Modelscited by 35
  16. 2024Collaborative Perception for Connected and Autonomous Driving: Challenges, Possible Solutions and Opportunitiescited by 32
  17. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  18. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  19. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  20. 2024AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Mergingcited by 22
  21. 2022Emergence of hierarchical reference systems in multi-agent communicationcited by 8
  22. 2024Enhancing Human-Centered Dynamic Scene Understanding via Multiple LLMs Collaborated Reasoningcited by 6
  23. 2024AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learningcited by 6
  24. 2024Hallucination Elimination and Semantic Enhancement Framework for Vision-Language Models in Traffic Scenarioscited by 4
  25. 2024Tapping in a Remote Vehicle's onboard LLM to Complement the Ego Vehicle's Field-of-Viewcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 11.

21 papers.

Sort:

By area:

  1. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  2. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  3. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 29
  4. 2026When Users Change Their Mind: Evaluating Interruptible Agents in Long-Horizon Web Navigationcited by 14
  5. 2025PSG-Agent: Personality-Aware Safety Guardrail for LLM-based Agentscited by 12
  6. 2025A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomycited by 9
  7. 2025Embracing Trustworthy Brain-Agent Collaboration as Paradigm Extension for Intelligent Assistive Technologiescited by 5
  8. 2025Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Frameworkcited by 3
  9. 2026When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditingcited by 3
  10. 2025FuncPoison: Poisoning Function Library to Hijack Multi-agent Autonomous Driving Systemscited by 2
  11. 2025MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extractioncited by 2
  12. 2026Weak-Link Optimization for Multi-Agent Reasoning and Collaborationcited by 1
9 more
  1. 2026What's Hidden Matters: Identifying Planning-Critical Occluded Agents using Vision-Language Modelscited by 1
  2. 2025From Human Intention to Action Prediction: Intention-Driven End-to-End Autonomous Drivingcited by 0
  3. 2026Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Sciencecited by 0
  4. 2026WaterVideoQA: ASV-Centric Perception and Rule-Compliant Reasoning via Multi-Modal Agentscited by 0
  5. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  6. 2026SwarmDrive: Semantic V2V Coordination for Latency-Constrained Cooperative Autonomous Drivingcited by 0
  7. 2026LLM-enabled Social Agentscited by 0
  8. 2026Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systemscited by 0
  9. 2026REARL: A Closed-loop Autonomous Driving Simulation Enhancement Framework with Real Traffic Data and Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Models for Human-like Autonomous Driving: A Survey
  2. 2024KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  4. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends
  5. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models
  6. 2026Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving
  7. 2026Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving
  8. 2025CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learning