mlsanityData to 7 Oct 2026

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

arXiv 2408.09675 ↗First posted 19 Aug 2024cs.AI cs.MA cs.ROcited by 30

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Reinforcement Learning (RL) is a potent tool for sequential decision-making and has achieved performance surpassing human capabilities across many challenging real-world tasks. As the extension of RL in the multi-agent system domain, multi-agent RL (MARL) not only need to learn the control policy but also requires consideration regarding interactions with all other agents in the environment, mutual influences among different system components, and the distribution of computational resources. This augments the complexity of algorithmic design and poses higher requirements on computational resources. Simultaneously, simulators are crucial to obtain realistic data, which is the fundamentals of RL. In this paper, we first propose a series of metrics of simulators and summarize the features of existing benchmarks. Second, to ease comprehension, we recall the foundational knowledge and then synthesize the recently advanced studies of MARL-related autonomous driving and intelligent transportation systems. Specifically, we examine their environmental modeling, state representation, perception units, and algorithm design. Conclusively, we discuss open challenges as well as prospects and opportunities. We hope this paper can help the researchers integrate MARL technologies and trigger more insightful ideas toward the intelligent and autonomous driving.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 158 of its 247 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

158 of 247 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset3 Apr
  2. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving31 Oct
  3. 2022Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving24 Aug
  4. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization17 Feb
  5. 2021Safe Driving via Expert Guided Policy Optimization13 Oct
  6. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
  7. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving15 Apr

Other references · 151

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  10. 2015Continuous control with deep reinforcement learningcited by 3,279
  11. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  12. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
38 more
  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  9. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  10. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  11. 2016End to End Learning for Self-Driving Carscited by 1,000
  12. 2020Mastering Atari with Discrete World Modelscited by 990
  13. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  14. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  15. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  16. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  17. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  18. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  19. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  20. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  21. 2019The StarCraft Multi-Agent Challengecited by 589
  22. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  23. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  24. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  25. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  26. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  27. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  28. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  29. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  30. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  31. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  32. 2021Deep Long-Tailed Learning: A Surveycited by 416
  33. 2016Learning Multiagent Communication with Backpropagationcited by 355
  34. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  35. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  36. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  37. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  38. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251

50 most cited of 151 All 151 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 18; 2026: 11.

30 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectivescited by 27
  2. 2025LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarioscited by 25
  3. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
  4. 2024Language-Driven Policy Distillation for Cooperative Driving in Multi-Agent Reinforcement Learningcited by 13
  5. 2025CoLMDriver: LLM-based Negotiation Benefits Cooperative Autonomous Drivingcited by 13
  6. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learningcited by 10
  7. 2025OM2P: Offline Multi-Agent Mean-Flow Policycited by 7
  8. 2025Deep Meta Coordination Graphs for Multi-agent Reinforcement Learningcited by 6
  9. 2025QLLM: Do We Really Need a Mixing Network for Credit Assignment in Multi-Agent Reinforcement Learning?cited by 6
  10. 2025Automated Vehicles Should be Connected with Natural Languagecited by 5
  11. 2026GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent Systemscited by 3
  12. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
18 more
  1. 2026Puzzle it Out: Local-to-Global World Model for Offline Multi-Agent Reinforcement Learningcited by 2
  2. 2025Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learningcited by 1
  3. 2025PolicyEvolve: Evolving Programmatic Policies by LLMs for multi-player games via Population-Based Trainingcited by 1
  4. 2025From Solo to Symphony: Orchestrating Multi-Agent Collaboration with Single-Agent Demoscited by 1
  5. 2026Markov Potential Game and Multi-Agent Reinforcement Learning for Autonomous Drivingcited by 1
  6. 2025Learning to Drive by Imitating Surrounding Vehiclescited by 0
  7. 2025Deep Learning Agents Trained For Avoidance Behave Like Hawks And Dovescited by 0
  8. 2025Dynamic Sight Range Selection in Multi-Agent Reinforcement Learningcited by 0
  9. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Reviewcited by 0
  10. 2025Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learningcited by 0
  11. 2026Agentic AI for Self-Driving Laboratories in Soft Matter: Taxonomy, Benchmarks,and Open Challengescited by 0
  12. 2026Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environmentcited by 0
  13. 2026QSIM: Mitigating Overestimation in Multi-Agent Reinforcement Learning via Action Similarity Weighted Q-Learningcited by 0
  14. 2026COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systemscited by 0
  15. 2026Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertaintycited by 0
  16. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Controlcited by 0
  17. 2026ACPO: Agent-Chained Policy Optimization for Multi-Agent Reinforcement Learningcited by 0
  18. 2026DriveZero: End-to-End Driving Beyond Human Demonstrationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  2. 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
  3. 2024Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study
  4. 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systems
  5. 2026Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Driving
  6. 2025Impact of Collective Behaviors of Autonomous Vehicles on Urban Traffic Dynamics: A Multi-Agent Reinforcement Learning Approach
  7. 2021A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles
  8. 2022Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning