mlsanityData to 14 Oct 2026

Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning

arXiv 1911.04175 ↗First posted 11 Nov 2019cs.LG cs.AI cs.MAcited by 53

Not on the map. Why?

Cited by 27 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The capability to learn and adapt to changes in the driving environment is crucial for developing autonomous driving systems that are scalable beyond geo-fenced operational design domains. Deep Reinforcement Learning (RL) provides a promising and scalable framework for developing adaptive learning based solutions. Deep RL methods usually model the problem as a (Partially Observable) Markov Decision Process in which an agent acts in a stationary environment to learn an optimal behavior policy. However, driving involves complex interaction between multiple, intelligent (artificial or human) agents in a highly non-stationary environment. In this paper, we propose the use of Partially Observable Markov Games(POSG) for formulating the connected autonomous driving problems with realistic assumptions. We provide a taxonomy of multi-agent learning environments based on the nature of tasks, nature of agents and the nature of the environment to help in categorizing various autonomous driving problems that can be addressed under the proposed formulation. As our main contributions, we provide MACAD-Gym, a Multi-Agent Connected, Autonomous Driving agent learning platform for furthering research in this direction. Our MACAD-Gym platform provides an extensible set of Connected Autonomous Driving (CAD) simulation environments that enable the research and development of Deep RL- based integrated sensing, perception, planning and control algorithms for CAD systems with unlimited operational design domain under realistic, multi-agent settings. We also share the MACAD-Agents that were trained successfully using the MACAD-Gym platform to learn control policies for multiple vehicle agents in a partially observable, stop-sign controlled, 3-way urban intersection environment with raw (camera) sensor observations.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 10; 2024: 14; 2025: 5; 2026: 1.

53 papers.

Sort:

By area:

  1. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  2. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  3. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 160
  4. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  5. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Drivingcited by 47
  6. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 45
  7. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  8. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learningcited by 40
  9. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  10. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  11. 2021Social Coordination and Altruism in Autonomous Drivingcited by 28
  12. 2023Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Drivingcited by 26
38 more
  1. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  2. 2022End-to-end Autonomous Driving with Semantic Depth Cloud Mapping and Multi-agentcited by 21
  3. 2021Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Trafficcited by 20
  4. 2023A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problemcited by 19
  5. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 18
  6. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 17
  7. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 17
  8. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  9. 2022Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptationcited by 12
  10. 2021Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Criticcited by 11
  11. 2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
  12. 2021Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policiescited by 10
  13. 2021LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammatescited by 9
  14. 2024MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaborationcited by 8
  15. 2023FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusioncited by 7
  16. 2022Distributional Reward Estimation for Effective Multi-Agent Deep Reinforcement Learningcited by 6
  17. 2022Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learningcited by 6
  18. 2024TorchDriveEnv: A Reinforcement Learning Benchmark for Autonomous Driving with Reactive, Realistic, and Diverse Non-Playable Characterscited by 6
  19. 2024Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependenciescited by 6
  20. 2024Scenario-Based Curriculum Generation for Multi-Agent Autonomous Drivingcited by 3
  21. 2024Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Reviewcited by 3
  22. 2025Getting SMARTER for Motion Planning in Autonomous Driving Systemscited by 3
  23. 2021Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environmentcited by 2
  24. 2024Learning to Model Diverse Driving Behaviors in Highly Interactive Autonomous Driving Scenarios with Multi-Agent Reinforcement Learningcited by 2
  25. 2024SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systemscited by 2
  26. 2022Concept-modulated model-based offline reinforcement learning for rapid generalizationcited by 1
  27. 2022LearningGroup: A Real-Time Sparse Training on FPGA via Learnable Weight Grouping for Multi-Agent Reinforcement Learningcited by 1
  28. 2023Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Informationcited by 1
  29. 2023Neurosymbolic Meta-Reinforcement Lookahead Learning Achieves Safe Self-Driving in Non-Stationary Environmentscited by 1
  30. 2023A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving Systemcited by 1
  31. 2025Finite-sample guarantees for data-driven forward-backward operator methodscited by 1
  32. 2022Cognitive Level-k Meta-Learning for Safe and Pedestrian-Aware Autonomous Drivingcited by 0
  33. 2023Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flowcited by 0
  34. 2023On Reducing Undesirable Behavior in Deep Reinforcement Learning Modelscited by 0
  35. 2024Collision Avoidance Verification of Multiagent Systems with Learned Policiescited by 0
  36. 2024A Measure for Level of Autonomy Based on Observable System Behaviorcited by 0
  37. 2024An End-to-End Collaborative Learning Approach for Connected Autonomous Vehicles in Occluded Scenarioscited by 0
  38. 2025Position: Emergent Machina Sapiens Urge Rethinking Multi-Agent Paradigmscited by 0

50 most cited of 53 All 53 (JSON)