# Multi-Agent Reinforcement Learning-based Cooperative Autonomous Driving in Smart Intersections

arXiv: https://arxiv.org/abs/2505.04231 · HTML: https://mlsanity.com/paper/2505.04231/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 11 of 11)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections](https://mlsanity.com/paper/1911.07602/) (2019)
- [Smart Mobility Digital Twin Based Automated Vehicle Navigation System: A Proof of Concept](https://mlsanity.com/paper/2402.12682/) (2024)
- [Digital Twin-Assisted Cooperative Driving at Non-Signalized Intersections](https://mlsanity.com/paper/2105.01357/) (2021)
- [Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills](https://mlsanity.com/paper/2309.13614/) (2023)
- [Digital Twins for Autonomous Driving: A Comprehensive Implementation and Demonstration](https://mlsanity.com/paper/2401.08653/) (2023)
- [Proof of Travel for Trust-Based Data Validation in V2I Communication](https://mlsanity.com/paper/2104.05070/) (2021)
- [Roadside Units Assisted Localized Automated Vehicle Maneuvering: An Offline Reinforcement Learning Approach](https://mlsanity.com/paper/2405.03935/) (2024)

## Built on by (most cited first, 2 of 2)

- [Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty](https://mlsanity.com/paper/2605.20255/) (2026)
- [Multi-Agent Embodied Autonomous Driving (MAEAD): From V2X Information Exchange to Shared World Models](https://mlsanity.com/paper/2606.13840/) (2026)

## Abstract (arXiv)

Unsignalized intersections pose significant safety and efficiency challenges due to complex traffic flows. This paper proposes a novel roadside unit (RSU)-centric cooperative driving system leveraging global perception and vehicle-to-infrastructure (V2I) communication. The core of the system is an RSU-based decision-making module using a two-stage hybrid reinforcement learning (RL) framework. At first, policies are pre-trained offline using conservative Q-learning (CQL) combined with behavior cloning (BC) on collected dataset. Subsequently, these policies are fine-tuned in the simulation using multi-agent proximal policy optimization (MAPPO), aligned with a self-attention mechanism to effectively solve inter-agent dependencies. RSUs perform real-time inference based on the trained models to realize vehicle control via V2I communications. Extensive experiments in CARLA environment demonstrate high effectiveness of the proposed system, by: textit(i) achieving failure rates below 0.03% in coordinating three connected and autonomous vehicles (CAVs) through complex intersection scenarios, significantly outperforming the traditional Autoware control method, and textit(ii) exhibiting strong robustness across varying numbers of controlled agents and shows promising generalization capabilities on other maps.