# A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

arXiv: https://arxiv.org/abs/2109.11672 · HTML: https://mlsanity.com/paper/2109.11672/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Formulation of Deep Reinforcement Learning Architecture Toward Autonomous Driving for On-Ramp Merge](https://mlsanity.com/paper/1709.02066/) (2017)
- [Control of Connected and Automated Vehicles: State of the Art and Future Challenges](https://mlsanity.com/paper/1804.03757/) (2018)
- [Enhanced Mobility With Connectivity and Automation: A Review of Shared Autonomous Vehicle Systems](https://mlsanity.com/paper/1905.12602/) (2019)
- [Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space](https://mlsanity.com/paper/1803.09203/) (2018)
- [Deep Reinforcement Learning and Transportation Research: A Comprehensive Review](https://mlsanity.com/paper/2010.06187/) (2020)
- [A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities](https://mlsanity.com/paper/2011.03137/) (2020)
- [Demonstration of a Time-Efficient Mobility System Using a Scaled Smart City](https://mlsanity.com/paper/1903.01632/) (2019)
- [Cooperative Highway Work Zone Merge Control based on Reinforcement Learning in A Connected and Automated Environment](https://mlsanity.com/paper/2001.08581/) (2020)
- [A Decentralized Time- and Energy-Optimal Control Framework for Connected Automated Vehicles: From Simulation to Field Test](https://mlsanity.com/paper/1911.01380/) (2019)
- [Experimental Validation of a Real-Time Optimal Controller for Coordination of CAVs in a Multi-Lane Roundabout](https://mlsanity.com/paper/2001.11176/) (2020)

## Built on by (most cited first, 7 of 7)

- [Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications](https://mlsanity.com/paper/2306.06808/) (2023)
- [AI Recommendation Systems for Lane-Changing Using Adherence-Aware Reinforcement Learning](https://mlsanity.com/paper/2504.20187/) (2025)
- [Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey](https://mlsanity.com/paper/2508.20315/) (2025)
- [CorrA: Leveraging Large Language Models for Dynamic Obstacle Avoidance of Autonomous Vehicles](https://mlsanity.com/paper/2503.02076/) (2025)
- [On the Robustness of Derivative-free Methods for Linear Quadratic Regulator](https://mlsanity.com/paper/2506.12596/) (2025)
- [A Communication-Efficient Decentralized Actor-Critic Algorithm](https://mlsanity.com/paper/2510.19199/) (2025)
- [A Decentralized Partially Observable Team Decision Methodology with Delayed Information Sharing](https://mlsanity.com/paper/2609.26783/) (2026)

## Abstract (arXiv)

The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging scenario and propose a framework for coordinating CAVs such that stop-and-go driving is eliminated. We use a decentralized form of the actor-critic approach to deep reinforcement learning-multi-agent deep deterministic policy gradient. We demonstrate the coordination of CAVs through numerical simulations and show that a smooth traffic flow is achieved by eliminating stop-and-go driving. Videos and plots of the simulation results can be found at this supplemental hrefhttps://sites.google.com/view/ud-ids-lab/MADRLsite.