# Traffic Shaping and Hysteresis Mitigation Using Deep Reinforcement Learning in a Connected Driving Environment

arXiv: https://arxiv.org/abs/2302.03141 · HTML: https://mlsanity.com/paper/2302.03141/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 8 of its 50 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 8 of 8)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.04402/) (2021)
- [Large-Scale Traffic Signal Control Using a Novel Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1908.03761/) (2019)
- [A New Framework for Multi-Agent Reinforcement Learning -- Centralized Training and Exploration with Decentralized Execution via Policy Distillation](https://mlsanity.com/paper/1910.09152/) (2019)
- [A Deep Reinforcement Learning Approach for Fair Traffic Signal Control](https://mlsanity.com/paper/2107.10146/) (2021)
- [Leveraging the Capabilities of Connected and Autonomous Vehicles and Multi-Agent Reinforcement Learning to Mitigate Highway Bottleneck Congestion](https://mlsanity.com/paper/2010.05436/) (2020)
- [Dynamic optimal congestion pricing in multi-region urban networks by application of a Multi-Layer-Neural network](https://mlsanity.com/paper/2107.14585/) (2021)

## Abstract (arXiv)

A multi-agent deep reinforcement learning-based framework for traffic shaping. The proposed framework offers a key advantage over existing congestion management strategies which is the ability to mitigate hysteresis phenomena. Unlike existing congestion management strategies that focus on breakdown prevention, the proposed framework is extremely effective after breakdown formation. The proposed framework assumes partial connectivity between the automated vehicles which share information. The framework requires a basic level of autonomy defined by one-dimensional longitudinal control. This framework is primarily built using a centralized training, centralized execution multi-agent deep reinforcement learning approach, where longitudinal control is defined by signals of acceleration or deceleration commands which are then executed by all agents uniformly. The model undertaken for training and testing of the framework is based on the well-known Double Deep Q-Learning algorithm which takes the average state of flow within the traffic stream as the model input and outputs actions in the form of acceleration or deceleration values. We demonstrate the ability of the model to shape the state of traffic, mitigate the negative effects of hysteresis, and even improve traffic flow beyond its original level. This paper also identifies the minimum percentage of CAVs required to successfully shape the traffic under an assumption of uniformly distributed CAVs within the loop system. The framework illustrated in this work doesnt just show the theoretical applicability of reinforcement learning to tackle such challenges but also proposes a realistic solution that only requires partial connectivity and continuous monitoring of the average speed of the system, which can be achieved using readily available sensors that measure the speeds of vehicles in reasonable proximity to the CAVs.