# Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning

arXiv: https://arxiv.org/abs/2403.14879 · HTML: https://mlsanity.com/paper/2403.14879/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers](https://mlsanity.com/paper/1804.07871/) (2018)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning](https://mlsanity.com/paper/2306.08094/) (2023)
- [Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles](https://mlsanity.com/paper/1812.06120/) (2018)
- [Reinforcement Learning for Mixed Autonomy Intersections](https://mlsanity.com/paper/2111.04686/) (2021)
- [ADAPS: Autonomous Driving Via Principled Simulations](https://mlsanity.com/paper/1907.08874/) (2019)
- [Learning to Control and Coordinate Mixed Traffic Through Robot Vehicles at Complex and Unsignalized Intersections](https://mlsanity.com/paper/2301.05294/) (2023)
- [Large-scale Mixed Traffic Control Using Dynamic Vehicle Routing and Privacy-Preserving Crowdsourcing](https://mlsanity.com/paper/2311.11347/) (2023)
- [Mixed Traffic Control and Coordination from Pixels](https://mlsanity.com/paper/2302.09167/) (2023)
- [A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network](https://mlsanity.com/paper/2010.05437/) (2020)
- [EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning](https://mlsanity.com/paper/2311.12261/) (2023)
- [Optimal Control of Connected and Automated Vehicles at Roundabouts: An Investigation in a Mixed-Traffic Environment](https://mlsanity.com/paper/1710.11295/) (2017)
- [Long-term Microscopic Traffic Simulation with History-Masked Multi-agent Imitation Learning](https://mlsanity.com/paper/2306.06401/) (2023)

## Built on by (most cited first, 1 of 1)

- [CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control](https://mlsanity.com/paper/2404.00796/) (2024)

## Abstract (arXiv)

The management of mixed traffic that consists of robot vehicles (RVs) and human-driven vehicles (HVs) at complex intersections presents a multifaceted challenge. Traditional signal controls often struggle to adapt to dynamic traffic conditions and heterogeneous vehicle types. Recent advancements have turned to strategies based on reinforcement learning (RL), leveraging its model-free nature, real-time operation, and generalizability over different scenarios. We introduce a hierarchical RL framework to manage mixed traffic through precise longitudinal and lateral control of RVs. Our proposed hierarchical framework combines the state-of-the-art mixed traffic control algorithm as a high level decision maker to improve the performance and robustness of the whole system. Our experiments demonstrate that the framework can reduce the average waiting time by up to 54% compared to the state-of-the-art mixed traffic control method. When the RV penetration rate exceeds 60%, our technique consistently outperforms conventional traffic signal control programs in terms of the average waiting time for all vehicles at the intersection.