# Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction

arXiv: https://arxiv.org/abs/2112.03759 · HTML: https://mlsanity.com/paper/2112.03759/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments](https://mlsanity.com/paper/1705.01693/) (2017)
- [Robust Reinforcement Learning via Adversarial training with Langevin Dynamics](https://mlsanity.com/paper/2002.06063/) (2020)
- [Scalable Multiagent Driving Policies For Reducing Traffic Congestion](https://mlsanity.com/paper/2103.00058/) (2021)
- [Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL](https://mlsanity.com/paper/2011.00120/) (2020)

## Built on by (most cited first, 4 of 4)

- [Decentralized Traffic Flow Optimization Through Intrinsic Motivation](https://mlsanity.com/paper/2505.11520/) (2025)
- [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023)
- [Highway Congestion Reduction through Reinforcement Learning Based Eulerian Headway Control](https://mlsanity.com/paper/2412.02520/) (2024)
- [Multi-Agent Coordination in Autonomous Vehicle Routing: A Simulation-Based Study of Communication, Memory, and Routing Loops](https://mlsanity.com/paper/2511.17656/) (2025)

## Abstract (arXiv)

In most modern cities, traffic congestion is one of the most salient societal challenges. Past research has shown that inserting a limited number of autonomous vehicles (AVs) within the traffic flow, with driving policies learned specifically for the purpose of reducing congestion, can significantly improve traffic conditions. However, to date these AV policies have generally been evaluated under the same limited conditions under which they were trained. On the other hand, to be considered for practical deployment, they must be robust to a wide variety of traffic conditions. This article establishes for the first time that a multiagent driving policy can be trained in such a way that it generalizes to different traffic flows, AV penetration, and road geometries, including on multi-lane roads. Inspired by our successful results in a high-fidelity microsimulation, this article further contributes a novel extension of the well-known Cell Transmission Model (CTM) that, unlike past CTMs, is suitable for modeling congestion in traffic networks, and is thus suitable for studying congestion-reduction policies such as those considered in this article.