# Cooperative Advisory Residual Policies for Congestion Mitigation

arXiv: https://arxiv.org/abs/2407.00553 · HTML: https://mlsanity.com/paper/2407.00553/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 86 references that are in the map.
- First posted on arXiv 30 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments](https://mlsanity.com/paper/1705.01693/) (2017)
- [Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncing](https://mlsanity.com/paper/1808.03246/) (2018)
- [Simultaneous Policy Learning and Latent State Inference for Imitating Driver Behavior](https://mlsanity.com/paper/1704.05566/) (2017)
- [Reinforcement Learning for Mixed Autonomy Intersections](https://mlsanity.com/paper/2111.04686/) (2021)
- [Unified Automatic Control of Vehicular Systems with Reinforcement Learning](https://mlsanity.com/paper/2208.00268/) (2022)
- [A Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behavior](https://mlsanity.com/paper/2006.08832/) (2020)
- [Learning Eco-Driving Strategies at Signalized Intersections](https://mlsanity.com/paper/2204.12561/) (2022)
- [Learning to Navigate Intersections with Unsupervised Driver Trait Inference](https://mlsanity.com/paper/2109.06783/) (2021)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [Cooperative Driving for Speed Harmonization in Mixed-Traffic Environments](https://mlsanity.com/paper/2302.07453/) (2023)
- [PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems](https://mlsanity.com/paper/2308.00864/) (2023)
- [CAN Coach: Vehicular Control through Human Cyber-Physical Systems](https://mlsanity.com/paper/2104.06264/) (2021)
- [Integrated Analysis of Coarse-Grained Guidance for Traffic Flow Stability](https://mlsanity.com/paper/2301.04043/) (2023)
- [Enabling Mixed Autonomy Traffic Control](https://mlsanity.com/paper/2310.18776/) (2023)
- [Mitigation of stop-and-go traffic waves with intelligent vehicles at low market penetration rates](https://mlsanity.com/paper/2309.01834/) (2023)

## Built on by (most cited first, 1 of 1)

- [Model-Based Transfer Learning for Contextual Reinforcement Learning](https://mlsanity.com/paper/2408.04498/) (2024)

## Abstract (arXiv)

Fleets of autonomous vehicles can mitigate traffic congestion through simple actions, thus improving many socioeconomic factors such as commute time and gas costs. However, these approaches are limited in practice as they assume precise control over autonomous vehicle fleets, incur extensive installation costs for a centralized sensor ecosystem, and also fail to account for uncertainty in driver behavior. To this end, we develop a class of learned residual policies that can be used in cooperative advisory systems and only require the use of a single vehicle with a human driver. Our policies advise drivers to behave in ways that mitigate traffic congestion while accounting for diverse driver behaviors, particularly drivers' reactions to instructions, to provide an improved user experience. To realize such policies, we introduce an improved reward function that explicitly addresses congestion mitigation and driver attitudes to advice. We show that our residual policies can be personalized by conditioning them on an inferred driver trait that is learned in an unsupervised manner with a variational autoencoder. Our policies are trained in simulation with our novel instruction adherence driver model, and evaluated in simulation and through a user study (N=16) to capture the sentiments of human drivers. Our results show that our approaches successfully mitigate congestion while adapting to different driver behaviors, with up to 20% and 40% improvement as measured by a combination metric of speed and deviations in speed across time over baselines in our simulation tests and user study, respectively. Our user study further shows that our policies are human-compatible and personalize to drivers.