# Reinforcement Learning for Mixed Autonomy Intersections

arXiv: https://arxiv.org/abs/2111.04686 · HTML: https://mlsanity.com/paper/2111.04686/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 8 Nov 2021.

## Builds on (most cited first, 4 of 4)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Policy Distillation](https://mlsanity.com/paper/1511.06295/) (2015)
- [CoLight: Learning Network-level Cooperation for Traffic Signal Control](https://mlsanity.com/paper/1905.05717/) (2019)
- [Polling-systems-based Autonomous Vehicle Coordination in Traffic Intersections with No Traffic Signals](https://mlsanity.com/paper/1607.07896/) (2016)

## Built on by (most cited first, 16 of 16)

- [Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning](https://mlsanity.com/paper/2306.08094/) (2023)
- [Unified Automatic Control of Vehicular Systems with Reinforcement Learning](https://mlsanity.com/paper/2208.00268/) (2022)
- [Learning to Control and Coordinate Mixed Traffic Through Robot Vehicles at Complex and Unsignalized Intersections](https://mlsanity.com/paper/2301.05294/) (2023)
- [Mixed Traffic Control and Coordination from Pixels](https://mlsanity.com/paper/2302.09167/) (2023)
- [Learning Eco-Driving Strategies at Signalized Intersections](https://mlsanity.com/paper/2204.12561/) (2022)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems](https://mlsanity.com/paper/2308.00864/) (2023)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2504.04691/) (2025)
- [Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2403.14879/) (2024)
- [A Distributed Approach to Autonomous Intersection Management via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2405.08655/) (2024)
- [Cooperative Advisory Residual Policies for Congestion Mitigation](https://mlsanity.com/paper/2407.00553/) (2024)
- [Lessons in Cooperation: A Qualitative Analysis of Driver Sentiments towards Real-Time Advisory Systems from a Driving Simulator User Study](https://mlsanity.com/paper/2407.13775/) (2024)
- [Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks](https://mlsanity.com/paper/2412.12622/) (2024)
- [Optimizing Efficiency of Mixed Traffic through Reinforcement Learning: A Topology-Independent Approach and Benchmark](https://mlsanity.com/paper/2501.16728/) (2025)
- [Multi-Objective Reinforcement Learning for Large-Scale Mixed Traffic Control](https://mlsanity.com/paper/2512.11247/) (2025)

## Abstract (arXiv)

We propose a model-free reinforcement learning method for controlling mixed autonomy traffic in simulated traffic networks with through-traffic-only two-way and four-way intersections. Our method utilizes multi-agent policy decomposition which allows decentralized control based on local observations for an arbitrary number of controlled vehicles. We demonstrate that, even without reward shaping, reinforcement learning learns to coordinate the vehicles to exhibit traffic signal-like behaviors, achieving near-optimal throughput with 33-50% controlled vehicles. With the help of multi-task learning and transfer learning, we show that this behavior generalizes across inflow rates and size of the traffic network. Our code, models, and videos of results are available at https://github.com/ZhongxiaYan/mixed_autonomy_intersections.