# Flow: A Modular Learning Framework for Mixed Autonomy Traffic

arXiv: https://arxiv.org/abs/1710.05465 · HTML: https://mlsanity.com/paper/1710.05465/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2017.

## Built on by (most cited first, 30 of 73)

- [The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models](https://mlsanity.com/paper/2201.03544/) (2022)
- [Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking](https://mlsanity.com/paper/2403.03185/) (2024)
- [Multi-Agent Coordination across Diverse Applications: A Survey](https://mlsanity.com/paper/2502.14743/) (2025)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [BAFFLE: Hiding Backdoors in Offline Reinforcement Learning Datasets](https://mlsanity.com/paper/2210.04688/) (2022)
- [CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic](https://mlsanity.com/paper/2410.14368/) (2024)
- [Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning](https://mlsanity.com/paper/2306.08094/) (2023)
- [Reward Reports for Reinforcement Learning](https://mlsanity.com/paper/2204.10817/) (2022)
- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset](https://mlsanity.com/paper/2404.02429/) (2024)
- [Unified Automatic Control of Vehicular Systems with Reinforcement Learning](https://mlsanity.com/paper/2208.00268/) (2022)
- [Learning to Control and Coordinate Mixed Traffic Through Robot Vehicles at Complex and Unsignalized Intersections](https://mlsanity.com/paper/2301.05294/) (2023)
- [Mixed Traffic Control and Coordination from Pixels](https://mlsanity.com/paper/2302.09167/) (2023)
- [Learning Eco-Driving Strategies at Signalized Intersections](https://mlsanity.com/paper/2204.12561/) (2022)
- [EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning](https://mlsanity.com/paper/2311.12261/) (2023)
- [CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control](https://mlsanity.com/paper/2404.00796/) (2024)
- [Traffic expertise meets residual RL: Knowledge-informed model-based residual reinforcement learning for CAV trajectory control](https://mlsanity.com/paper/2408.17380/) (2024)
- [IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning](https://mlsanity.com/paper/2410.15221/) (2024)
- [CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles using Deep Reinforcement Learning](https://mlsanity.com/paper/2201.13143/) (2022)
- [Reinforcement Learning in Practice: Opportunities and Challenges](https://mlsanity.com/paper/2202.11296/) (2022)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [Autonomous Driving Small-Scale Cars: A Survey of Recent Development](https://mlsanity.com/paper/2404.06229/) (2024)
- [Learning to Control DC Motor for Micromobility in Real Time with Reinforcement Learning](https://mlsanity.com/paper/2108.00138/) (2021)
- [Offline Reinforcement Learning for Road Traffic Control](https://mlsanity.com/paper/2201.02381/) (2022)
- [Generalizing Cooperative Eco-driving via Multi-residual Task Learning](https://mlsanity.com/paper/2403.04232/) (2024)
- [Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2507.14850/) (2025)
- [Learning a Robust Multiagent Driving Policy for Traffic Congestion Reduction](https://mlsanity.com/paper/2112.03759/) (2021)
- [Cooperation for Scalable Supervision of Autonomy in Mixed Traffic](https://mlsanity.com/paper/2112.07569/) (2021)
- [PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems](https://mlsanity.com/paper/2308.00864/) (2023)
- [Towards Co-operative Congestion Mitigation](https://mlsanity.com/paper/2302.09140/) (2023)

## Abstract (arXiv)

The rapid development of autonomous vehicles (AVs) holds vast potential for transportation systems through improved safety, efficiency, and access to mobility. However, the progression of these impacts, as AVs are adopted, is not well understood. Numerous technical challenges arise from the goal of analyzing the partial adoption of autonomy: partial control and observation, multi-vehicle interactions, and the sheer variety of scenarios represented by real-world networks. To shed light into near-term AV impacts, this article studies the suitability of deep reinforcement learning (RL) for overcoming these challenges in a low AV-adoption regime. A modular learning framework is presented, which leverages deep RL to address complex traffic dynamics. Modules are composed to capture common traffic phenomena (stop-and-go traffic jams, lane changing, intersections). Learned control laws are found to improve upon human driving performance, in terms of system-level velocity, by up to 57% with only 4-7% adoption of AVs. Furthermore, in single-lane traffic, a small neural network control law with only local observation is found to eliminate stop-and-go traffic - surpassing all known model-based controllers to achieve near-optimal performance - and generalize to out-of-distribution traffic densities.