# CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control

arXiv: https://arxiv.org/abs/2404.00796 · HTML: https://mlsanity.com/paper/2404.00796/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Imitating Driver Behavior with Generative Adversarial Networks](https://mlsanity.com/paper/1701.06699/) (2017)
- [Urban Driving with Conditional Imitation Learning](https://mlsanity.com/paper/1912.00177/) (2019)
- [Flow: A Modular Learning Framework for Mixed Autonomy Traffic](https://mlsanity.com/paper/1710.05465/) (2017)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Multi-Agent Imitation Learning for Driving Simulation](https://mlsanity.com/paper/1803.01044/) (2018)
- [Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.00569/) (2019)
- [Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments](https://mlsanity.com/paper/1705.01693/) (2017)
- [Modeling Human Driving Behavior through Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2006.06412/) (2020)
- [I-24 MOTION: An instrument for freeway traffic science](https://mlsanity.com/paper/2301.11198/) (2023)
- [A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning](https://mlsanity.com/paper/2007.05156/) (2020)
- [Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learning](https://mlsanity.com/paper/2306.08094/) (2023)
- [Limitations and Improvements of the Intelligent Driver Model (IDM)](https://mlsanity.com/paper/2104.02583/) (2021)
- [ADAPS: Autonomous Driving Via Principled Simulations](https://mlsanity.com/paper/1907.08874/) (2019)
- [Learning to Control and Coordinate Mixed Traffic Through Robot Vehicles at Complex and Unsignalized Intersections](https://mlsanity.com/paper/2301.05294/) (2023)
- [Large-scale Mixed Traffic Control Using Dynamic Vehicle Routing and Privacy-Preserving Crowdsourcing](https://mlsanity.com/paper/2311.11347/) (2023)
- [Modeling car-following behavior on urban expressways in Shanghai: A naturalistic driving study](https://mlsanity.com/paper/1811.06395/) (2018)
- [Mixed Traffic Control and Coordination from Pixels](https://mlsanity.com/paper/2302.09167/) (2023)
- [EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning](https://mlsanity.com/paper/2311.12261/) (2023)
- [Predicting Station-Level Bike-Sharing Demands Using Graph Convolutional Neural Network](https://mlsanity.com/paper/2004.08723/) (2020)
- [Car-Following Models: A Multidisciplinary Review](https://mlsanity.com/paper/2304.07143/) (2023)
- [LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulation](https://mlsanity.com/paper/2403.17601/) (2024)
- [Black-box Adversarial Attacks on Network-wide Multi-step Traffic State Prediction Models](https://mlsanity.com/paper/2110.08712/) (2021)
- [Learning to Control DC Motor for Micromobility in Real Time with Reinforcement Learning](https://mlsanity.com/paper/2108.00138/) (2021)
- [Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model](https://mlsanity.com/paper/2203.04749/) (2022)
- [AutoJoin: Efficient Adversarial Training against Gradient-Free Perturbations for Robust Maneuvering via Denoising Autoencoder and Joint Learning](https://mlsanity.com/paper/2205.10933/) (2022)
- [Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2403.14879/) (2024)

## Built on by (most cited first, 7 of 7)

- [Mixture-of-Personas Language Models for Population Simulation](https://mlsanity.com/paper/2504.05019/) (2025)
- [Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2504.04691/) (2025)
- [Exploring Transformer-Augmented LSTM for Temporal and Spatial Feature Learning in Trajectory Prediction](https://mlsanity.com/paper/2412.13419/) (2024)
- [Neighbor-Aware Reinforcement Learning for Mixed Traffic Optimization in Large-scale Networks](https://mlsanity.com/paper/2412.12622/) (2024)
- [LDGNet: A Lightweight Difference Guiding Network for Remote Sensing Change Detection](https://mlsanity.com/paper/2504.05062/) (2025)
- [Multi-Objective Reinforcement Learning for Large-Scale Mixed Traffic Control](https://mlsanity.com/paper/2512.11247/) (2025)
- [Learning to Design City-scale Transit Routes](https://mlsanity.com/paper/2512.19767/) (2025)

## Abstract (arXiv)

Human-driven vehicles (HVs) exhibit complex and diverse behaviors. Accurately modeling such behavior is crucial for validating Robot Vehicles (RVs) in simulation and realizing the potential of mixed traffic control. However, existing approaches like parameterized models and data-driven techniques struggle to capture the full complexity and diversity. To address this, in this work, we introduce CARL, a hybrid approach that combines imitation learning for close proximity car-following and probabilistic sampling for larger headways. We also propose two classes of RL-based RVs: a safety RV focused on maximizing safety and an efficiency RV focused on maximizing efficiency. Our experiments show that the safety RV increases Time-to-Collision above the critical 4-second threshold and reduces Deceleration Rate to Avoid a Crash by up to 80%, while the efficiency RV achieves improvements in throughput of up to 49%. These results demonstrate the effectiveness of CARL in enhancing both safety and efficiency in mixed traffic.