# Cooperation-Aware Reinforcement Learning for Merging in Dense Traffic

arXiv: https://arxiv.org/abs/1906.11021 · HTML: https://mlsanity.com/paper/1906.11021/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 20 of 20)

- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Deep Structured Reactive Planning](https://mlsanity.com/paper/2101.06832/) (2021)
- [TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning](https://mlsanity.com/paper/2502.01387/) (2025)
- [Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios](https://mlsanity.com/paper/2107.04538/) (2021)
- [A Hybrid Rule-Based and Data-Driven Approach to Driver Modeling through Particle Filtering](https://mlsanity.com/paper/2108.12820/) (2021)
- [Prediction Based Decision Making for Autonomous Highway Driving](https://mlsanity.com/paper/2209.02106/) (2022)
- [Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation](https://mlsanity.com/paper/2210.03772/) (2022)
- [Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guarantees](https://mlsanity.com/paper/2508.05441/) (2025)
- [Intent-Aware Autonomous Driving: A Case Study on Highway Merging Scenarios](https://mlsanity.com/paper/2309.13206/) (2023)
- [High-level Decisions from a Safe Maneuver Catalog with Reinforcement Learning for Safe and Cooperative Automated Merging](https://mlsanity.com/paper/2107.07413/) (2021)
- [Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments](https://mlsanity.com/paper/2201.12776/) (2022)
- [Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed traffic](https://mlsanity.com/paper/2211.11963/) (2022)
- [Importance Filtering with Risk Models for Complex Driving Situations](https://mlsanity.com/paper/2303.06935/) (2023)
- [Bayesian inference for data-efficient, explainable, and safe robotic motion planning: A review](https://mlsanity.com/paper/2307.08024/) (2023)
- [Evaluating Scenario-based Decision-making for Interactive Autonomous Driving Using Rational Criteria: A Survey](https://mlsanity.com/paper/2501.01886/) (2025)
- [Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review](https://mlsanity.com/paper/2505.07911/) (2025)
- [Decision Making for Autonomous Driving in Interactive Merge Scenarios via Learning-based Prediction](https://mlsanity.com/paper/2303.16821/) (2023)
- [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023)
- [Bench4Merge: A Comprehensive Benchmark for Merging in Realistic Dense Traffic with Micro-Interactive Vehicles](https://mlsanity.com/paper/2410.15912/) (2024)
- [Assistive Decision-Making for Right of Way Navigation at Uncontrolled Intersections](https://mlsanity.com/paper/2509.18407/) (2025)

## Abstract (arXiv)

Decision making in dense traffic can be challenging for autonomous vehicles. An autonomous system only relying on predefined road priorities and considering other drivers as moving objects will cause the vehicle to freeze and fail the maneuver. Human drivers leverage the cooperation of other drivers to avoid such deadlock situations and convince others to change their behavior. Decision making algorithms must reason about the interaction with other drivers and anticipate a broad range of driver behaviors. In this work, we present a reinforcement learning approach to learn how to interact with drivers with different cooperation levels. We enhanced the performance of traditional reinforcement learning algorithms by maintaining a belief over the level of cooperation of other drivers. We show that our agent successfully learns how to navigate a dense merging scenario with less deadlocks than with online planning methods.