# Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic

arXiv: https://arxiv.org/abs/2005.11895 · HTML: https://mlsanity.com/paper/2005.11895/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2020.

## Built on by (most cited first, 10 of 10)

- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks](https://mlsanity.com/paper/2108.05030/) (2021)
- [Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2201.06539/) (2022)
- [Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios](https://mlsanity.com/paper/2107.04538/) (2021)
- [Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Data](https://mlsanity.com/paper/2103.04289/) (2021)
- [Informed Reinforcement Learning for Situation-Aware Traffic Rule Exceptions](https://mlsanity.com/paper/2402.04168/) (2024)
- [Deep Binary Reinforcement Learning for Scalable Verification](https://mlsanity.com/paper/2203.05704/) (2022)
- [K-Level Policy Gradients for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2509.12117/) (2025)
- [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023)
- [Intrinsic Vicarious Conditioning for Deep Reinforcement Learning](https://mlsanity.com/paper/2605.12224/) (2026)

## Abstract (arXiv)

Maneuvering in dense traffic is a challenging task for autonomous vehicles because it requires reasoning about the stochastic behaviors of many other participants. In addition, the agent must achieve the maneuver within a limited time and distance. In this work, we propose a combination of reinforcement learning and game theory to learn merging behaviors. We design a training curriculum for a reinforcement learning agent using the concept of level-k behavior. This approach exposes the agent to a broad variety of behaviors during training, which promotes learning policies that are robust to model discrepancies. We show that our approach learns more efficient policies than traditional training methods.