# Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios

arXiv: https://arxiv.org/abs/2107.04538 · HTML: https://mlsanity.com/paper/2107.04538/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 67 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations](https://mlsanity.com/paper/2008.05930/) (2020)
- [Imitating Driver Behavior with Generative Adversarial Networks](https://mlsanity.com/paper/1701.06699/) (2017)
- [Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction](https://mlsanity.com/paper/1710.09483/) (2017)
- [Hierarchical Game-Theoretic Planning for Autonomous Vehicles](https://mlsanity.com/paper/1810.05766/) (2018)
- [A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers](https://mlsanity.com/paper/1804.07871/) (2018)
- [Driving in Dense Traffic with Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.06710/) (2019)
- [Deep Learning-based Vehicle Behaviour Prediction For Autonomous Driving Applications: A Review](https://mlsanity.com/paper/1912.11676/) (2019)
- [MATS: An Interpretable Trajectory Forecasting Representation for Planning and Control](https://mlsanity.com/paper/2009.07517/) (2020)
- [Reinforcement Learning with Probabilistic Guarantees for Autonomous Driving](https://mlsanity.com/paper/1904.07189/) (2019)
- [Model Predictive Contouring Control for Collision Avoidance in Unstructured Dynamic Environments](https://mlsanity.com/paper/2010.10190/) (2020)
- [Cooperation-Aware Reinforcement Learning for Merging in Dense Traffic](https://mlsanity.com/paper/1906.11021/) (2019)
- [Safe Multi-Agent Interaction through Robust Control Barrier Functions with Learned Uncertainties](https://mlsanity.com/paper/2004.05273/) (2020)
- [LUCIDGames: Online Unscented Inverse Dynamic Games for Adaptive Trajectory Prediction and Planning](https://mlsanity.com/paper/2011.08152/) (2020)
- [Cooperation-Aware Lane Change Maneuver in Dense Traffic based on Model Predictive Control with Recurrent Neural Network](https://mlsanity.com/paper/1909.05665/) (2019)
- [Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning](https://mlsanity.com/paper/1810.10469/) (2018)
- [Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic](https://mlsanity.com/paper/2005.11895/) (2020)
- [Learning When to Drive in Intersections by Combining Reinforcement Learning and Model Predictive Control](https://mlsanity.com/paper/1908.00177/) (2019)
- [Social-VRNN: One-Shot Multi-modal Trajectory Prediction for Interacting Pedestrians](https://mlsanity.com/paper/2010.09056/) (2020)
- [I-Planner: Intention-Aware Motion Planning Using Learning Based Human Motion Prediction](https://mlsanity.com/paper/1608.04837/) (2016)
- [Sparse Interacting Gaussian Processes: Efficiency and Optimality Theorems of Autonomous Crowd Navigation](https://mlsanity.com/paper/1705.03639/) (2017)

## Built on by (most cited first, 7 of 7)

- [Testing of Deep Reinforcement Learning Agents with Surrogate Models](https://mlsanity.com/paper/2305.12751/) (2023)
- [Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills](https://mlsanity.com/paper/2209.12072/) (2022)
- [Interaction-Aware Trajectory Planning for Autonomous Vehicles with Analytic Integration of Neural Networks into Model Predictive Control](https://mlsanity.com/paper/2301.05393/) (2023)
- [Learning-Aided Warmstart of Model Predictive Control in Uncertain Fast-Changing Traffic](https://mlsanity.com/paper/2310.02918/) (2023)
- [Generative AI for Autonomous Driving: A Review](https://mlsanity.com/paper/2505.15863/) (2025)
- [Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2406.13223/) (2024)
- [Momentum-constrained Hybrid Heuristic Trajectory Optimization Framework with Residual-enhanced DRL for Visually Impaired Scenarios](https://mlsanity.com/paper/2509.15582/) (2025)

## Abstract (arXiv)

Autonomous navigation in dense traffic scenarios remains challenging for autonomous vehicles (AVs) because the intentions of other drivers are not directly observable and AVs have to deal with a wide range of driving behaviors. To maneuver through dense traffic, AVs must be able to reason how their actions affect others (interaction model) and exploit this reasoning to navigate through dense traffic safely. This paper presents a novel framework for interaction-aware motion planning in dense traffic scenarios. We explore the connection between human driving behavior and their velocity changes when interacting. Hence, we propose to learn, via deep Reinforcement Learning (RL), an interaction-aware policy providing global guidance about the cooperativeness of other vehicles to an optimization-based planner ensuring safety and kinematic feasibility through constraint satisfaction. The learned policy can reason and guide the local optimization-based planner with interactive behavior to pro-actively merge in dense traffic while remaining safe in case the other vehicles do not yield. We present qualitative and quantitative results in highly interactive simulation environments (highway merging and unprotected left turns) against two baseline approaches, a learning-based and an optimization-based method. The presented results demonstrate that our method significantly reduces the number of collisions and increases the success rate with respect to both learning-based and optimization-based baselines.