# Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/1809.02926 · HTML: https://mlsanity.com/paper/1809.02926/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 27 references that are in the map.
- First posted on arXiv 9 Sep 2018.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 2 of 2)

- [Probabilistic Prediction of Vehicle Semantic Intention and Motion](https://mlsanity.com/paper/1804.03629/) (2018)
- [Courteous Autonomous Cars](https://mlsanity.com/paper/1808.02633/) (2018)

## Built on by (most cited first, 28 of 28)

- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)
- [Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking](https://mlsanity.com/paper/2102.09117/) (2021)
- [Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving](https://mlsanity.com/paper/2006.13704/) (2020)
- [The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models](https://mlsanity.com/paper/2204.10759/) (2022)
- [Integrated Decision and Control: Towards Interpretable and Computationally Efficient Driving Intelligence](https://mlsanity.com/paper/2103.10290/) (2021)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [Transferable and Adaptable Driving Behavior Prediction](https://mlsanity.com/paper/2202.05140/) (2022)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Driver Digital Twin for Online Prediction of Personalized Lane Change Behavior](https://mlsanity.com/paper/2211.01294/) (2022)
- [On complementing end-to-end human behavior predictors with planning](https://mlsanity.com/paper/2103.05661/) (2021)
- [Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2203.04421/) (2022)
- [Grounded Relational Inference: Domain Knowledge Driven Explainable Autonomous Driving](https://mlsanity.com/paper/2102.11905/) (2021)
- [Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Data](https://mlsanity.com/paper/2103.04289/) (2021)
- [Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations](https://mlsanity.com/paper/2503.20105/) (2025)
- [End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering](https://mlsanity.com/paper/2103.00345/) (2021)
- [Approximating Discontinuous Nash Equilibrial Values of Two-Player General-Sum Differential Games](https://mlsanity.com/paper/2207.01773/) (2022)
- [Value Approximation for Two-Player General-Sum Differential Games with State Constraints](https://mlsanity.com/paper/2311.16520/) (2023)
- [OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences](https://mlsanity.com/paper/2402.04567/) (2024)
- [Preemptive Motion Planning for Human-to-Robot Indirect Placement Handovers](https://mlsanity.com/paper/2203.00156/) (2022)
- [Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Data](https://mlsanity.com/paper/2105.01820/) (2021)
- [Adaptive Decision-Making for Autonomous Vehicles: A Learning-Enhanced Game-Theoretic Approach in Interactive Environments](https://mlsanity.com/paper/2402.11467/) (2024)
- [Modeling Human Driver Interactions Using an Infinite Policy Space Through Gaussian Processes](https://mlsanity.com/paper/2201.01733/) (2022)
- [Identifying Reaction-Aware Driving Styles of Stochastic Model Predictive Controlled Vehicles by Inverse Reinforcement Learning](https://mlsanity.com/paper/2308.12069/) (2023)
- [Generalizable Trajectory Prediction via Inverse Reinforcement Learning with Mamba-Graph Architecture](https://mlsanity.com/paper/2506.12474/) (2025)
- [ILNet: Trajectory Prediction with Inverse Learning Attention for Enhancing Intention Capture](https://mlsanity.com/paper/2507.06531/) (2025)
- [Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study](https://mlsanity.com/paper/2608.06445/) (2026)
- [Q-based Variational Inverse Reinforcement Learning](https://mlsanity.com/paper/2608.16888/) (2026)

## Abstract (arXiv)

Autonomous vehicles (AVs) are on the road. To safely and efficiently interact with other road participants, AVs have to accurately predict the behavior of surrounding vehicles and plan accordingly. Such prediction should be probabilistic, to address the uncertainties in human behavior. Such prediction should also be interactive, since the distribution over all possible trajectories of the predicted vehicle depends not only on historical information, but also on future plans of other vehicles that interact with it. To achieve such interaction-aware predictions, we propose a probabilistic prediction approach based on hierarchical inverse reinforcement learning (IRL). First, we explicitly consider the hierarchical trajectory-generation process of human drivers involving both discrete and continuous driving decisions. Based on this, the distribution over all future trajectories of the predicted vehicle is formulated as a mixture of distributions partitioned by the discrete decisions. Then we apply IRL hierarchically to learn the distributions from real human demonstrations. A case study for the ramp-merging driving scenario is provided. The quantitative results show that the proposed approach can accurately predict both the discrete driving decisions such as yield or pass as well as the continuous trajectories.