# A deep inverse reinforcement learning approach to route choice modeling with context-dependent rewards

arXiv: https://arxiv.org/abs/2206.10598 · HTML: https://mlsanity.com/paper/2206.10598/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [Deep Gravity: enhancing mobility flows generation with deep neural networks and geographic information](https://mlsanity.com/paper/2012.00489/) (2020)
- [TrajGAIL: Generating Urban Vehicle Trajectories using Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2007.14189/) (2020)
- [NetTraj: A Network-based Vehicle Trajectory Prediction Model with Directional Representation and Spatiotemporal Attention Mechanisms](https://mlsanity.com/paper/2106.11175/) (2021)
- [A tutorial on recursive models for analyzing and predicting path choice behavior](https://mlsanity.com/paper/1905.00883/) (2019)

## Built on by (most cited first, 10 of 10)

- [AI-Driven Day-to-Day Route Choice](https://mlsanity.com/paper/2412.03338/) (2024)
- [Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2501.06946/) (2025)
- [Aligning LLM agents with human learning and adjustment behavior: a dual agent approach](https://mlsanity.com/paper/2511.00993/) (2025)
- [Incorporating graph neural network into route choice model](https://mlsanity.com/paper/2503.02315/) (2025)
- [Learning the Value Systems of Agents with Preference-based and Inverse Reinforcement Learning](https://mlsanity.com/paper/2602.04518/) (2026)
- [Global path preference and local response: A reward decomposition approach for network path choice analysis in the presence of locally perceived attributes](https://mlsanity.com/paper/2307.08646/) (2023)
- [Analyzing sequential activity and travel decisions with interpretable deep inverse reinforcement learning](https://mlsanity.com/paper/2503.12761/) (2025)
- [Where You Go is Who You Are: Behavioral Theory-Guided LLMs for Inverse Reinforcement Learning](https://mlsanity.com/paper/2505.17249/) (2025)
- [Group Effect Enhanced Generative Adversarial Imitation Learning for Individual Travel Behavior Modeling under Incentives](https://mlsanity.com/paper/2509.06656/) (2025)
- [Contextual Preference Distribution Learning](https://mlsanity.com/paper/2603.17139/) (2026)

## Abstract (arXiv)

Route choice modeling is a fundamental task in transportation planning and demand forecasting. Classical methods generally adopt the discrete choice model (DCM) framework with linear utility functions and high-level route characteristics. While several recent studies have started to explore the applicability of deep learning for route choice modeling, they are limited to path-based models with relatively simple model architectures and relying on predefined choice sets. Existing link-based models can capture the dynamic nature of link choices within the trip without the need for choice set generation, but still assume linear relationships and link-additive features. To address these issues, this study proposes a general deep inverse reinforcement learning (IRL) framework for link-based route choice modeling, which is capable of incorporating diverse features (of the state, action and trip context) and capturing complex relationships. Specifically, we adapt an adversarial IRL model to the route choice problem for efficient estimation of context-dependent reward functions without value iteration. Experiment results based on taxi GPS data from Shanghai, China validate the superior prediction performance of the proposed model over conventional DCMs and other imitation learning baselines, even for destinations unseen in the training data. Further analysis show that the model exhibits competitive computational efficiency and reasonable interpretability. The proposed methodology provides a new direction for future development of route choice models. It is general and can be adaptable to other route choice problems across different modes and networks.