# Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving

arXiv: https://arxiv.org/abs/1905.00229 · HTML: https://mlsanity.com/paper/1905.00229/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 4 of its 26 references that are in the map.
- First posted on arXiv 1 May 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 4 of 4)

- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [On a Formal Model of Safe and Scalable Self-driving Cars](https://mlsanity.com/paper/1708.06374/) (2017)

## Built on by (most cited first, 30 of 30)

- [Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving](https://mlsanity.com/paper/2103.10690/) (2021)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)
- [Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving](https://mlsanity.com/paper/2212.08787/) (2022)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Model](https://mlsanity.com/paper/2412.04209/) (2024)
- [Generative Planning for Temporally Coordinated Exploration in Reinforcement Learning](https://mlsanity.com/paper/2201.09765/) (2022)
- [Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles](https://mlsanity.com/paper/2212.03323/) (2022)
- [Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2306.16169/) (2023)
- [From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Driving](https://mlsanity.com/paper/2409.11694/) (2024)
- [Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.17373/) (2024)
- [Planning on the fast lane: Learning to interact using attention mechanisms in path integral inverse reinforcement learning](https://mlsanity.com/paper/2007.05798/) (2020)
- [Self-Supervised Simultaneous Multi-Step Prediction of Road Dynamics and Cost Map](https://mlsanity.com/paper/2103.01039/) (2021)
- [Reinforcement Learning based Negotiation-aware Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2107.03600/) (2021)
- [SafePath: Conformal Prediction for Safe LLM-Based Autonomous Navigation](https://mlsanity.com/paper/2505.09427/) (2025)
- [Inverse Reinforcement Learning Based Stochastic Driver Behavior Learning](https://mlsanity.com/paper/2107.06344/) (2021)
- [Automated Controller Calibration by Kalman Filtering](https://mlsanity.com/paper/2111.10832/) (2021)
- [PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learning](https://mlsanity.com/paper/2407.18569/) (2024)
- [Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Driving](https://mlsanity.com/paper/2505.05223/) (2025)
- [On the Choice of Data for Efficient Training and Validation of End-to-End Driving Models](https://mlsanity.com/paper/2206.00608/) (2022)
- [IR-STP: Enhancing Autonomous Driving with Interaction Reasoning in Spatio-Temporal Planning](https://mlsanity.com/paper/2311.02850/) (2023)
- [Accelerating the Evolution of Personalized Automated Lane Change through Lesson Learning](https://mlsanity.com/paper/2405.07543/) (2024)
- [Model Predictive Adversarial Imitation Learning for Planning from Observation](https://mlsanity.com/paper/2507.21533/) (2025)
- [Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Data](https://mlsanity.com/paper/2105.01820/) (2021)
- [Automatic Evaluation of Excavator Operators using Learned Reward Functions](https://mlsanity.com/paper/2211.07941/) (2022)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Pixel State Value Network for Combined Prediction and Planning in Interactive Environments](https://mlsanity.com/paper/2310.07706/) (2023)
- [Towards A General-Purpose Motion Planning for Autonomous Vehicles Using Fluid Dynamics](https://mlsanity.com/paper/2406.05708/) (2024)
- [Automatic Parameter Tuning of Self-Driving Vehicles](https://mlsanity.com/paper/2406.17757/) (2024)
- [Distributional Inverse Reinforcement Learning](https://mlsanity.com/paper/2510.03013/) (2025)
- [Q-based Variational Inverse Reinforcement Learning](https://mlsanity.com/paper/2608.16888/) (2026)

## Abstract (arXiv)

Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a complex feature space into rewards. Second, the reward function has to be manually tuned by an expert. Manually tuning this reward function becomes a tedious task. In this paper, we propose an approach that relies on human driving demonstrations to automatically tune reward functions. This study offers important insights into the driving style optimization of general-purpose planners with maximum entropy inverse reinforcement learning. We evaluate our approach based on the expected value difference between learned and demonstrated policies. Furthermore, we compare the similarity of human driven trajectories with optimal policies of our planner under learned and expert-tuned reward functions. Our experiments show that we are able to learn reward functions exceeding the level of manual expert tuning without prior domain knowledge.