# Driving in Real Life with Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2206.03004 · HTML: https://mlsanity.com/paper/2206.03004/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)
- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 24 of 24)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](https://mlsanity.com/paper/2101.06557/) (2021)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [Learning to drive from a world on rails](https://mlsanity.com/paper/2105.00636/) (2021)
- [SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies](https://mlsanity.com/paper/2109.13602/) (2021)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)

## Built on by (most cited first, 18 of 18)

- [Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives](https://mlsanity.com/paper/2303.09824/) (2023)
- [Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directions](https://mlsanity.com/paper/2307.07527/) (2023)
- [CCIL: Context-conditioned imitation learning for urban driving](https://mlsanity.com/paper/2305.02649/) (2023)
- [End-To-End Planning of Autonomous Driving in Industry and Academia: 2022-2023](https://mlsanity.com/paper/2401.08658/) (2023)
- [A Bayesian Approach to Robust Inverse Reinforcement Learning](https://mlsanity.com/paper/2309.08571/) (2023)
- [Differentiable Constrained Imitation Learning for Robot Motion Planning and Control](https://mlsanity.com/paper/2210.11796/) (2022)
- [Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles](https://mlsanity.com/paper/2212.03323/) (2022)
- [Blending Data-Driven Priors in Dynamic Games](https://mlsanity.com/paper/2402.14174/) (2024)
- [Autonomous Driving with Deep Reinforcement Learning in CARLA Simulation](https://mlsanity.com/paper/2306.11217/) (2023)
- [LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Models](https://mlsanity.com/paper/2501.05057/) (2025)
- [End-To-End Training and Testing Gamification Framework to Learn Human Highway Driving](https://mlsanity.com/paper/2404.10849/) (2024)
- [Automated Feature Selection for Inverse Reinforcement Learning](https://mlsanity.com/paper/2403.15079/) (2024)
- [Hybrid Imitation-Learning Motion Planner for Urban Driving](https://mlsanity.com/paper/2409.02871/) (2024)
- [Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Survey](https://mlsanity.com/paper/2502.10477/) (2025)
- [A Framework for Learning Scoring Rules in Autonomous Driving Planning Systems](https://mlsanity.com/paper/2502.11352/) (2025)
- [Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality](https://mlsanity.com/paper/2503.17865/) (2025)
- [NashFormer: Leveraging Local Nash Equilibria for Semantically Diverse Trajectory Prediction](https://mlsanity.com/paper/2305.17600/) (2023)
- [Q-based Variational Inverse Reinforcement Learning](https://mlsanity.com/paper/2608.16888/) (2026)

## Abstract (arXiv)

In this paper, we introduce the first learning-based planner to drive a car in dense, urban traffic using Inverse Reinforcement Learning (IRL). Our planner, DriveIRL, generates a diverse set of trajectory proposals, filters these trajectories with a lightweight and interpretable safety filter, and then uses a learned model to score each remaining trajectory. The best trajectory is then tracked by the low-level controller of our self-driving vehicle. We train our trajectory scoring model on a 500+ hour real-world dataset of expert driving demonstrations in Las Vegas within the maximum entropy IRL framework. DriveIRL's benefits include: a simple design due to only learning the trajectory scoring function, relatively interpretable features, and strong real-world performance. We validated DriveIRL on the Las Vegas Strip and demonstrated fully autonomous driving in heavy traffic, including scenarios involving cut-ins, abrupt braking by the lead vehicle, and hotel pickup/dropoff zones. Our dataset will be made public to help further research in this area.