# Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving

arXiv: https://arxiv.org/abs/2212.08787 · HTML: https://mlsanity.com/paper/2212.08787/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving.
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 17 Dec 2022.

## Placed in

- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)
- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2109.01827/) (2021)
- [Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients](https://mlsanity.com/paper/2109.13333/) (2021)
- [M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction](https://mlsanity.com/paper/2202.11884/) (2022)
- [PiP: Planning-informed Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2003.11476/) (2020)
- [Multi-modal Motion Prediction with Transformer-based Neural Network for Autonomous Driving](https://mlsanity.com/paper/2109.06446/) (2021)
- [Differentiable Integrated Motion Prediction and Planning with Learnable Cost Function for Autonomous Driving](https://mlsanity.com/paper/2207.10422/) (2022)
- [Jointly Learnable Behavior and Trajectory Planning for Self-Driving Vehicles](https://mlsanity.com/paper/1910.04586/) (2019)
- [Identifying Driver Interactions via Conditional Behavior Prediction](https://mlsanity.com/paper/2104.09959/) (2021)
- [Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving](https://mlsanity.com/paper/2103.10690/) (2021)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [Multi-modal Sensor Fusion-Based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding](https://mlsanity.com/paper/2005.09202/) (2020)
- [Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving](https://mlsanity.com/paper/1905.00229/) (2019)
- [Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving](https://mlsanity.com/paper/2102.09243/) (2021)
- [Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving](https://mlsanity.com/paper/2208.12263/) (2022)
- [Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving](https://mlsanity.com/paper/2109.12516/) (2021)
- [Narrowing the Coordinate-frame Gap in Behavior Prediction Models: Distillation for Efficient and Accurate Scene-centric Motion Forecasting](https://mlsanity.com/paper/2206.03970/) (2022)
- [ReCoAt: A Deep Learning-based Framework for Multi-Modal Motion Prediction in Autonomous Driving Application](https://mlsanity.com/paper/2207.00726/) (2022)
- [Interventional Behavior Prediction: Avoiding Overly Confident Anticipation in Interactive Prediction](https://mlsanity.com/paper/2204.08665/) (2022)
- [An Autonomous Driving Framework for Long-term Decision-making and Short-term Trajectory Planning on Frenet Space](https://mlsanity.com/paper/2011.13099/) (2020)

## Built on by (most cited first, 30 of 34)

- [GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving](https://mlsanity.com/paper/2303.05760/) (2023)
- [DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving](https://mlsanity.com/paper/2310.05885/) (2023)
- [Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuning](https://mlsanity.com/paper/2410.05582/) (2024)
- [HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving](https://mlsanity.com/paper/2401.03160/) (2024)
- [Hybrid-Prediction Integrated Planning for Autonomous Driving](https://mlsanity.com/paper/2402.02426/) (2024)
- [DistillDrive: End-to-End Multi-Mode Autonomous Driving Distillation by Isomorphic Hetero-Source Planning Model](https://mlsanity.com/paper/2508.05402/) (2025)
- [CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Model](https://mlsanity.com/paper/2412.04209/) (2024)
- [InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions](https://mlsanity.com/paper/2311.12943/) (2023)
- [End-To-End Planning of Autonomous Driving in Industry and Academia: 2022-2023](https://mlsanity.com/paper/2401.08658/) (2023)
- [Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/2401.11792/) (2024)
- [Predictive Planner for Autonomous Driving with Consistency Models](https://mlsanity.com/paper/2502.08033/) (2025)
- [GoIRL: Graph-Oriented Inverse Reinforcement Learning for Multimodal Trajectory Prediction](https://mlsanity.com/paper/2506.21121/) (2025)
- [Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Driving](https://mlsanity.com/paper/2302.03939/) (2023)
- [Pre-trained Transformer-Enabled Strategies with Human-Guided Fine-Tuning for End-to-end Navigation of Autonomous Vehicles](https://mlsanity.com/paper/2402.12666/) (2024)
- [Foresight in Motion: Reinforcing Trajectory Prediction with Reward Heuristics](https://mlsanity.com/paper/2507.12083/) (2025)
- [ManiCast: Collaborative Manipulation with Cost-Aware Human Forecasting](https://mlsanity.com/paper/2310.13258/) (2023)
- [Learning Online Belief Prediction for Efficient POMDP Planning in Autonomous Driving](https://mlsanity.com/paper/2401.15315/) (2024)
- [A Reinforcement Learning-Boosted Motion Planning Framework: Comprehensive Generalization Performance in Autonomous Driving](https://mlsanity.com/paper/2402.01465/) (2024)
- [RESET: Revisiting Trajectory Sets for Conditional Behavior Prediction](https://mlsanity.com/paper/2304.05856/) (2023)
- [PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learning](https://mlsanity.com/paper/2407.18569/) (2024)
- [IR-STP: Enhancing Autonomous Driving with Interaction Reasoning in Spatio-Temporal Planning](https://mlsanity.com/paper/2311.02850/) (2023)
- [Inverse Reinforcement Learning for Minimum-Exposure Paths in Spatiotemporally Varying Scalar Fields](https://mlsanity.com/paper/2503.06611/) (2025)
- [Offline Reinforcement Learning using Human-Aligned Reward Labeling for Autonomous Emergency Braking in Occluded Pedestrian Crossing](https://mlsanity.com/paper/2504.08704/) (2025)
- [Evolving Testing Scenario Generation Method and Intelligence Evaluation Framework for Automated Vehicles](https://mlsanity.com/paper/2306.07142/) (2023)
- [Measuring Sociality in Driving Interaction](https://mlsanity.com/paper/2306.13992/) (2023)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Driving](https://mlsanity.com/paper/2506.03568/) (2025)
- [Trajectory Entropy: Modeling Game State Stability from Multimodality Trajectory Prediction](https://mlsanity.com/paper/2506.05810/) (2025)
- [Orchestrate, Generate, Reflect: A VLM-Based Multi-Agent Collaboration Framework for Automated Driving Policy Learning](https://mlsanity.com/paper/2509.17042/) (2025)
- [Self-Interest and Systemic Benefits: Emergence of Collective Rationality in Mixed Autonomy Traffic Through Deep Reinforcement Learning](https://mlsanity.com/paper/2511.04883/) (2025)

## Abstract (arXiv)

Making safe and human-like decisions is an essential capability of autonomous driving systems, and learning-based behavior planning presents a promising pathway toward achieving this objective. Distinguished from existing learning-based methods that directly output decisions, this work introduces a predictive behavior planning framework that learns to predict and evaluate from human driving data. This framework consists of three components: a behavior generation module that produces a diverse set of candidate behaviors in the form of trajectory proposals, a conditional motion prediction network that predicts future trajectories of other agents based on each proposal, and a scoring module that evaluates the candidate plans using maximum entropy inverse reinforcement learning (IRL). We validate the proposed framework on a large-scale real-world urban driving dataset through comprehensive experiments. The results show that the conditional prediction model can predict distinct and reasonable future trajectories given different trajectory proposals and the IRL-based scoring module can select plans that are close to human driving. The proposed framework outperforms other baseline methods in terms of similarity to human driving trajectories. Additionally, we find that the conditional prediction model improves both prediction and planning performance compared to the non-conditional model. Lastly, we note that learning the scoring module is crucial for aligning the evaluations with human drivers.