mlsanityData to 7 Oct 2026

Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning

arXiv 2201.06539 ↗First posted 17 Jan 2022cs.RO cs.AIcited by 10

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatiotemporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 38 references that are in the map. First posted on arXiv 17 Jan 2022.

Builds on

23 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning17 Apr
  2. 2018Integrating kinematics and environment context into deep inverse reinforcement learning for predicting off-road vehicle trajectories16 Oct

Other references · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  10. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  11. 2019Planning with Goal-Conditioned Policiescited by 144
  12. 2019Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Trafficcited by 50
9 more
  1. 2019Safe Reinforcement Learning on Autonomous Vehiclescited by 48
  2. 2019Trust Region-Guided Proximal Policy Optimizationcited by 36
  3. 2020Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationshipscited by 36
  4. 2016Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environmentscited by 33
  5. 2019Driving in Dense Traffic with Model-Free Reinforcement Learningcited by 33
  6. 2020Waymo Public Road Safety Performance Datacited by 24
  7. 2019Cooperation-Aware Lane Change Maneuver in Dense Traffic based on Model Predictive Control with Recurrent Neural Networkcited by 14
  8. 2019Interactive Decision Making for Autonomous Vehicles in Dense Trafficcited by 11
  9. 2020Reinforcement Learning with Iterative Reasoning for Merging in Dense Trafficcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 3.

10 papers.

  1. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  2. 2023Learning Risk-Aware Costmaps via Inverse Reinforcement Learning for Off-Road Navigationcited by 15
  3. 2022Receding Horizon Inverse Reinforcement Learningcited by 10
  4. 2024Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Surveycited by 9
  5. 2025Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learningcited by 3
  6. 2024ARMCHAIR: integrated inverse reinforcement learning and model predictive control for human-robot collaborationcited by 2
  7. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2
  8. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  9. 2023On the Effective Horizon of Inverse Reinforcement Learningcited by 0
  10. 2023Pixel State Value Network for Combined Prediction and Planning in Interactive Environmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Planning on the fast lane: Learning to interact using attention mechanisms in path integral inverse reinforcement learning
  2. 2022Driving in Real Life with Inverse Reinforcement Learning
  3. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving
  4. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving
  5. 2025Generalizable Trajectory Prediction via Inverse Reinforcement Learning with Mamba-Graph Architecture
  6. 2021Inverse Reinforcement Learning Based Stochastic Driver Behavior Learning
  7. 2026Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study
  8. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving