mlsanityData to 7 Oct 2026

Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving

arXiv 2006.13704 ↗First posted 22 Jun 2020cs.RO cs.AI cs.LGcited by 34

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

In the past decades, we have witnessed significant progress in the domain of autonomous driving. Advanced techniques based on optimization and reinforcement learning (RL) become increasingly powerful at solving the forward problem: given designed reward/cost functions, how should we optimize them and obtain driving policies that interact with the environment safely and efficiently. Such progress has raised another equally important question: what should we optimize? Instead of manually specifying the reward functions, it is desired that we can extract what human drivers try to optimize from real traffic data and assign that to autonomous vehicles to enable more naturalistic and transparent interaction between humans and intelligent agents. To address this issue, we present an efficient sampling-based maximum-entropy inverse reinforcement learning (IRL) algorithm in this paper. Different from existing IRL algorithms, by introducing an efficient continuous-domain trajectory sampler, the proposed algorithm can directly learn the reward functions in the continuous domain while considering the uncertainties in demonstrated trajectories from human drivers. We evaluate the proposed algorithm on real driving data, including both non-interactive and interactive scenarios. The experimental results show that the proposed algorithm achieves more accurate prediction performance with faster convergence speed and better generalization compared to other baseline IRL algorithms.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 27 references that are in the map. First posted on arXiv 22 Jun 2020.

Builds on

6 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning9 Sep

Other references · 5

  1. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  2. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  3. 2014Online Planning Algorithms for POMDPscited by 42
  4. 2020LESS is More: Rethinking Probabilistic Models of Human Behaviorcited by 17
  5. 2018Courteous Autonomous Carscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 5; 2022: 1; 2023: 9; 2024: 6; 2025: 9; 2026: 2.

34 papers.

Sort:

By area:

  1. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  2. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  3. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  4. 2023Machine Learning for Autonomous Vehicle's Trajectory Prediction: A comprehensive survey, Challenges, and Future Research Directionscited by 23
  5. 2023Towards Theoretical Understanding of Inverse Reinforcement Learningcited by 21
  6. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 17
  7. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  8. 2020Regularized Inverse Reinforcement Learningcited by 9
  9. 2023Offline Imitation Learning with Variational Counterfactual Reasoningcited by 7
  10. 2021Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Datacited by 6
  11. 2025Foresight in Motion: Reinforcing Trajectory Prediction with Reward Heuristicscited by 6
  12. 2024Blending Data-Driven Priors in Dynamic Gamescited by 5
22 more
  1. 2021Inverse Reinforcement Learning Based Stochastic Driver Behavior Learningcited by 3
  2. 2022Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Searchcited by 3
  3. 2024PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learningcited by 3
  4. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  5. 2021How To Not Drive: Learning Driving Constraints from Demonstrationcited by 2
  6. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  7. 2021Calibration of Human Driving Behavior and Preference Using Naturalistic Traffic Datacited by 1
  8. 2023Active Inverse Learning in Stackelberg Trajectory Gamescited by 1
  9. 2024Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulationscited by 1
  10. 2025Offline Reinforcement Learning using Human-Aligned Reward Labeling for Autonomous Emergency Braking in Occluded Pedestrian Crossingcited by 1
  11. 2025Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithmcited by 1
  12. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  13. 2023Human-Like Implicit Intention Expression for Autonomous Driving Motion Planning: A Method Based on Learning Human Intention Priorscited by 0
  14. 2023Identifying Reaction-Aware Driving Styles of Stochastic Model Predictive Controlled Vehicles by Inverse Reinforcement Learningcited by 0
  15. 2023ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Eventscited by 0
  16. 2024Reinforced Imitative Trajectory Planning for Urban Automated Drivingcited by 0
  17. 2025A physics-informed Bayesian optimization method for rapid development of electrical machinescited by 0
  18. 2025Recursive Deep Inverse Reinforcement Learningcited by 0
  19. 2025Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learningcited by 0
  20. 2025Distributional Inverse Reinforcement Learningcited by 0
  21. 2026Online Intention Prediction via Control-Informed Learningcited by 0
  22. 2026Loop-Free Inverse Reinforcement Learning via Sequential Value Recovery with Q-Score Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving
  2. 2022Driving in Real Life with Inverse Reinforcement Learning
  3. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving
  4. 2026Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study
  5. 2024Adaptive Decision-Making for Autonomous Vehicles: A Learning-Enhanced Game-Theoretic Approach in Interactive Environments
  6. 2020Expressing Diverse Human Driving Behavior with Probabilistic Rewards and Online Inference
  7. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  8. 2025V-Max: A Reinforcement Learning Framework for Autonomous Driving