mlsanityData to 7 Oct 2026

Stochastic Inverse Reinforcement Learning

arXiv 1905.08513 ↗First posted 21 May 2019cs.LG cs.AI stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

The goal of the inverse reinforcement learning (IRL) problem is to recover the reward functions from expert demonstrations. However, the IRL problem like any ill-posed inverse problem suffers the congenital defect that the policy may be optimal for many reward functions, and expert demonstrations may be optimal for many policies. In this work, we generalize the IRL problem to a well-posed expectation optimization problem stochastic inverse reinforcement learning (SIRL) to recover the probability distribution over reward functions. We adopt the Monte Carlo expectation-maximization (MCEM) method to estimate the parameter of the probability distribution as the first solution to the SIRL problem. The solution is succinct, robust, and transferable for a learning task and can generate alternative solutions to the IRL problem. Through our formulation, it is possible to observe the intrinsic property of the IRL problem from a global viewpoint, and our approach achieves a considerable performance on the objectworld.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 35 references that are in the map. First posted on arXiv 21 May 2019.

Builds on

9 of 35 references.

References · 9

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  3. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  4. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  5. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  6. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  7. 2017Robust Imitation of Diverse Behaviorscited by 85
  8. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  9. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
  2. 2025Inverse Reinforcement Learning via Convex Optimization
  3. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  4. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
  5. 2025Apprenticeship learning with prior beliefs using inverse optimization
  6. 2024Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
  7. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
  8. 2023Towards Theoretical Understanding of Inverse Reinforcement Learning