mlsanityData to 14 Oct 2026

Generalized Maximum Causal Entropy for Inverse Reinforcement Learning

arXiv 1911.06928 ↗First posted 16 Nov 2019cs.LG stat.MLcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

We consider the problem of learning from demonstrated trajectories with inverse reinforcement learning (IRL). Motivated by a limitation of the classical maximum entropy model in capturing the structure of the network of states, we propose an IRL model based on a generalized version of the causal entropy maximization problem, which allows us to generate a class of maximum entropy IRL models. Our generalized model has an advantage of being able to recover, in addition to a reward function, another expert's function that would (partially) capture the impact of the connecting structure of the states on experts' decisions. Empirical evaluation on a real-world dataset and a grid-world dataset shows that our generalized model outperforms the classical ones, in terms of recovering reward functions and demonstrated trajectories.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 5 of its 23 references that are in the map. First posted on arXiv 16 Nov 2019.

Builds on

5 of 23 references.

References · 5

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  4. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  5. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 1.

2 papers.

  1. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
  2. 2021An Adaptable Approach to Learn Realistic Legged Locomotion without Examplescited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
  2. 2021Inverse Reinforcement Learning with Explicit Policy Estimates
  3. 2022Weighted Maximum Entropy Inverse Reinforcement Learning
  4. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learning
  5. 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
  6. 2026Fast Rates for Inverse Reinforcement Learning
  7. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning
  8. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch