mlsanityData to 7 Oct 2026

A Primer on Maximum Causal Entropy Inverse Reinforcement Learning

arXiv 2203.11409 ↗First posted 22 Mar 2022cs.LG cs.AIcited by 13

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse Reinforcement Learning (IRL) algorithms infer a reward function that explains demonstrations provided by an expert acting in the environment. Maximum Causal Entropy (MCE) IRL is currently the most popular formulation of IRL, with numerous extensions. In this tutorial, we present a compressed derivation of MCE IRL and the key results from contemporary implementations of MCE IRL algorithms. We hope this will serve both as an introductory resource for those new to the field, and as a concise reference for those already familiar with these topics.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 32 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

17 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Identifiability in inverse reinforcement learning7 Jun
  2. 2018Inverse reinforcement learning for video games24 Oct

Other references · 15

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2021Convex optimizationcited by 2,479
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  5. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  6. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  7. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  9. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  10. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
  11. 2020Quantifying Differences in Reward Functionscited by 62
  12. 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
3 more
  1. 2018A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learningcited by 35
  2. 2019Preferences Implicit in the State of the Worldcited by 33
  3. 2017Repeated Inverse Reinforcement Learningcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 5; 2026: 1.

13 papers.

Sort:

By area:

  1. 2022On the Sensitivity of Reward Inference to Misspecified Human Modelscited by 25
  2. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  3. 2023A Bayesian Approach to Robust Inverse Reinforcement Learningcited by 9
  4. 2023Maximum Causal Entropy Inverse Constrained Reinforcement Learningcited by 7
  5. 2024Measuring Goal-Directednesscited by 7
  6. 2024Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RLcited by 4
  7. 2025Kernel Based Maximum Entropy Inverse Reinforcement Learning for Mean-Field Gamescited by 3
  8. 2023Probabilistic inverse optimal control for non-linear partially observable systems disentangles perceptual uncertainty and behavioral costscited by 1
  9. 2025Efficient Reward Identification In Max Entropy Reinforcement Learning with Sparsity and Rank Priorscited by 1
  10. 2025Learning from Preferences and Mixed Demonstrations in General Settingscited by 1
  11. 2025Distributional Inverse Reinforcement Learningcited by 0
  12. 2025Incoherence in Goal-Conditioned Autoregressive Modelscited by 0
1 more
  1. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Rewardcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning
  2. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
  3. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
  4. 2021Inverse Reinforcement Learning with Explicit Policy Estimates
  5. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch
  6. 2022Weighted Maximum Entropy Inverse Reinforcement Learning
  7. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning
  8. 2022Task-Guided IRL in POMDPs that Scales