mlsanityData to 7 Oct 2026

Inverse Reinforcement Learning With Constraint Recovery

arXiv 2305.08130 ↗First posted 14 May 2023cs.LG cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

In this work, we propose a novel inverse reinforcement learning (IRL) algorithm for constrained Markov decision process (CMDP) problems. In standard IRL problems, the inverse learner or agent seeks to recover the reward function of the MDP, given a set of trajectory demonstrations for the optimal policy. In this work, we seek to infer not only the reward functions of the CMDP, but also the constraints. Using the principle of maximum entropy, we show that the IRL with constraint recovery (IRL-CR) problem can be cast as a constrained non-convex optimization problem. We reduce it to an alternating constrained optimization problem whose sub-problems are convex. We use exponentiated gradient descent algorithm to solve it. Finally, we demonstrate the efficacy of our algorithm for the grid world environment.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 22 references that are in the map. First posted on arXiv 14 May 2023.

Builds on

9 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Learning Soft Constraints From Constrained Expert Demonstrations2 Jun
  2. 2022X-MEN: Guaranteed XOR-Maximum Entropy Constrained Inverse Reinforcement Learning22 Mar
  3. 2020Deep Inverse Q-learning with Constraints4 Aug
  4. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning12 Sep

Other references · 5

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2019Introduction to Online Convex Optimizationcited by 793
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  4. 2020Inverse Constrained Reinforcement Learningcited by 45
  5. 2020Uncertainty-Aware Constraint Learning for Adaptive Safe Motion Planning from Demonstrationscited by 10

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Generalized Exponentiated Gradient Algorithms and Their Application to On-Line Portfolio Selectioncited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025Inverse Reinforcement Learning via Convex Optimization
  2. 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
  3. 2023FP-IRL: Fokker--Planck Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processes
  4. 2021Task-Guided Inverse Reinforcement Learning Under Partial Information
  5. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
  6. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learning
  7. 2019On the Correctness and Sample Complexity of Inverse Reinforcement Learning
  8. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning