mlsanityData to 14 Oct 2026

Inverse Reinforcement Learning via Convex Optimization

arXiv 2501.15957 ↗First posted 27 Jan 2025cs.LG cs.CE math.OCcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

We consider the inverse reinforcement learning (IRL) problem, where an unknown reward function of some Markov decision process is estimated based on observed expert demonstrations. In most existing approaches, IRL is formulated and solved as a nonconvex optimization problem, posing challenges in scenarios where robustness and reproducibility are critical. We discuss a convex formulation of the IRL problem (CIRL) initially proposed by Ng and Russel, and reformulate the problem such that the domain-specific language CVXPY can be applied directly to specify and solve the convex problem. We also extend the CIRL problem to scenarios where the expert policy is not given analytically but by trajectory as state-action pairs, which can be strongly inconsistent with optimality, by augmenting some of the constraints. Theoretical analysis and practical implementation for hyperparameter auto-selection are introduced. This note helps the users to easily apply CIRL for their problems, without background knowledge on convex optimization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 5 of its 16 references that are in the map. First posted on arXiv 27 Jan 2025.

Builds on

5 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Multi-intention Inverse Q-learning for Interpretable Behavior Representation23 Nov
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 3

  1. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 641
  2. 2023Chain-of-Thought Predictive Controlcited by 18
  3. 2019Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamicscited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
  2. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  3. 2023Towards Theoretical Understanding of Inverse Reinforcement Learning
  4. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach
  5. 2019On the Correctness and Sample Complexity of Inverse Reinforcement Learning
  6. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges
  7. 2025Offline Reinforcement Learning via Inverse Optimization
  8. 2020Regularized Inverse Reinforcement Learning