mlsanityData to 7 Oct 2026

Model Selection for Inverse Reinforcement Learning via Structural Risk Minimization

arXiv 2312.16566 ↗First posted 27 Dec 2023cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to contain the ideal reward function, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward function class from a hypothesis set minimizing both estimation error and model complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the model penalty of hypothesis function classes. The SRM learning guarantee is further presented. In particular, we provide the explicit form for the linear weighted sum setting. Simulations demonstrate the performance and efficiency of our algorithm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 29 references that are in the map. First posted on arXiv 27 Dec 2023.

Builds on

6 of 29 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Teaching Inverse Reinforcement Learners via Features and Demonstrations21 Oct
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 4

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2018Behavioral Cloning from Observationcited by 482
  3. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  4. 2023Structural Risk Minimization for Learning Nonlinear Dynamicscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees
  2. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
  3. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  4. 2026A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models
  5. 2024Automated Feature Selection for Inverse Reinforcement Learning
  6. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
  7. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach
  8. 2021A Generalised Inverse Reinforcement Learning Framework