mlsanityData to 7 Oct 2026

Towards Generalized Inverse Reinforcement Learning

arXiv 2402.07246 ↗First posted 11 Feb 2024cs.LGcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

This paper studies generalized inverse reinforcement learning (GIRL) in Markov decision processes (MDPs), that is, the problem of learning the basic components of an MDP given observed behavior (policy) that might not be optimal. These components include not only the reward function and transition probability matrices, but also the action space and state space that are not exactly known but are known to belong to given uncertainty sets. We address two key challenges in GIRL: first, the need to quantify the discrepancy between the observed policy and the underlying optimal policy; second, the difficulty of mathematically characterizing the underlying optimal policy when the basic components of an MDP are unobservable or partially observable. Then, we propose the mathematical formulation for GIRL and develop a fast heuristic algorithm. Numerical results on both finite and infinite state problems show the merit of our formulation and algorithm.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 7 of its 33 references that are in the map. First posted on arXiv 11 Feb 2024.

Builds on

7 of 33 references.

References · 7

  1. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  2. 2016Cooperative Inverse Reinforcement Learningcited by 298
  3. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  4. 2017Repeated Inverse Reinforcement Learningcited by 29
  5. 2018Generalized Inverse Optimization through Online Learningcited by 12
  6. 2020Wasserstein Distributionally Robust Inverse Multiobjective Optimizationcited by 4
  7. 2018Inferring Parameters Through Inverse Multiobjective Optimizationcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Inverse Reinforcement Learning for Minimum-Exposure Paths in Spatiotemporally Varying Scalar Fieldscited by 1
  2. 2024Inference of Utilities and Time Preference in Sequential Decision-Makingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Generalised Inverse Reinforcement Learning Framework
  2. 2019On the Correctness and Sample Complexity of Inverse Reinforcement Learning
  3. 2019Inverse Reinforcement Learning in Contextual MDPs
  4. 2020Generalized Inverse Planning: Learning Lifted non-Markovian Utility for Generalizable Task Representation ↗
  5. 2024How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approach
  6. 2023Inverse Reinforcement Learning With Constraint Recovery
  7. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learning
  8. 2021Identifiability in inverse reinforcement learning