mlsanityData to 14 Oct 2026

Quantifying Potential Observation Missingness in Inverse Reinforcement Learning

arXiv 2605.12831 ↗First posted 12 May 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Inverse reinforcement learning (IRL), which infers reward functions from demonstrations, is a valuable tool for modeling and understanding decision-making behavior. Many variants of IRL have been developed to capture complexities of human decision-making, such as subjective beliefs, imperfect planning, and dynamic goals. However, an often-overlooked issue in real-world behavioral datasets is that the recorded data may be missing observations that were available to the original decision-maker. In use-inspired settings such as healthcare, this can make expert actions appear suboptimal, even when they were near-optimal given the information available at the time. As a result, the rewards learned by standard IRL may be misleading. In this paper, we identify the minimal perturbations to the recorded observations needed for the expert's actions to appear optimal. We develop a practical algorithm for this problem and demonstrate its utility for quantifying the possible extent of missing observations in behavioral datasets through extensive experiments on synthetic navigation tasks, a cancer treatment simulator, and ICU treatment data.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 37 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

16 of 37 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Inverse Reinforcement Learning with Sub-optimal Experts8 Jan
  2. 2018Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior21 May

Other references · 14

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2017Robust Imitation of Diverse Behaviorscited by 86
  3. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  4. 2020Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learningcited by 57
  5. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  6. 2021Learning Multimodal Rewards from Rankingscited by 41
  7. 2022Causal Imitation Learning with Unobserved Confounderscited by 41
  8. 2020Provably Efficient Causal Reinforcement Learning with Confounded Observational Datacited by 40
  9. 2019Truly Batch Apprenticeship Learning with Deep Successor Featurescited by 27
  10. 2022Sequential Causal Imitation Learning with Unobserved Confounderscited by 25
  11. 2021Inverse Contextual Bandits: Learning How Behavior Evolves over Timecited by 12
  12. 2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
2 more
  1. 2019Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviorscited by 9
  2. 2023Explaining by Imitating: Understanding Decisions by Interpretable Policy Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2019Inverse Reinforcement Learning with Missing Data
  2. 2024Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecification
  3. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  4. 2017Inverse Reinforcement Learning from Summary Data
  5. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  6. 2024Pruning the Path to Optimal Care: Identifying Systematically Suboptimal Medical Decision-Making with Inverse Reinforcement Learning
  7. 2024Partial Identifiability and Misspecification in Inverse Reinforcement Learning
  8. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles