# Quantifying Potential Observation Missingness in Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.12831 · HTML: https://mlsanity.com/paper/2605.12831/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 16 of 16)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Robust Imitation of Diverse Behaviors](https://mlsanity.com/paper/1707.02747/) (2017)
- [Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences](https://mlsanity.com/paper/2002.09089/) (2020)
- [Confounding-Robust Policy Evaluation in Infinite-Horizon Reinforcement Learning](https://mlsanity.com/paper/2002.04518/) (2020)
- [Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning](https://mlsanity.com/paper/2010.03118/) (2020)
- [Learning Multimodal Rewards from Rankings](https://mlsanity.com/paper/2109.12750/) (2021)
- [Causal Imitation Learning with Unobserved Confounders](https://mlsanity.com/paper/2208.06267/) (2022)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [Provably Efficient Causal Reinforcement Learning with Confounded Observational Data](https://mlsanity.com/paper/2006.12311/) (2020)
- [Truly Batch Apprenticeship Learning with Deep Successor Features](https://mlsanity.com/paper/1903.10077/) (2019)
- [Sequential Causal Imitation Learning with Unobserved Confounders](https://mlsanity.com/paper/2208.06276/) (2022)
- [Inverse Contextual Bandits: Learning How Behavior Evolves over Time](https://mlsanity.com/paper/2107.06317/) (2021)
- [Inverse Decision Modeling: Learning Interpretable Representations of Behavior](https://mlsanity.com/paper/2310.18591/) (2023)
- [Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors](https://mlsanity.com/paper/1903.10304/) (2019)
- [Inverse Reinforcement Learning with Sub-optimal Experts](https://mlsanity.com/paper/2401.03857/) (2024)
- [Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning](https://mlsanity.com/paper/2310.19831/) (2023)

## Abstract (arXiv)

Inverse reinforcement learning (IRL), which infers reward functions from demonstrations, is a valuable tool for modeling and understanding decision-making behavior. Many variants of IRL have been developed to capture complexities of human decision-making, such as subjective beliefs, imperfect planning, and dynamic goals. However, an often-overlooked issue in real-world behavioral datasets is that the recorded data may be missing observations that were available to the original decision-maker. In use-inspired settings such as healthcare, this can make expert actions appear suboptimal, even when they were near-optimal given the information available at the time. As a result, the rewards learned by standard IRL may be misleading. In this paper, we identify the minimal perturbations to the recorded observations needed for the expert's actions to appear optimal. We develop a practical algorithm for this problem and demonstrate its utility for quantifying the possible extent of missing observations in behavioral datasets through extensive experiments on synthetic navigation tasks, a cancer treatment simulator, and ICU treatment data.