# Inverse Reinforcement Learning with Missing Data

arXiv: https://arxiv.org/abs/1911.06930 · HTML: https://mlsanity.com/paper/1911.06930/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 16 references that are in the map.
- First posted on arXiv 16 Nov 2019.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 2 of 2)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Inverse Reinforcement Learning Under Noisy Observations](https://mlsanity.com/paper/1710.10116/) (2017)

## Built on by (most cited first, 2 of 2)

- [Reinforcement Learning Approaches for Traffic Signal Control under Missing Data](https://mlsanity.com/paper/2304.10722/) (2023)
- [Marginal MAP Estimation for Inverse RL under Occlusion with Observer Noise](https://mlsanity.com/paper/2109.07788/) (2021)

## Abstract (arXiv)

We consider the problem of recovering an expert's reward function with inverse reinforcement learning (IRL) when there are missing/incomplete state-action pairs or observations in the demonstrated trajectories. This issue of missing trajectory data or information occurs in many situations, e.g., GPS signals from vehicles moving on a road network are intermittent. In this paper, we propose a tractable approach to directly compute the log-likelihood of demonstrated trajectories with incomplete/missing data. Our algorithm is efficient in handling a large number of missing segments in the demonstrated trajectories, as it performs the training with incomplete data by solving a sequence of systems of linear equations, and the number of such systems to be solved does not depend on the number of missing segments. Empirical evaluation on a real-world dataset shows that our training algorithm outperforms other conventional techniques.