# Marginal MAP Estimation for Inverse RL under Occlusion with Observer Noise

arXiv: https://arxiv.org/abs/2109.07788 · HTML: https://mlsanity.com/paper/2109.07788/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 27 references that are in the map.
- First posted on arXiv 16 Sep 2021.

## Builds on (most cited first, 6 of 6)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Learning from Suboptimal Demonstration via Self-Supervised Reward Regression](https://mlsanity.com/paper/2010.11723/) (2020)
- [Inverse Reinforcement Learning Under Noisy Observations](https://mlsanity.com/paper/1710.10116/) (2017)
- [Inverse Reinforcement Learning with Missing Data](https://mlsanity.com/paper/1911.06930/) (2019)

## Built on by (most cited first, 3 of 3)

- [MVSA-Net: Multi-View State-Action Recognition for Robust and Deployable Trajectory Generation](https://mlsanity.com/paper/2311.08393/) (2023)
- [Visual IRL for Human-Like Robotic Manipulation](https://mlsanity.com/paper/2412.11360/) (2024)
- [Open Human-Robot Collaboration using Decentralized Inverse Reinforcement Learning](https://mlsanity.com/paper/2410.01790/) (2024)

## Abstract (arXiv)

We consider the problem of learning the behavioral preferences of an expert engaged in a task from noisy and partially-observable demonstrations. This is motivated by real-world applications such as a line robot learning from observing a human worker, where some observations are occluded by environmental objects that cannot be removed. Furthermore, robotic perception tends to be imperfect and noisy. Previous techniques for inverse reinforcement learning (IRL) take the approach of either omitting the missing portions or inferring it as part of expectation-maximization, which tends to be slow and prone to local optima. We present a new method that generalizes the well-known Bayesian maximum-a-posteriori (MAP) IRL method by marginalizing the occluded portions of the trajectory. This is additionally extended with an observation model to account for perception noise. We show that the marginal MAP (MMAP) approach significantly improves on the previous IRL technique under occlusion in both formative evaluations on a toy problem and in a summative evaluation on an onion sorting line task by a robot.