# Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems

arXiv: https://arxiv.org/abs/2007.03481 · HTML: https://mlsanity.com/paper/2007.03481/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 14 of its 111 references that are in the map.
- First posted on arXiv 7 Jul 2020.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 14 of 14)

- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations](https://mlsanity.com/paper/1904.06387/) (2019)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics](https://mlsanity.com/paper/1604.03912/) (2016)
- [The Assistive Multi-Armed Bandit](https://mlsanity.com/paper/1901.08654/) (2019)
- [Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamics](https://mlsanity.com/paper/1908.04696/) (2019)
- [Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks](https://mlsanity.com/paper/1612.03653/) (2016)
- [Identifying Cognitive Radars -- Inverse Reinforcement Learning using Revealed Preferences](https://mlsanity.com/paper/1912.00331/) (2019)
- [Rationally Inattentive Inverse Reinforcement Learning Explains YouTube Commenting Behavior](https://mlsanity.com/paper/1910.11703/) (2019)
- [Inverse reinforcement learning in continuous time and space](https://mlsanity.com/paper/1801.07663/) (2018)
- [Regularized Contextual Bandits](https://mlsanity.com/paper/1810.05065/) (2018)
- [Computer-aided diagnosis of lung nodule using gradient tree boosting and Bayesian optimization](https://mlsanity.com/paper/1708.05897/) (2017)
- [Adversarial Radar Inference. From Inverse Tracking to Inverse Reinforcement Learning of Cognitive Radar](https://mlsanity.com/paper/2002.10910/) (2020)

## Built on by (most cited first, 2 of 2)

- [Interacting Large Language Model Agents. Interpretable Models and Social Learning](https://mlsanity.com/paper/2411.01271/) (2024)
- [DO-IQS: Dynamics-Aware Offline Inverse Q-Learning for Optimal Stopping with Unknown Gain Functions](https://mlsanity.com/paper/2503.03515/) (2025)

## Abstract (arXiv)

This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identify if these actions are consistent with optimizing a cost function. In a Bayesian (partially observed) setting, the inverse learner can at best identify optimality wrt the observed strategies. Our IRL algorithm identifies optimality and then constructs set-valued estimates of the cost function.To achieve this IRL objective, we use novel ideas from Bayesian revealed preferences stemming from microeconomics. We illustrate the proposed IRL scheme using two important examples of stopping time problems, namely, sequential hypothesis testing and Bayesian search. As a real-world example, we illustrate using a YouTube dataset comprising metadata from 190000 videos how the proposed IRL method predicts user engagement in online multimedia platforms with high accuracy. Finally, for finite datasets, we propose an IRL detection algorithm and give finite sample bounds on its error probabilities.