Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identify if these actions are consistent with optimizing a cost function. In a Bayesian (partially observed) setting, the inverse learner can at best identify optimality wrt the observed strategies. Our IRL algorithm identifies optimality and then constructs set-valued estimates of the cost function.To achieve this IRL objective, we use novel ideas from Bayesian revealed preferences stemming from microeconomics. We illustrate the proposed IRL scheme using two important examples of stopping time problems, namely, sequential hypothesis testing and Bayesian search. As a real-world example, we illustrate using a YouTube dataset comprising metadata from 190000 videos how the proposed IRL method predicts user engagement in online multimedia platforms with high accuracy. Finally, for finite datasets, we propose an IRL detection algorithm and give finite sample bounds on its error probabilities.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 14 of its 111 references that are in the map. First posted on arXiv 7 Jul 2020.
Builds on
14 of 111 references.
Closest ancestors · 4
Same direction, newest first
- 2019Identifying Cognitive Radars -- Inverse Reinforcement Learning using Revealed Preferences1 Dec
- 2019Rationally Inattentive Inverse Reinforcement Learning Explains YouTube Commenting Behavior24 Oct
- 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr
- 2018Inverse reinforcement learning in continuous time and space23 Jan
Other references · 10
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
- 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
- 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
- 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
- 2019The Assistive Multi-Armed Banditcited by 19
- 2019Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamicscited by 16
- 2016Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networkscited by 15
- 2018Regularized Contextual Banditscited by 4
- 2017Computer-aided diagnosis of lung nodule using gradient tree boosting and Bayesian optimizationcited by 1
- 2020Adversarial Radar Inference. From Inverse Tracking to Inverse Reinforcement Learning of Cognitive Radarcited by 1
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
- 2026A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models
- 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
- 2024Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
- 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
- 2024Walking the Values in Bayesian Inverse Reinforcement Learning
- 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
- 2024How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approach