Kernel Density Bayesian Inverse Reinforcement Learning
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
Inverse reinforcement learning (IRL) methods infer an agent's reward function using demonstrations of expert behavior. A Bayesian IRL approach models a distribution over candidate reward functions, capturing a degree of uncertainty in the inferred reward function. This is critical in some applications, such as those involving clinical data. Typically, Bayesian IRL algorithms require large demonstration datasets, which may not be available in practice. In this work, we incorporate existing domain-specific data to achieve better posterior concentration rates. We study a common setting in clinical and biological applications where we have access to expert demonstrations and known reward functions for a set of training tasks. Our aim is to learn the reward function of a new test task given limited expert demonstrations. Existing Bayesian IRL methods impose restrictions on the form of input data, thus limiting the incorporation of training task data. To better leverage information from training tasks, we introduce kernel density Bayesian inverse reinforcement learning (KD-BIRL). Our approach employs a conditional kernel density estimator, which uses the known reward functions of the training tasks to improve the likelihood estimation across a range of reward functions and demonstration samples. Our empirical results highlight KD-BIRL's faster concentration rate in comparison to baselines, particularly in low test task expert demonstration data regimes. Additionally, we are the first to provide theoretical guarantees of posterior concentration for a Bayesian IRL algorithm. Taken together, this work introduces a principled and theoretically grounded framework that enables Bayesian IRL to be applied across a variety of domains.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 56 references that are in the map. First posted on arXiv 13 Mar 2023.
Builds on
11 of 56 references.
Closest ancestors · 6
Same direction, newest first
- 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
- 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization17 Nov
- 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
- 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May
- 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May
- 2018Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling1 Mar
Other references · 5
- 2016Variational Inference: A Review for Statisticianscited by 1,472
- 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
- 2017Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimationcited by 34
- 2016Nonparametric Conditional Density Estimation in a High-Dimensional Regression Settingcited by 20
- 2017Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learningcited by 17
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Bayesian Inverse Reinforcement Learning for Non-Markovian Rewards
- 2026Nonparametric Bayesian Inverse Reinforcement Learning with Data-Parallel Gibbs Sampling
- 2024Walking the Values in Bayesian Inverse Reinforcement Learning
- 2022A Hierarchical Bayesian Approach to Inverse Reinforcement Learning with Symbolic Reward Machines
- 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
- 2019CWAE-IRL: Formulating a supervised approach to Inverse Reinforcement Learning problem
- 2025BiCQL-ML: A Bi-Level Conservative Q-Learning Framework for Maximum Likelihood Inverse Reinforcement Learning
- 2020Bayesian Robust Optimization for Imitation Learning