Q-based Variational Inverse Reinforcement Learning
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
The development of safe and beneficial AI requires that systems can learn and act in accordance with human preferences. However, explicitly specifying these preferences by hand is often infeasible. Inverse reinforcement learning (IRL) addresses this challenge by inferring preferences, represented as reward functions, from expert behaviour. We introduce Q-based Variational IRL (QVIRL), a novel Bayesian IRL method that recovers a posterior distribution over rewards from expert demonstrations via primarily learning a variational distribution over optimal Q-values. Unlike previous approaches, QVIRL combines scalability with uncertainty quantification, important for safety-critical applications as well as active learning. We demonstrate QVIRL's strong performance in apprenticeship learning across various tasks, including gridworlds, Lunar Lander, the Highway Environment, and two ATARI games both with static expert data and with active learning. It is the first method for Bayesian IRL that demonstrates training from raw pixel observations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 41 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
18 of 41 references.
Closest ancestors · 7
Same direction, newest first
- 2025PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning5 Aug
- 2024Walking the Values in Bayesian Inverse Reinforcement Learning15 Jul
- 2023Massively Scalable Inverse Reinforcement Learning in Google Maps18 May
- 2021Identifiability in inverse reinforcement learning7 Jun
- 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
- 2019Risk-Aware Active Inverse Reinforcement Learning8 Jan
- 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun
Other references · 11
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2017Distributional Reinforcement Learning with Quantile Regressioncited by 474
- 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
- 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
- 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
- 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
- 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
- 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Drivingcited by 30
- 2018Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learningcited by 28
- 2022Driving in Real Life with Inverse Reinforcement Learningcited by 18
- 2020Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimationcited by 16
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
- 2025BiCQL-ML: A Bi-Level Conservative Q-Learning Framework for Maximum Likelihood Inverse Reinforcement Learning
- 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
- 2019CWAE-IRL: Formulating a supervised approach to Inverse Reinforcement Learning problem
- 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
- 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
- 2022Active Exploration for Inverse Reinforcement Learning
- 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables