# Full Bayesian Reinforcement Learning via LF-IBIS

arXiv: https://arxiv.org/abs/2607.01741 · HTML: https://mlsanity.com/paper/2607.01741/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [(More) Efficient Reinforcement Learning via Posterior Sampling](https://mlsanity.com/paper/1306.0940/) (2013)
- [Deep Multiagent Reinforcement Learning: Challenges and Directions](https://mlsanity.com/paper/2106.15691/) (2021)
- [Approximate Bayesian computation with the Wasserstein distance](https://mlsanity.com/paper/1905.03747/) (2019)
- [Model-Based Bayesian Exploration](https://mlsanity.com/paper/1301.6690/) (2013)
- [Response-adaptive randomization in clinical trials: from myths to practical considerations](https://mlsanity.com/paper/2005.00564/) (2020)
- [ABC Reinforcement Learning](https://mlsanity.com/paper/1303.6977/) (2013)
- [Classification and Bayesian Optimization for Likelihood-Free Inference](https://mlsanity.com/paper/1502.05503/) (2015)
- [A Markov Decision Process for Response-Adaptive Randomization in Clinical Trials](https://mlsanity.com/paper/2109.14642/) (2021)

## Abstract (arXiv)

Reinforcement Learning (RL) is a sequential decision-making framework in which an agent learns optimal policies through interaction with an environment by maximizing cumulative rewards. Among RL methods, Bayesian Reinforcement Learning (BRL) addresses common practical challenges related to data scarcity by leveraging prior knowledge about the environment and sequential belief updates. However, most BRL approaches require an explicit likelihood function, which is frequently inaccessible or intractable in real-world settings. We propose Likelihood-Free Iterated Batch Importance Sampling (LF-IBIS), a novel algorithm for BRL that updates the agent's beliefs online as new interactions become available. By combining Approximate Bayesian Computation with Iterated Batch Importance Sampling, LF-IBIS enables full Bayesian inference in settings where the environment dynamics are not described by an explicit or tractable likelihood. The method yields approximate posterior distributions over both environment parameters and optimal policies, providing a quantification of policy uncertainty useful for a Bayesian treatment of the exploration-exploitation trade-off. We test the method on a simulation study in response-adaptive randomization in clinical trials, where closed-form posteriors enable validation. Additional experiments address settings where the posterior has no closed form and illustrate online policy updating based on the posterior distribution of the optimal policy.