# Bayesian Reinforcement Learning: A Survey

arXiv: https://arxiv.org/abs/1609.04436 · HTML: https://mlsanity.com/paper/1609.04436/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2016.

## Built on by (most cited first, 2 of 2)

- [Can You Trust Your Preference Models? Stackelberg Games for Alignment](https://mlsanity.com/paper/2610.15008/) (2026)
- [CLASSO: a practical primal path-following algorithm for the constrained LASSO](https://mlsanity.com/paper/2610.17024/) (2026)

## Abstract (arXiv)

Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporating Bayesian reasoning in RL are: 1) it provides an elegant approach to action-selection (exploration/exploitation) as a function of the uncertainty in learning; and 2) it provides a machinery to incorporate prior knowledge into the algorithms. We first discuss models and methods for Bayesian inference in the simple single-step Bandit model. We then review the extensive recent literature on Bayesian methods for model-based RL, where prior information can be expressed on the parameters of the Markov model. We also present Bayesian methods for model-free RL, where priors are expressed over the value function or policy class. The objective of the paper is to provide a comprehensive survey on Bayesian RL algorithms and their theoretical and empirical properties.