# Bayesian brains and the R'enyi divergence

arXiv: https://arxiv.org/abs/2107.05438 · HTML: https://mlsanity.com/paper/2107.05438/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 77 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [Predictive Coding Approximates Backprop along Arbitrary Computation Graphs](https://mlsanity.com/paper/2006.04182/) (2020)
- [Deep active inference agents using Monte-Carlo methods](https://mlsanity.com/paper/2006.04176/) (2020)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Policy Optimization via Importance Sampling](https://mlsanity.com/paper/1809.06098/) (2018)
- [f-Divergence Variational Inference](https://mlsanity.com/paper/2009.13093/) (2020)
- [A Comparison of Optimization Algorithms for Deep Learning](https://mlsanity.com/paper/2007.14166/) (2020)
- [Wasserstein Variational Inference](https://mlsanity.com/paper/1805.11284/) (2018)
- [Variational f-divergence Minimization](https://mlsanity.com/paper/1907.11891/) (2019)
- [Thompson Sampling with Approximate Inference](https://mlsanity.com/paper/1908.04970/) (2019)
- [Alpha-Beta Divergence For Variational Inference](https://mlsanity.com/paper/1805.01045/) (2018)
- [Chance-Constrained Active Inference](https://mlsanity.com/paper/2102.08792/) (2021)
- [Goal-Directed Planning by Reinforcement Learning and Active Inference](https://mlsanity.com/paper/2106.09938/) (2021)

## Built on by (most cited first, 6 of 6)

- [On efficient computation in active inference](https://mlsanity.com/paper/2307.00504/) (2023)
- [Function-space regularized R'enyi divergences](https://mlsanity.com/paper/2210.04974/) (2022)
- [Active Inference and Epistemic Value in Graphical Models](https://mlsanity.com/paper/2109.00541/) (2021)
- [Deep Active Inference Agents for Delayed and Long-Horizon Environments](https://mlsanity.com/paper/2505.19867/) (2025)
- [R'enyi Divergence Deep Mutual Learning](https://mlsanity.com/paper/2209.05732/) (2022)
- [Active Inference Meeting Energy-Efficient Control of Parallel and Identical Machines](https://mlsanity.com/paper/2406.09322/) (2024)

## Abstract (arXiv)

Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioural preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alternative account of behavioural variability using R'enyi divergences and their associated variational bounds. R'enyi bounds are analogous to the variational free energy (or evidence lower bound) and can be derived under the same assumptions. Importantly, these bounds provide a formal way to establish behavioural differences through an α parameter, given fixed priors. This rests on changes in α that alter the bound (on a continuous scale), inducing different posterior estimates and consequent variations in behaviour. Thus, it looks as if individuals have different priors, and have reached different conclusions. More specifically, α → 0+ optimisation leads to mass-covering variational estimates and increased variability in choice behaviour. Furthermore, α → + ∞ optimisation leads to mass-seeking variational posteriors and greedy preferences. We exemplify this formulation through simulations of the multi-armed bandit task. We note that these α parameterisations may be especially relevant, i.e., shape preferences, when the true posterior is not in the same family of distributions as the assumed (simpler) approximate density, which may be the case in many real-world scenarios. The ensuing departure from vanilla variational inference provides a potentially useful explanation for differences in behavioural preferences of biological (or artificial) agents under the assumption that the brain performs variational Bayesian inference.