# Bayesian model reduction

arXiv: https://arxiv.org/abs/1805.07092 · HTML: https://mlsanity.com/paper/1805.07092/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2018.

## Built on by (most cited first, 19 of 19)

- [Designing Ecosystems of Intelligence from First Principles](https://mlsanity.com/paper/2212.01354/) (2022)
- [Supervised structure learning](https://mlsanity.com/paper/2311.10300/) (2023)
- [Branching Time Active Inference: empirical study and complexity class analysis](https://mlsanity.com/paper/2111.11276/) (2021)
- [Principled Pruning of Bayesian Neural Networks through Variational Free Energy Minimization](https://mlsanity.com/paper/2210.09134/) (2022)
- [Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents](https://mlsanity.com/paper/2203.10592/) (2022)
- [Exploring and Learning Structure: Active Inference Approach in Navigational Agents](https://mlsanity.com/paper/2408.05982/) (2024)
- [Toward Design of Synthetic Active Inference Agents by Mere Mortals](https://mlsanity.com/paper/2307.14145/) (2023)
- [Possible Principles for Aligned Structure Learning Agents](https://mlsanity.com/paper/2410.00258/) (2024)
- [Learning Dynamic Cognitive Map with Autonomous Navigation](https://mlsanity.com/paper/2411.08447/) (2024)
- [AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models](https://mlsanity.com/paper/2505.24784/) (2025)
- [Active inference and artificial reasoning](https://mlsanity.com/paper/2512.21129/) (2025)
- [BMRS: Bayesian Model Reduction for Structured Pruning](https://mlsanity.com/paper/2406.01345/) (2024)
- [Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker](https://mlsanity.com/paper/2104.11798/) (2021)
- [A Mathematical Walkthrough and Discussion of the Free Energy Principle](https://mlsanity.com/paper/2108.13343/) (2021)
- [Bayesian sparsification for deep neural networks with Bayesian model reduction](https://mlsanity.com/paper/2309.12095/) (2023)
- [Structure learning with Temporal Gaussian Mixture for model-based Reinforcement Learning](https://mlsanity.com/paper/2411.11511/) (2024)
- [AIDA: An Active Inference-based Design Agent for Audio Processing Algorithms](https://mlsanity.com/paper/2112.13366/) (2021)
- [Automating Model Comparison in Factor Graphs](https://mlsanity.com/paper/2306.05965/) (2023)
- [Deep hybrid models: infer and plan in a dynamic world](https://mlsanity.com/paper/2402.10088/) (2024)

## Abstract (arXiv)

This paper reviews recent developments in statistical structure learning; namely, Bayesian model reduction. Bayesian model reduction is a method for rapidly computing the evidence and parameters of probabilistic models that differ only in their priors. In the setting of variational Bayes this has an analytical solution, which finesses the problem of scoring large model spaces in model comparison or structure learning. In this technical note, we review Bayesian model reduction and provide the relevant equations for several discrete and continuous probability distributions. We provide worked examples in the context of multivariate linear regression, Gaussian mixture models and dynamical systems (dynamic causal modelling). These examples are accompanied by the Matlab scripts necessary to reproduce the results. Finally, we briefly review recent applications in the fields of neuroimaging and neuroscience. Specifically, we consider structure learning and hierarchical or empirical Bayes that can be regarded as a metaphor for neurobiological processes like abductive reasoning.