# A Mathematical Walkthrough and Discussion of the Free Energy Principle

arXiv: https://arxiv.org/abs/2108.13343 · HTML: https://mlsanity.com/paper/2108.13343/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 100 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 17 of 17)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [Active inference on discrete state-spaces: a synthesis](https://mlsanity.com/paper/2001.07203/) (2020)
- [The free energy principle for action and perception: A mathematical review](https://mlsanity.com/paper/1705.09156/) (2017)
- [Deep Active Inference as Variational Policy Gradients](https://mlsanity.com/paper/1907.03876/) (2019)
- [Reinforcement Learning through Active Inference](https://mlsanity.com/paper/2002.12636/) (2020)
- [A free energy principle for a particular physics](https://mlsanity.com/paper/1906.10184/) (2019)
- [Bayesian model reduction](https://mlsanity.com/paper/1805.07092/) (2018)
- [Bayesian Mechanics for Stationary Processes](https://mlsanity.com/paper/2106.13830/) (2021)
- [How particular is the physics of the free energy principle?](https://mlsanity.com/paper/2105.11203/) (2021)
- [Generalizing the No-U-Turn Sampler to Riemannian Manifolds](https://mlsanity.com/paper/1304.1920/) (2013)
- [Some interesting observations on the free energy principle](https://mlsanity.com/paper/2002.04501/) (2020)
- [Neural and phenotypic representation under the free-energy principle](https://mlsanity.com/paper/2008.03238/) (2020)
- [A Technical Critique of Some Parts of the Free Energy Principle](https://mlsanity.com/paper/2001.06408/) (2020)

## Built on by (most cited first, 1 of 1)

- [Successor Representation Active Inference](https://mlsanity.com/paper/2207.09897/) (2022)

## Abstract (arXiv)

The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a deep and powerful connection between the stochastic thermodynamics of self-organization and learning through variational inference. Specifically, it claims that any self-organizing system which can be statistically separated from its environment, and which maintains itself at a non-equilibrium steady state, can be construed as minimizing an information-theoretic functional -- the variational free energy -- and thus performing variational Bayesian inference to infer the hidden state of its environment. This principle has also been applied extensively in neuroscience, and is beginning to make inroads in machine learning by spurring the construction of novel and powerful algorithms by which action, perception, and learning can all be unified under a single objective. While its expansive and often grandiose claims have spurred significant debates in both philosophy and theoretical neuroscience, the mathematical depth and lack of accessible introductions and tutorials for the core claims of the theory have often precluded a deep understanding within the literature. Here, we aim to provide a mathematically detailed, yet intuitive walk-through of the formulation and central claims of the FEP while also providing a discussion of the assumptions necessary and potential limitations of the theory. Additionally, since the FEP is a still a living theory, subject to internal controversy, change, and revision, we also present a detailed appendix highlighting and condensing current perspectives as well as controversies about the nature, applicability, and the mathematical assumptions and formalisms underlying the FEP.