# Local Max-Entropy and Free Energy Principles, Belief Diffusions and their Singularities

arXiv: https://arxiv.org/abs/2310.02946 · HTML: https://mlsanity.com/paper/2310.02946/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 59 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [Topological Deep Learning: Going Beyond Graph Data](https://mlsanity.com/paper/2206.00606/) (2022)
- [Loopy Belief Propagation for Approximate Inference: An Empirical Study](https://mlsanity.com/paper/1301.6725/) (2013)
- [Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors](https://mlsanity.com/paper/1702.05882/) (2017)
- [Equivariant Message Passing Neural Network for Crystal Material Discovery](https://mlsanity.com/paper/2302.00485/) (2023)
- [Belief Propagation in Conditional RBMs for Structured Prediction](https://mlsanity.com/paper/1703.00986/) (2017)

## Abstract (arXiv)

A comprehensive picture of three Bethe-Kikuchi variational principles including their relationship to belief propagation (BP) algorithms on hypergraphs is given. The structure of BP equations is generalized to define continuous-time diffusions, solving localized versions of the max-entropy principle (A), the variational free energy principle (B), and a less usual equilibrium free energy principle (C), Legendre dual to A. Both critical points of Bethe-Kikuchi functionals and stationary beliefs are shown to lie at the non-linear intersection of two constraint surfaces, enforcing energy conservation and marginal consistency respectively. The hypersurface of singular beliefs, accross which equilibria become unstable as the constraint surfaces meet tangentially, is described by polynomial equations in the convex polytope of consistent beliefs. This polynomial is expressed by a loop series expansion for graphs of binary variables.