# Self-orthogonalizing attractor neural networks emerging from the free energy principle

arXiv: https://arxiv.org/abs/2505.22749 · HTML: https://mlsanity.com/paper/2505.22749/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 78 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 15 of 15)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Attractor and integrator networks in the brain](https://mlsanity.com/paper/2112.03978/) (2021)
- [Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?](https://mlsanity.com/paper/2202.09467/) (2022)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [A free energy principle for a particular physics](https://mlsanity.com/paper/1906.10184/) (2019)
- [The free energy principle made simpler but not too simple](https://mlsanity.com/paper/2201.06387/) (2022)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [Path integrals, particular kinds, and strange things](https://mlsanity.com/paper/2210.12761/) (2022)
- [A Theoretical Framework for Inference and Learning in Predictive Coding Networks](https://mlsanity.com/paper/2207.12316/) (2022)
- [Thermodynamic Computing System for AI Applications](https://mlsanity.com/paper/2312.04836/) (2023)
- [Dreaming neural networks: forgetting spurious memories and reinforcing pure ones](https://mlsanity.com/paper/1810.12217/) (2018)
- [Markov Blankets in the Brain](https://mlsanity.com/paper/2006.02741/) (2020)
- [The explanatory power of activity flow models of brain function](https://mlsanity.com/paper/2402.02191/) (2024)

## Built on by (most cited first, 2 of 2)

- [Cycle is All You Need: More Is Different](https://mlsanity.com/paper/2509.21340/) (2025)
- [Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks](https://mlsanity.com/paper/2602.14885/) (2026)

## Abstract (arXiv)

Attractor dynamics are a hallmark of many complex systems, including the brain. Understanding how such self-organizing dynamics emerge from first principles is crucial for advancing our understanding of neuronal computations and the design of artificial intelligence systems. Here we formalize how attractor networks emerge from the free energy principle applied to a universal partitioning of random dynamical systems. Our approach obviates the need for explicitly imposed learning and inference rules and identifies emergent, but efficient and biologically plausible inference and learning dynamics for such self-organizing systems. These result in a collective, multi-level Bayesian active inference process. Attractors on the free energy landscape encode prior beliefs; inference integrates sensory data into posterior beliefs; and learning fine-tunes couplings to minimize long-term surprise. Analytically and via simulations, we establish that the proposed networks favor approximately orthogonalized attractor representations, a consequence of simultaneously optimizing predictive accuracy and model complexity. These attractors efficiently span the input subspace, enhancing generalization and the mutual information between hidden causes and observable effects. Furthermore, while random data presentation leads to symmetric and sparse couplings, sequential data fosters asymmetric couplings and non-equilibrium steady-state dynamics, offering a natural generalization of conventional Boltzmann Machines. Our findings offer a unifying theory of self-organizing attractor networks, providing novel insights for AI and neuroscience.