# Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors

arXiv: https://arxiv.org/abs/1702.05882 · HTML: https://mlsanity.com/paper/1702.05882/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2017.

## Built on by (most cited first, 25 of 25)

- [The emergence of a concept in shallow neural networks](https://mlsanity.com/paper/2109.00454/) (2021)
- [Quantum field-theoretic machine learning](https://mlsanity.com/paper/2102.09449/) (2021)
- [Supervised Hebbian Learning](https://mlsanity.com/paper/2203.01304/) (2022)
- [Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling](https://mlsanity.com/paper/2206.01310/) (2022)
- [Dense Associative Memory with Epanechnikov Energy](https://mlsanity.com/paper/2506.10801/) (2025)
- [Restricted Boltzmann Machines as Models of Interacting Variables](https://mlsanity.com/paper/2103.15917/) (2021)
- [Inferring effective couplings with Restricted Boltzmann Machines](https://mlsanity.com/paper/2309.02292/) (2023)
- [Hopfield model with planted patterns: a teacher-student self-supervised learning model](https://mlsanity.com/paper/2304.13710/) (2023)
- [Dense Hopfield Networks in the Teacher-Student Setting](https://mlsanity.com/paper/2401.04191/) (2024)
- [Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines](https://mlsanity.com/paper/2406.09924/) (2024)
- [The effect of priors on Learning with Restricted Boltzmann Machines](https://mlsanity.com/paper/2412.02623/) (2024)
- [Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting](https://mlsanity.com/paper/2410.16150/) (2024)
- [Equivalence between algorithmic instability and transition to replica symmetry breaking in perceptron learning systems](https://mlsanity.com/paper/2111.13302/) (2021)
- [Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories](https://mlsanity.com/paper/2307.08365/) (2023)
- [Neural Learning Rules from Associative Networks Theory](https://mlsanity.com/paper/2503.19922/) (2025)
- [Learning with Restricted Boltzmann Machines: Asymptotics of AMP and GD in High Dimensions](https://mlsanity.com/paper/2505.18046/) (2025)
- [On the role of non-linear latent features in bipartite generative neural networks](https://mlsanity.com/paper/2506.10552/) (2025)
- [Saddle Hierarchy in Dense Associative Memory](https://mlsanity.com/paper/2508.19151/) (2025)
- [Pattern reconstruction with restricted Boltzmann machines](https://mlsanity.com/paper/2205.07087/) (2022)
- [Local Max-Entropy and Free Energy Principles, Belief Diffusions and their Singularities](https://mlsanity.com/paper/2310.02946/) (2023)
- [Unsupervised and Supervised learning by Dense Associative Memory under replica symmetry breaking](https://mlsanity.com/paper/2312.09638/) (2023)
- [Dataset-Free Weight-Initialization on Restricted Boltzmann Machine](https://mlsanity.com/paper/2409.07708/) (2024)
- [Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines](https://mlsanity.com/paper/2605.19178/) (2026)
- [Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering](https://mlsanity.com/paper/2607.27077/) (2026)
- [Phases in a class of associative memories via hidden neurons](https://mlsanity.com/paper/2609.10976/) (2026)

## Abstract (arXiv)

Restricted Boltzmann Machines are described by the Gibbs measure of a bipartite spin glass, which in turn corresponds to the one of a generalised Hopfield network. This equivalence allows us to characterise the state of these systems in terms of retrieval capabilities, both at low and high load. We study the paramagnetic-spin glass and the spin glass-retrieval phase transitions, as the pattern (i.e. weight) distribution and spin (i.e. unit) priors vary smoothly from Gaussian real variables to Boolean discrete variables. Our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard Hopfield model with Boolean patterns. The retrieval region is larger when the pattern entries and retrieval units get more peaked and, conversely, when the hidden units acquire a broader prior and therefore have a stronger response to high fields. Moreover, at low load retrieval always exists below some critical temperature, for every pattern distribution ranging from the Boolean to the Gaussian case.