# Phase transitions in Restricted Boltzmann Machines with generic priors

arXiv: https://arxiv.org/abs/1612.03132 · HTML: https://mlsanity.com/paper/1612.03132/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2016.

## Built on by (most cited first, 20 of 20)

- [A phase transition between positional and semantic learning in a solvable model of dot-product attention](https://mlsanity.com/paper/2402.03902/) (2024)
- [Quantum field-theoretic machine learning](https://mlsanity.com/paper/2102.09449/) (2021)
- [Supervised Hebbian Learning](https://mlsanity.com/paper/2203.01304/) (2022)
- [Restricted Boltzmann Machines as Models of Interacting Variables](https://mlsanity.com/paper/2103.15917/) (2021)
- [Hopfield model with planted patterns: a teacher-student self-supervised learning model](https://mlsanity.com/paper/2304.13710/) (2023)
- [Unified CNNs and transformers underlying learning mechanism reveals multi-head attention modus vivendi](https://mlsanity.com/paper/2501.12900/) (2025)
- [Dense Hopfield Networks in the Teacher-Student Setting](https://mlsanity.com/paper/2401.04191/) (2024)
- [Low-latency vision transformers via large-scale multi-head attention](https://mlsanity.com/paper/2506.23832/) (2025)
- [Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines](https://mlsanity.com/paper/2406.09924/) (2024)
- [The effect of priors on Learning with Restricted Boltzmann Machines](https://mlsanity.com/paper/2412.02623/) (2024)
- [Recurrent neural networks that generalize from examples and optimize by dreaming](https://mlsanity.com/paper/2204.07954/) (2022)
- [Modeling Structured Data Learning with Restricted Boltzmann Machines in the Teacher-Student Setting](https://mlsanity.com/paper/2410.16150/) (2024)
- [Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories](https://mlsanity.com/paper/2307.08365/) (2023)
- [Role of Delay in Brain Dynamics](https://mlsanity.com/paper/2410.11384/) (2024)
- [On the role of non-linear latent features in bipartite generative neural networks](https://mlsanity.com/paper/2506.10552/) (2025)
- [Saddle Hierarchy in Dense Associative Memory](https://mlsanity.com/paper/2508.19151/) (2025)
- [Phase diagram and eigenvalue dynamics of stochastic gradient descent in multilayer neural networks](https://mlsanity.com/paper/2509.01349/) (2025)
- [Unsupervised and Supervised learning by Dense Associative Memory under replica symmetry breaking](https://mlsanity.com/paper/2312.09638/) (2023)
- [Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modeling](https://mlsanity.com/paper/2512.11415/) (2025)
- [Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines](https://mlsanity.com/paper/2605.19178/) (2026)

## Abstract (arXiv)

We study Generalised Restricted Boltzmann Machines with generic priors for units and weights, interpolating between Boolean and Gaussian variables. We present a complete analysis of the replica symmetric phase diagram of these systems, which can be regarded as Generalised Hopfield models. We underline the role of the retrieval phase for both inference and learning processes and we show that retrieval is robust for a large class of weight and unit priors, beyond the standard Hopfield scenario. Furthermore we show how the paramagnetic phase boundary is directly related to the optimal size of the training set necessary for good generalisation in a teacher-student scenario of unsupervised learning.