On the role of non-linear latent features in bipartite generative neural networks
Not on the map. Why?
Abstract (arXiv)
We investigate the phase diagram and memory retrieval capabilities of bipartite energy-based neural networks, namely Restricted Boltzmann Machines (RBMs), as a function of the prior distribution imposed on their hidden units - including binary, multi-state, and ReLU-like activations. Drawing connections to the Hopfield model and employing analytical tools from statistical physics of disordered systems, we explore how the architectural choices and activation functions shape the thermodynamic properties of these models. Our analysis reveals that standard RBMs with binary hidden nodes and extensive connectivity suffer from reduced critical capacity, limiting their effectiveness as associative memories. To address this, we examine several modifications, such as introducing local biases and adopting richer hidden unit priors. These adjustments restore ordered retrieval phases and markedly improve recall performance, even at finite temperatures. Our theoretical findings, supported by finite-size Monte Carlo simulations, highlight the importance of hidden unit design in enhancing the expressive power of RBMs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 25 of its 71 references that are in the map. First posted on arXiv 12 Jun 2025.
Builds on
25 of 71 references.
References · 25
- 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
- 2020A Review on Generative Adversarial Networks: Algorithms, Theory, and Applicationscited by 209
- 2021The physics of higher-order interactions in complex systemscited by 63
- 2016Interaction Screening: Efficient and Sample-Optimal Learning of Ising Modelscited by 43
- 2020Restricted Boltzmann Machine, recent advances and mean-field theorycited by 31
- 2021Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machinescited by 27
- 2017Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priorscited by 25
- 2018Thermodynamics of Restricted Boltzmann Machines and related learning dynamicscited by 25
- 2016Emergence of Compositional Representations in Restricted Boltzmann Machinescited by 23
- 2016Phase transitions in Restricted Boltzmann Machines with generic priorscited by 20
- 2018Dreaming neural networks: forgetting spurious memories and reinforcing pure onescited by 15
- 2024Cascade of phase transitions in the training of Energy-based modelscited by 12
13 more
- 2022Learning a Restricted Boltzmann Machine using biased Monte Carlo samplingcited by 11
- 2023Storage and Learning phase transitions in the Random-Features Hopfield Modelcited by 11
- 2019'Place-cell' emergence and learning of invariant data with restricted Boltzmann machines: breaking and dynamical restoration of continuous symmetries in the weight spacecited by 10
- 2021Restricted Boltzmann Machines as Models of Interacting Variablescited by 9
- 2024Fast training and sampling of Restricted Boltzmann Machinescited by 9
- 2023Inferring effective couplings with Restricted Boltzmann Machinescited by 8
- 2023Hopfield model with planted patterns: a teacher-student self-supervised learning modelcited by 7
- 2023Fast and Functional Structured Data Generators Rooted in Out-of-Equilibrium Physicscited by 7
- 2025Inferring Higher-Order Couplings with Neural Networkscited by 7
- 2018Dreaming neural networks: rigorous resultscited by 6
- 2025The unbearable lightness of Restricted Boltzmann Machines: Theoretical Insights and Biological Applicationscited by 4
- 2024Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machinescited by 3
- 2024The effect of priors on Learning with Restricted Boltzmann Machinescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.