# Barriers and Dynamical Paths in Alternating Gibbs Sampling of Restricted Boltzmann Machines

arXiv: https://arxiv.org/abs/2107.06013 · HTML: https://mlsanity.com/paper/2107.06013/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 45 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 3 of 3)

- [Restricted Boltzmann Machine, recent advances and mean-field theory](https://mlsanity.com/paper/2011.11307/) (2020)
- [Thermodynamics of Restricted Boltzmann Machines and related learning dynamics](https://mlsanity.com/paper/1803.01960/) (2018)
- [Statistical Physics and Representations in Real and Artificial Neural Networks](https://mlsanity.com/paper/1709.02470/) (2017)

## Built on by (most cited first, 11 of 11)

- [Supervised Hebbian Learning](https://mlsanity.com/paper/2203.01304/) (2022)
- [Learning a Restricted Boltzmann Machine using biased Monte Carlo sampling](https://mlsanity.com/paper/2206.01310/) (2022)
- [Interpretable machine learning of amino acid patterns in proteins: a statistical ensemble approach](https://mlsanity.com/paper/2303.15228/) (2023)
- [Parallel Learning by Multitasking Neural Networks](https://mlsanity.com/paper/2308.04106/) (2023)
- [Fundamental operating regimes, hyper-parameter fine-tuning and glassiness: towards an interpretable replica-theory for trained restricted Boltzmann machines](https://mlsanity.com/paper/2406.09924/) (2024)
- [Recurrent neural networks that generalize from examples and optimize by dreaming](https://mlsanity.com/paper/2204.07954/) (2022)
- [Three Learning Stages and Accuracy-Efficiency Tradeoff of Restricted Boltzmann Machines](https://mlsanity.com/paper/2209.00873/) (2022)
- [Toward Unlimited Self-Learning MCMC with Parallel Adaptive Annealing](https://mlsanity.com/paper/2211.14024/) (2022)
- [Investigating the generative dynamics of energy-based neural networks](https://mlsanity.com/paper/2305.06745/) (2023)
- [Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modeling](https://mlsanity.com/paper/2512.11415/) (2025)
- [Learning from Noise: Effective-Rank Collapse and Out-of-Distribution Rejection in Restricted Boltzmann Machines](https://mlsanity.com/paper/2607.10506/) (2026)

## Abstract (arXiv)

Restricted Boltzmann Machines (RBM) are bi-layer neural networks used for the unsupervised learning of model distributions from data. The bipartite architecture of RBM naturally defines an elegant sampling procedure, called Alternating Gibbs Sampling (AGS), where the configurations of the latent-variable layer are sampled conditional to the data-variable layer, and vice versa. We study here the performance of AGS on several analytically tractable models borrowed from statistical mechanics. We show that standard AGS is not more efficient than classical Metropolis-Hastings (MH) sampling of the effective energy landscape defined on the data layer. However, RBM can identify meaningful representations of training data in their latent space. Furthermore, using these representations and combining Gibbs sampling with the MH algorithm in the latent space can enhance the sampling performance of the RBM when the hidden units encode weakly dependent features of the data. We illustrate our findings on three datasets: Bars and Stripes and MNIST, well known in machine learning, and the so-called Lattice Proteins, introduced in theoretical biology to study the sequence-to-structure mapping in proteins.