# Gaussian-Bernoulli RBMs Without Tears

arXiv: https://arxiv.org/abs/2210.10318 · HTML: https://mlsanity.com/paper/2210.10318/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 9 of 9)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [High-dimensional Bayesian inference via the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1605.01559/) (2016)
- [Gaussian-binary Restricted Boltzmann Machines on Modeling Natural Image Statistics](https://mlsanity.com/paper/1401.5900/) (2014)
- [Mean-Field Inference in Gaussian Restricted Boltzmann Machine](https://mlsanity.com/paper/1512.00927/) (2015)
- [Learning Gaussian-Bernoulli RBMs using Difference of Convex Functions Optimization](https://mlsanity.com/paper/2102.06228/) (2021)

## Built on by (most cited first, 17 of 17)

- [Training Deep Boltzmann Networks with Sparse Ising Machines](https://mlsanity.com/paper/2303.10728/) (2023)
- [Explaining the effects of non-convergent sampling in the training of Energy-Based Models](https://mlsanity.com/paper/2301.09428/) (2023)
- [Fast training and sampling of Restricted Boltzmann Machines](https://mlsanity.com/paper/2405.15376/) (2024)
- [Attention in a family of Boltzmann machines emerging from modern Hopfield networks](https://mlsanity.com/paper/2212.04692/) (2022)
- [Fast and Functional Structured Data Generators Rooted in Out-of-Equilibrium Physics](https://mlsanity.com/paper/2307.06797/) (2023)
- [Diffusion-Based Hypothesis Testing and Change-Point Detection](https://mlsanity.com/paper/2506.16089/) (2025)
- [Robust Score-Based Quickest Change Detection](https://mlsanity.com/paper/2407.11094/) (2024)
- [Neural Boltzmann Machines](https://mlsanity.com/paper/2305.08337/) (2023)
- [End-to-end Training of Deep Boltzmann Machines by Unbiased Contrastive Divergence with Local Mode Initialization](https://mlsanity.com/paper/2305.19684/) (2023)
- [Density Operator Expectation Maximization](https://mlsanity.com/paper/2507.22786/) (2025)
- [Exploration of the generative capabilities of Boltzmann machines applied to social systems under the majority rule](https://mlsanity.com/paper/2607.23349/) (2026)
- [Can RBMs be trained with zero step contrastive divergence?](https://mlsanity.com/paper/2211.02174/) (2022)
- [Investigating the generative dynamics of energy-based neural networks](https://mlsanity.com/paper/2305.06745/) (2023)
- [Fusing Multi- and Hyperspectral Satellite Data for Harmful Algal Bloom Monitoring with Self-Supervised and Hierarchical Deep Learning](https://mlsanity.com/paper/2510.02763/) (2025)
- [Fraud detection in credit card transactions using Quantum-Assisted Restricted Boltzmann Machines](https://mlsanity.com/paper/2512.17660/) (2025)
- [Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering](https://mlsanity.com/paper/2607.27077/) (2026)
- [Classifying Directional Trajectories Near Criticality in the Three-State Majority-Vote Model with Deep Belief Networks and Bidirectional GRUs](https://mlsanity.com/paper/2608.18235/) (2026)

## Abstract (arXiv)

We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines (GRBMs), introducing two innovations. We propose a novel Gibbs-Langevin sampling algorithm that outperforms existing methods like Gibbs sampling. We propose a modified contrastive divergence (CD) algorithm so that one can generate images with GRBMs starting from noise. This enables direct comparison of GRBMs with deep generative models, improving evaluation protocols in the RBM literature. Moreover, we show that modified CD and gradient clipping are enough to robustly train GRBMs with large learning rates, thus removing the necessity of various tricks in the literature. Experiments on Gaussian Mixtures, MNIST, FashionMNIST, and CelebA show GRBMs can generate good samples, despite their single-hidden-layer architecture. Our code is released at: urlhttps://github.com/lrjconan/GRBM.