# Learning Gaussian-Bernoulli RBMs using Difference of Convex Functions Optimization

arXiv: https://arxiv.org/abs/2102.06228 · HTML: https://mlsanity.com/paper/2102.06228/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 39 references that are in the map.
- First posted on arXiv 11 Feb 2021.

## Builds on (most cited first, 5 of 5)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Gaussian-binary Restricted Boltzmann Machines on Modeling Natural Image Statistics](https://mlsanity.com/paper/1401.5900/) (2014)
- [Hierarchical Models as Marginals of Hierarchical Models](https://mlsanity.com/paper/1508.03606/) (2015)
- [Restricted Boltzmann Machines with Gaussian Visible Units Guided by Pairwise Constraints](https://mlsanity.com/paper/1701.03647/) (2017)
- [Learning RBM with a DC programming Approach](https://mlsanity.com/paper/1709.07149/) (2017)

## Built on by (most cited first, 2 of 2)

- [Gaussian-Bernoulli RBMs Without Tears](https://mlsanity.com/paper/2210.10318/) (2022)
- [Neural Boltzmann Machines](https://mlsanity.com/paper/2305.08337/) (2023)

## Abstract (arXiv)

The Gaussian-Bernoulli restricted Boltzmann machine (GB-RBM) is a useful generative model that captures meaningful features from the given n-dimensional continuous data. The difficulties associated with learning GB-RBM are reported extensively in earlier studies. They indicate that the training of the GB-RBM using the current standard algorithms, namely, contrastive divergence (CD) and persistent contrastive divergence (PCD), needs a carefully chosen small learning rate to avoid divergence which, in turn, results in slow learning. In this work, we alleviate such difficulties by showing that the negative log-likelihood for a GB-RBM can be expressed as a difference of convex functions if we keep the variance of the conditional distribution of visible units (given hidden unit states) and the biases of the visible units, constant. Using this, we propose a stochastic em difference of convex functions (DC) programming (S-DCP) algorithm for learning the GB-RBM. We present extensive empirical studies on several benchmark datasets to validate the performance of this S-DCP algorithm. It is seen that S-DCP is better than the CD and PCD algorithms in terms of speed of learning and the quality of the generative model learnt.