# Metric-Free Natural Gradient for Joint-Training of Boltzmann Machines

arXiv: https://arxiv.org/abs/1301.3545 · HTML: https://mlsanity.com/paper/1301.3545/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2013.

## Built on by (most cited first, 1 of 1)

- [Natural gradient and parameter estimation for quantum Boltzmann machines](https://mlsanity.com/paper/2410.24058/) (2024)

## Abstract (arXiv)

This paper introduces the Metric-Free Natural Gradient (MFNG) algorithm for training Boltzmann Machines. Similar in spirit to the Hessian-Free method of Martens [8], our algorithm belongs to the family of truncated Newton methods and exploits an efficient matrix-vector product to avoid explicitely storing the natural gradient metric L. This metric is shown to be the expected second derivative of the log-partition function (under the model distribution), or equivalently, the variance of the vector of partial derivatives of the energy function. We evaluate our method on the task of joint-training a 3-layer Deep Boltzmann Machine and show that MFNG does indeed have faster per-epoch convergence compared to Stochastic Maximum Likelihood with centering, though wall-clock performance is currently not competitive.