# Ratio Divergence Learning Using Target Energy in Restricted Boltzmann Machines: Beyond Kullback--Leibler Divergence Learning

arXiv: https://arxiv.org/abs/2409.07679 · HTML: https://mlsanity.com/paper/2409.07679/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 19 references that are in the map.
- First posted on arXiv 12 Sep 2024.

## Builds on (most cited first, 3 of 3)

- [Variational Neural Annealing](https://mlsanity.com/paper/2101.10154/) (2021)
- [Normalizing flows for atomic solids](https://mlsanity.com/paper/2111.08696/) (2021)
- [Frameworks to Design Approximation Algorithms for Finding Diverse Solutions in Combinatorial Problems](https://mlsanity.com/paper/2201.08940/) (2022)

## Built on by (most cited first, 1 of 1)

- [Understanding temperature tuning in energy-based models](https://mlsanity.com/paper/2512.09152/) (2025)

## Abstract (arXiv)

We propose ratio divergence (RD) learning for discrete energy-based models, a method that utilizes both training data and a tractable target energy function. We apply RD learning to restricted Boltzmann machines (RBMs), which are a minimal model that satisfies the universal approximation theorem for discrete distributions. RD learning combines the strength of both forward and reverse Kullback-Leibler divergence (KLD) learning, effectively addressing the "notorious" issues of underfitting with the forward KLD and mode-collapse with the reverse KLD. Since the summation of forward and reverse KLD seems to be sufficient to combine the strength of both approaches, we include this learning method as a direct baseline in numerical experiments to evaluate its effectiveness. Numerical experiments demonstrate that RD learning significantly outperforms other learning methods in terms of energy function fitting, mode-covering, and learning stability across various discrete energy-based models. Moreover, the performance gaps between RD learning and the other learning methods become more pronounced as the dimensions of target models increase.