# Training Deep Boltzmann Networks with Sparse Ising Machines

arXiv: https://arxiv.org/abs/2303.10728 · HTML: https://mlsanity.com/paper/2303.10728/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 82 references that are in the map.
- First posted on arXiv 19 Mar 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Application of Quantum Annealing to Training of Deep Neural Networks](https://mlsanity.com/paper/1510.06356/) (2015)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [Massively Parallel Probabilistic Computing with Sparse Ising Machines](https://mlsanity.com/paper/2110.02481/) (2021)
- [Training and Classification using a Restricted Boltzmann Machine on the D-Wave 2000Q](https://mlsanity.com/paper/2005.03247/) (2020)
- [Gaussian-Bernoulli RBMs Without Tears](https://mlsanity.com/paper/2210.10318/) (2022)
- [Thermodynamic AI and the fluctuation frontier](https://mlsanity.com/paper/2302.06584/) (2023)
- [A full-stack view of probabilistic computing with p-bits: devices, architectures and algorithms](https://mlsanity.com/paper/2302.06457/) (2023)
- [Scrambled Linear Pseudorandom Number Generators](https://mlsanity.com/paper/1805.01407/) (2018)
- [Noise-injected analog Ising machines enable ultrafast statistical sampling and machine learning](https://mlsanity.com/paper/2112.11534/) (2021)
- [Efficient Optimization with Higher-Order Ising Machines](https://mlsanity.com/paper/2212.03426/) (2022)
- [High Performance Monte Carlo Simulation of Ising Model on TPU Clusters](https://mlsanity.com/paper/1903.11714/) (2019)
- [A Hybrid Quantum enabled RBM Advantage: Convolutional Autoencoders For Quantum Image Compression and Generative Learning](https://mlsanity.com/paper/2001.11946/) (2020)
- [Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks](https://mlsanity.com/paper/1611.01427/) (2016)
- [Accelerating Deep Learning with Memcomputing](https://mlsanity.com/paper/1801.00512/) (2018)
- [Joint Training Deep Boltzmann Machines for Classification](https://mlsanity.com/paper/1301.3568/) (2013)
- [Deep Restricted Boltzmann Networks](https://mlsanity.com/paper/1611.07917/) (2016)

## Built on by (most cited first, 24 of 24)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [CMOS + stochastic nanomagnets: heterogeneous computers for probabilistic inference and learning](https://mlsanity.com/paper/2304.05949/) (2023)
- [An efficient probabilistic hardware architecture for diffusion-like models](https://mlsanity.com/paper/2510.23972/) (2025)
- [Local Learning Rules for Out-of-Equilibrium Physical Generative Models](https://mlsanity.com/paper/2506.19136/) (2025)
- [Training Multi-layer Neural Networks on Ising Machine](https://mlsanity.com/paper/2311.03408/) (2023)
- [Learning at the Speed of Physics: Equilibrium Propagation on Oscillator Ising Machines](https://mlsanity.com/paper/2510.12934/) (2025)
- [Formalized Hopfield Networks and Boltzmann Machines](https://mlsanity.com/paper/2512.07766/) (2025)
- [Generative Models on Analog Hardware with Dynamics](https://mlsanity.com/paper/2606.27294/) (2026)
- [A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing](https://mlsanity.com/paper/2607.16183/) (2026)
- [Machine Learning Quantum Systems with Magnetic p-bits](https://mlsanity.com/paper/2310.06679/) (2023)
- [Probabilistic Quantum SVM Training on Ising Machine](https://mlsanity.com/paper/2503.16363/) (2025)
- [Nonlocal Monte Carlo via Reinforcement Learning](https://mlsanity.com/paper/2508.10520/) (2025)
- [Probabilistic Computers for Neural Quantum States](https://mlsanity.com/paper/2512.24558/) (2025)
- [From Independent to Correlated Diffusion: Generalized Generative Modeling with Probabilistic Computers](https://mlsanity.com/paper/2603.27996/) (2026)
- [Improved Sparse Ising Optimization](https://mlsanity.com/paper/2311.09275/) (2023)
- [Mean-Field Assisted Deep Boltzmann Learning with Probabilistic Computers](https://mlsanity.com/paper/2401.01996/) (2024)
- [Bayesian Reasoning Enabled by Spin-Orbit Torque Magnetic Tunnel Junctions](https://mlsanity.com/paper/2504.08257/) (2025)
- [Efficient Optimization Accelerator Framework for Multistate Ising Problems](https://mlsanity.com/paper/2505.20250/) (2025)
- [Quantum Boltzmann Machines using Parallel Annealing for Medical Image Classification](https://mlsanity.com/paper/2507.14116/) (2025)
- [Photonic restricted Boltzmann machine for content generation tasks](https://mlsanity.com/paper/2508.20472/) (2025)
- [Quadratic Unconstrained Binary Optimisation for Training and Regularisation of Binary Neural Networks](https://mlsanity.com/paper/2601.00449/) (2026)
- [Optimizing Energy-based Neural Network Training with Coherent Ising Machine](https://mlsanity.com/paper/2606.09117/) (2026)
- [Scaling Up Thermodynamic AI Models](https://mlsanity.com/paper/2607.00170/) (2026)

## Abstract (arXiv)

The slowing down of Moore's law has driven the development of unconventional computing paradigms, such as specialized Ising machines tailored to solve combinatorial optimization problems. In this paper, we show a new application domain for probabilistic bit (p-bit) based Ising machines by training deep generative AI models with them. Using sparse, asynchronous, and massively parallel Ising machines we train deep Boltzmann networks in a hybrid probabilistic-classical computing setup. We use the full MNIST and Fashion MNIST (FMNIST) dataset without any downsampling and a reduced version of CIFAR-10 dataset in hardware-aware network topologies implemented in moderately sized Field Programmable Gate Arrays (FPGA). For MNIST, our machine using only 4,264 nodes (p-bits) and about 30,000 parameters achieves the same classification accuracy (90%) as an optimized software-based restricted Boltzmann Machine (RBM) with approximately 3.25 million parameters. Similar results follow for FMNIST and CIFAR-10. Additionally, the sparse deep Boltzmann network can generate new handwritten digits and fashion products, a task the 3.25 million parameter RBM fails at despite achieving the same accuracy. Our hybrid computer takes a measured 50 to 64 billion probabilistic flips per second, which is at least an order of magnitude faster than superficially similar Graphics and Tensor Processing Unit (GPU/TPU) based implementations. The massively parallel architecture can comfortably perform the contrastive divergence algorithm (CD-n) with up to n = 10 million sweeps per update, beyond the capabilities of existing software implementations. These results demonstrate the potential of using Ising machines for traditionally hard-to-train deep generative Boltzmann networks, with further possible improvement in nanodevice-based realizations.