# Deep Learning the Ising Model Near Criticality

arXiv: https://arxiv.org/abs/1708.04622 · HTML: https://mlsanity.com/paper/1708.04622/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2017.

## Built on by (most cited first, 10 of 10)

- [Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines](https://mlsanity.com/paper/2105.13889/) (2021)
- [The R-mAtrIx Net](https://mlsanity.com/paper/2304.07247/) (2023)
- [Fine-tuning Vision Transformers for the Prediction of State Variables in Ising Models](https://mlsanity.com/paper/2109.13925/) (2021)
- [IsingFormer: Augmenting Parallel Tempering With Learned Proposals](https://mlsanity.com/paper/2509.23043/) (2025)
- [A Group-Equivariant Autoencoder for Identifying Spontaneously Broken Symmetries](https://mlsanity.com/paper/2202.06319/) (2022)
- [Mappings for Marginal Probabilities with Applications to Models in Statistical Physics](https://mlsanity.com/paper/2208.05333/) (2022)
- [New Metric Formulas that Include Measurement Errors in Machine Learning for Natural Sciences](https://mlsanity.com/paper/2209.15588/) (2022)
- [Physics-Guided Problem Decomposition for Scaling Deep Learning of High-dimensional Eigen-Solvers: The Case of Schr"odinger's Equation](https://mlsanity.com/paper/2202.05994/) (2022)
- [Attention to Order: Transformers Discover Phase Transitions via Learnability](https://mlsanity.com/paper/2510.07401/) (2025)
- [Learning Degenerate Manifolds of Frustrated Magnets with Boltzmann Machines](https://mlsanity.com/paper/2511.19879/) (2025)

## Abstract (arXiv)

It is well established that neural networks with deep architectures perform better than shallow networks for many tasks in machine learning. In statistical physics, while there has been recent interest in representing physical data with generative modelling, the focus has been on shallow neural networks. A natural question to ask is whether deep neural networks hold any advantage over shallow networks in representing such data. We investigate this question by using unsupervised, generative graphical models to learn the probability distribution of a two-dimensional Ising system. Deep Boltzmann machines, deep belief networks, and deep restricted Boltzmann networks are trained on thermal spin configurations from this system, and compared to the shallow architecture of the restricted Boltzmann machine. We benchmark the models, focussing on the accuracy of generating energetic observables near the phase transition, where these quantities are most difficult to approximate. Interestingly, after training the generative networks, we observe that the accuracy essentially depends only on the number of neurons in the first hidden layer of the network, and not on other model details such as network depth or model type. This is evidence that shallow networks are more efficient than deep networks at representing physical probability distributions associated with Ising systems near criticality.