# The autoregressive neural network architecture of the Boltzmann distribution of pairwise interacting spins systems

arXiv: https://arxiv.org/abs/2302.08347 · HTML: https://mlsanity.com/paper/2302.08347/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 64 references that are in the map.
- First posted on arXiv 16 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [Statistical physics of inference: Thresholds and algorithms](https://mlsanity.com/paper/1511.02476/) (2015)
- [Solving Statistical Mechanics Using Variational Autoregressive Networks](https://mlsanity.com/paper/1809.10606/) (2018)
- [Deep autoregressive models for the efficient variational simulation of many-body quantum systems](https://mlsanity.com/paper/1902.04057/) (2019)
- [Deep AutoRegressive Networks](https://mlsanity.com/paper/1310.8499/) (2013)
- [Variational Neural Annealing](https://mlsanity.com/paper/2101.10154/) (2021)
- [Inverse statistical problems: from the inverse Ising problem to data science](https://mlsanity.com/paper/1702.01522/) (2017)
- [Attention-based Quantum Tomography](https://mlsanity.com/paper/2006.12469/) (2020)
- [Autoregressive Energy Machines](https://mlsanity.com/paper/1904.05626/) (2019)
- [Autoregressive neural-network wavefunctions for ab initio quantum chemistry](https://mlsanity.com/paper/2109.12606/) (2021)
- [Unbiased Monte Carlo Cluster Updates with Autoregressive Neural Networks](https://mlsanity.com/paper/2105.05650/) (2021)
- [A Bayesian generative neural network framework for epidemic inference problems](https://mlsanity.com/paper/2111.03383/) (2021)

## Built on by (most cited first, 8 of 8)

- [Multilevel Generative Samplers for Investigating Critical Phenomena](https://mlsanity.com/paper/2503.08918/) (2025)
- [Performance of machine-learning-assisted Monte Carlo in sampling from simple statistical physics models](https://mlsanity.com/paper/2505.22598/) (2025)
- [Hierarchical autoregressive neural networks in three-dimensional statistical system](https://mlsanity.com/paper/2503.08610/) (2025)
- [Sampling two-dimensional spin systems with transformers](https://mlsanity.com/paper/2604.27738/) (2026)
- [Nonlocal Monte Carlo via Reinforcement Learning](https://mlsanity.com/paper/2508.10520/) (2025)
- [From Independent to Correlated Diffusion: Generalized Generative Modeling with Probabilistic Computers](https://mlsanity.com/paper/2603.27996/) (2026)
- [Variational Autoregressive Networks with probability priors](https://mlsanity.com/paper/2605.16020/) (2026)
- [Fixed-point neural samplers on discrete spaces](https://mlsanity.com/paper/2610.01739/) (2026)

## Abstract (arXiv)

Generative Autoregressive Neural Networks (ARNNs) have recently demonstrated exceptional results in image and language generation tasks, contributing to the growing popularity of generative models in both scientific and commercial applications. This work presents an exact mapping of the Boltzmann distribution of binary pairwise interacting systems into autoregressive form. The resulting ARNN architecture has weights and biases of its first layer corresponding to the Hamiltonian's couplings and external fields, featuring widely used structures such as the residual connections and a recurrent architecture with clear physical meanings. Moreover, its architecture's explicit formulation enables the use of statistical physics techniques to derive new ARNNs for specific systems. As examples, new effective ARNN architectures are derived from two well-known mean-field systems, the Curie-Weiss and Sherrington-Kirkpatrick models, showing superior performance in approximating the Boltzmann distributions of the corresponding physics model compared to other commonly used architectures. The connection established between the physics of the system and the neural network architecture provides a means to derive new architectures for different interacting systems and interpret existing ones from a physical perspective.