# Scaling Autoregressive Models for Lattice Thermodynamics

arXiv: https://arxiv.org/abs/2603.14695 · HTML: https://mlsanity.com/paper/2603.14695/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 53 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 11 of 11)

- [Equivariant message passing for the prediction of tensorial properties and molecular spectra](https://mlsanity.com/paper/2102.03150/) (2021)
- [Solving Statistical Mechanics Using Variational Autoregressive Networks](https://mlsanity.com/paper/1809.10606/) (2018)
- [LEAPS: A discrete neural sampler via locally equivariant networks](https://mlsanity.com/paper/2502.10843/) (2025)
- [Proximal Diffusion Neural Sampler](https://mlsanity.com/paper/2510.03824/) (2025)
- [Discrete Neural Flow Samplers with Locally Equivariant Transformer](https://mlsanity.com/paper/2505.17741/) (2025)
- [Flow Matching for Accelerated Simulation of Atomic Transport in Crystalline Materials](https://mlsanity.com/paper/2410.01464/) (2024)
- [Machine-learning-accelerated simulations to enable automatic surface reconstruction](https://mlsanity.com/paper/2305.07251/) (2023)
- [Learning Collective Variables with Synthetic Data Augmentation through Physics-Inspired Geodesic Interpolation](https://mlsanity.com/paper/2402.01542/) (2024)
- [Generative Marginalization Models](https://mlsanity.com/paper/2310.12920/) (2023)
- [Energy-based generator matching: A neural sampler for general state space](https://mlsanity.com/paper/2505.19646/) (2025)
- [Transferable Learning of Reaction Pathways from Geometric Priors](https://mlsanity.com/paper/2504.15370/) (2025)

## Built on by (most cited first, 5 of 5)

- [MetaDNS: Enhancing Exploration in Discrete Neural Samplers via Well-Tempered Metadynamics](https://mlsanity.com/paper/2605.21722/) (2026)
- [FrOGS: Discrete Neural Sampler for Independent Alloy Configurations Across Chemical Conditions](https://mlsanity.com/paper/2609.02948/) (2026)
- [Atomistic Modeling of Chemical Disorder in Materials: Bridging Conventional Methods and AI-Assisted Approaches](https://mlsanity.com/paper/2605.19124/) (2026)
- [Grand Canonical Generators](https://mlsanity.com/paper/2610.00683/) (2026)
- [Fixed-point neural samplers on discrete spaces](https://mlsanity.com/paper/2610.01739/) (2026)

## Abstract (arXiv)

Predicting how materials behave under realistic conditions requires understanding the statistical distribution of atomic configurations on crystal lattices, a problem central to alloy design, catalysis, and the study of phase transitions. Traditional Markov-chain Monte Carlo sampling suffers from slow convergence and critical slowing down near phase transitions, motivating the use of generative models that directly learn the thermodynamic distribution. Existing autoregressive models (ARMs), however, generate configurations in a fixed sequential order and incur high memory and training costs, limiting their applicability to realistic systems. Here, we develop a framework combining any-order ARMs, which generate configurations flexibly by conditioning on any known subset of lattice sites, with marginalization models (MAMs), which approximate the probability of any partial configuration in a single forward pass and substantially reduce memory requirements. This combination enables models trained on smaller lattices to be reused for sampling larger systems, while supporting expressive Transformer architectures with lattice-aware positional encodings at manageable computational cost. We demonstrate that Transformer-based any-order MAMs achieve more accurate free energies than multilayer perceptron-based ARMs on both the two-dimensional Ising model and CuAu alloys, faithfully capturing phase transitions and critical behavior. Overall, our framework scales from 10 × 10 to 20 × 20 Ising systems and from 2 × 2 × 4 to 4 × 4 × 8 CuAu supercells at reduced computational cost compared to conventional sampling methods.