# Learning stochastic dynamics and predicting emergent behavior using transformers

arXiv: https://arxiv.org/abs/2202.08708 · HTML: https://mlsanity.com/paper/2202.08708/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 64 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Variational approach for learning Markov processes from time series data](https://mlsanity.com/paper/1707.04659/) (2017)
- [Cellular automata as convolutional neural networks](https://mlsanity.com/paper/1809.02942/) (2018)
- [Physics-Guided Deep Learning for Dynamical Systems: A Survey](https://mlsanity.com/paper/2107.01272/) (2021)
- [Learning Graph Cellular Automata](https://mlsanity.com/paper/2110.14237/) (2021)
- [A reinforcement learning approach to rare trajectory sampling](https://mlsanity.com/paper/2005.12890/) (2020)
- [Correspondence between neuroevolution and gradient descent](https://mlsanity.com/paper/2008.06643/) (2020)
- [Maximum likelihood estimation of potential energy in interacting particle systems from single-trajectory data](https://mlsanity.com/paper/2007.11048/) (2020)
- [Dynamical large deviations of two-dimensional kinetically constrained models using a neural-network state ansatz](https://mlsanity.com/paper/2011.08657/) (2020)
- [Evolutionary reinforcement learning of dynamical large deviations](https://mlsanity.com/paper/1909.00835/) (2019)
- [Learning deterministic hydrodynamic equations from stochastic active particle dynamics](https://mlsanity.com/paper/2201.08623/) (2022)

## Built on by (most cited first, 5 of 5)

- [Generative methods for sampling transition paths in molecular dynamics](https://mlsanity.com/paper/2205.02818/) (2022)
- [Learning noise-induced transitions by multi-scaling reservoir computing](https://mlsanity.com/paper/2309.05413/) (2023)
- [Learning Individual Behavior in Agent-Based Models with Graph Diffusion Networks](https://mlsanity.com/paper/2505.21426/) (2025)
- [Neural Dynamic Data Valuation via Stochastic State-Adjoint Trajectories](https://mlsanity.com/paper/2404.19557/) (2024)
- [Learning Chaos Without Seeing Chaos: Extrapolation of Global Dynamics in Autoregressive Transformers](https://mlsanity.com/paper/2609.38814/) (2026)

## Abstract (arXiv)

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.