# AutomataGPT: Forecasting and Ruleset Inference for Two-Dimensional Cellular Automata

arXiv: https://arxiv.org/abs/2506.17333 · HTML: https://mlsanity.com/paper/2506.17333/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 66 references that are in the map.
- First posted on arXiv 19 Jun 2025.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means](https://mlsanity.com/paper/1904.04017/) (2019)
- [Lenia - Biology of Artificial Life](https://mlsanity.com/paper/1812.05433/) (2018)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities](https://mlsanity.com/paper/2310.10445/) (2023)
- [Attention-based Neural Cellular Automata](https://mlsanity.com/paper/2211.01233/) (2022)
- [Automating the Search for Artificial Life with Foundation Models](https://mlsanity.com/paper/2412.17799/) (2024)
- [PRefLexOR: Preference-based Recursive Language Modeling for Exploratory Optimization of Reasoning and Agentic Thinking](https://mlsanity.com/paper/2410.12375/) (2024)
- [It's Hard for Neural Networks To Learn the Game of Life](https://mlsanity.com/paper/2009.01398/) (2020)
- [Toward Artificial Open-Ended Evolution within Lenia using Quality-Diversity](https://mlsanity.com/paper/2406.04235/) (2024)
- [In-situ graph reasoning and knowledge expansion using Graph-PReFLexOR](https://mlsanity.com/paper/2501.08120/) (2025)
- [Learning spatio-temporal patterns with Neural Cellular Automata](https://mlsanity.com/paper/2310.14809/) (2023)
- [LifeGPT: Topology-Agnostic Generative Pretrained Transformer Model for Cellular Automata](https://mlsanity.com/paper/2409.12182/) (2024)
- [METAGENE-1: Metagenomic Foundation Model for Pandemic Monitoring](https://mlsanity.com/paper/2501.02045/) (2025)
- [STC-ViT: Spatio Temporal Continuous Vision Transformer for Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.17966/) (2024)
- [Learning Elementary Cellular Automata with Transformers](https://mlsanity.com/paper/2412.01417/) (2024)

## Built on by (most cited first, 3 of 3)

- [DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively](https://mlsanity.com/paper/2509.26603/) (2025)
- [Why Do Conventional World Models Fail to Learn Cellular Automata?](https://mlsanity.com/paper/2609.39604/) (2026)
- [Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata](https://mlsanity.com/paper/2610.09124/) (2026)

## Abstract (arXiv)

Cellular automata (CA) provide a minimal formalism for investigating how simple local interactions generate rich spatiotemporal behavior in domains as diverse as traffic flow, ecology, tissue morphogenesis and crystal growth. However, automatically discovering the local update rules for a given phenomenon and using them for quantitative prediction remains challenging. Here we present AutomataGPT, a decoder-only transformer pretrained on around 1 million simulated trajectories that span 100 distinct two-dimensional binary deterministic CA rules on toroidal grids. When evaluated on previously unseen rules drawn from the same CA family, AutomataGPT attains 98.5% perfect one-step forecasts and reconstructs the governing update rule with up to 96% functional (application) accuracy and 82% exact rule-matrix match. These results demonstrate that large-scale pretraining over wider regions of rule space yields substantial generalization in both the forward (state forecasting) and inverse (rule inference) problems, without hand-crafted priors. By showing that transformer models can faithfully infer and execute CA dynamics from data alone, our work lays the groundwork for abstracting real-world dynamical phenomena into data-efficient CA surrogates, opening avenues in biology, tissue engineering, physics and AI-driven scientific discovery.