# LifeGPT: Topology-Agnostic Generative Pretrained Transformer Model for Cellular Automata

arXiv: https://arxiv.org/abs/2409.12182 · HTML: https://mlsanity.com/paper/2409.12182/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 78 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game](https://mlsanity.com/paper/2310.18940/) (2023)
- [On the Turing Completeness of Modern Neural Network Architectures](https://mlsanity.com/paper/1901.03429/) (2019)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-inspired Materials](https://mlsanity.com/paper/2309.08788/) (2023)
- [Accelerating Scientific Discovery with Generative Knowledge Extraction, Graph-Based Representation, and Multimodal Intelligent Graph Reasoning](https://mlsanity.com/paper/2403.11996/) (2024)
- [It's Hard for Neural Networks To Learn the Game of Life](https://mlsanity.com/paper/2009.01398/) (2020)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [Generalization over different cellular automata rules learned by a deep feed-forward neural network](https://mlsanity.com/paper/2103.14886/) (2021)
- [STC-ViT: Spatio Temporal Continuous Vision Transformer for Medium-range Global Weather Forecasting](https://mlsanity.com/paper/2402.17966/) (2024)
- [Transformer for Times Series: an Application to the S&P500](https://mlsanity.com/paper/2403.02523/) (2024)

## Built on by (most cited first, 8 of 8)

- [Training Language Models via Neural Cellular Automata](https://mlsanity.com/paper/2603.10055/) (2026)
- [Intelligence at the Edge of Chaos](https://mlsanity.com/paper/2410.02536/) (2024)
- [Learning Elementary Cellular Automata with Transformers](https://mlsanity.com/paper/2412.01417/) (2024)
- [Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling](https://mlsanity.com/paper/2508.16745/) (2025)
- [AutomataGPT: Forecasting and Ruleset Inference for Two-Dimensional Cellular Automata](https://mlsanity.com/paper/2506.17333/) (2025)
- [ZapGPT: Free-form Language Prompting for Simulated Cellular Control](https://mlsanity.com/paper/2509.10660/) (2025)
- [The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoning](https://mlsanity.com/paper/2609.32927/) (2026)
- [Why Do Conventional World Models Fail to Learn Cellular Automata?](https://mlsanity.com/paper/2609.39604/) (2026)

## Abstract (arXiv)

Conway's Game of Life (Life), a well known algorithm within the broader class of cellular automata (CA), exhibits complex emergent dynamics, with extreme sensitivity to initial conditions. Modeling and predicting such intricate behavior without explicit knowledge of the system's underlying topology presents a significant challenge, motivating the development of algorithms that can generalize across various grid configurations and boundary conditions. We develop a decoder-only generative pretrained transformer (GPT) model to solve this problem, showing that our model can simulate Life on a toroidal grid with no prior knowledge on the size of the grid, or its periodic boundary conditions (LifeGPT). LifeGPT is topology-agnostic with respect to its training data and our results show that a GPT model is capable of capturing the deterministic rules of a Turing-complete system with near-perfect accuracy, given sufficiently diverse training data. We also introduce the idea of an `autoregressive autoregressor' to recursively implement Life using LifeGPT. Our results pave the path towards true universal computation within a large language model framework, synthesizing of mathematical analysis with natural language processing, and probing AI systems for situational awareness about the evolution of such algorithms without ever having to compute them. Similar GPTs could potentially solve inverse problems in multicellular self-assembly by extracting CA-compatible rulesets from real-world biological systems to create new predictive models, which would have significant consequences for the fields of bioinspired materials, tissue engineering, and architected materials design.