# Generalization over different cellular automata rules learned by a deep feed-forward neural network

arXiv: https://arxiv.org/abs/2103.14886 · HTML: https://mlsanity.com/paper/2103.14886/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Pretrained Transformers as Universal Computation Engines](https://mlsanity.com/paper/2103.05247/) (2021)
- [Cellular automata as convolutional neural networks](https://mlsanity.com/paper/1809.02942/) (2018)
- [Intrinsically Motivated Discovery of Diverse Patterns in Self-Organizing Systems](https://mlsanity.com/paper/1908.06663/) (2019)
- [Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systems](https://mlsanity.com/paper/2007.01195/) (2020)
- [It's Hard for Neural Networks To Learn the Game of Life](https://mlsanity.com/paper/2009.01398/) (2020)
- [Neural Cellular Automata Manifold](https://mlsanity.com/paper/2006.12155/) (2020)

## Built on by (most cited first, 6 of 6)

- [Learning Graph Cellular Automata](https://mlsanity.com/paper/2110.14237/) (2021)
- [LifeGPT: Topology-Agnostic Generative Pretrained Transformer Model for Cellular Automata](https://mlsanity.com/paper/2409.12182/) (2024)
- [Learning Elementary Cellular Automata with Transformers](https://mlsanity.com/paper/2412.01417/) (2024)
- [Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling](https://mlsanity.com/paper/2508.16745/) (2025)
- [Data-Centric Approach to Constrained Machine Learning: A Case Study on Conway's Game of Life](https://mlsanity.com/paper/2408.12778/) (2024)
- [Learning Traffic Flow Dynamics with Stochastic Physics-Informed Neural Cellular Automata](https://mlsanity.com/paper/2610.09946/) (2026)

## Abstract (arXiv)

To test generalization ability of a class of deep neural networks, we randomly generate a large number of different rule sets for 2-D cellular automata (CA), based on John Conway's Game of Life. Using these rules, we compute several trajectories for each CA instance. A deep convolutional encoder-decoder network with short and long range skip connections is trained on various generated CA trajectories to predict the next CA state given its previous states. Results show that the network is able to learn the rules of various, complex cellular automata and generalize to unseen configurations. To some extent, the network shows generalization to rule sets and neighborhood sizes that were not seen during the training at all. Code to reproduce the experiments is publicly available at: https://github.com/SLAMPAI/generalization-cellular-automata