# It's Hard for Neural Networks To Learn the Game of Life

arXiv: https://arxiv.org/abs/2009.01398 · HTML: https://mlsanity.com/paper/2009.01398/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2020.

## Built on by (most cited first, 13 of 13)

- [Illuminating Diverse Neural Cellular Automata for Level Generation](https://mlsanity.com/paper/2109.05489/) (2021)
- [Learning Graph Cellular Automata](https://mlsanity.com/paper/2110.14237/) (2021)
- [LifeGPT: Topology-Agnostic Generative Pretrained Transformer Model for Cellular Automata](https://mlsanity.com/paper/2409.12182/) (2024)
- [Generalization over different cellular automata rules learned by a deep feed-forward neural network](https://mlsanity.com/paper/2103.14886/) (2021)
- [Learning Elementary Cellular Automata with Transformers](https://mlsanity.com/paper/2412.01417/) (2024)
- [Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling](https://mlsanity.com/paper/2508.16745/) (2025)
- [AutomataGPT: Forecasting and Ruleset Inference for Two-Dimensional Cellular Automata](https://mlsanity.com/paper/2506.17333/) (2025)
- [Data-Centric Approach to Constrained Machine Learning: A Case Study on Conway's Game of Life](https://mlsanity.com/paper/2408.12778/) (2024)
- [A Local Optima Network Analysis of the Feedforward Neural Architecture Space](https://mlsanity.com/paper/2206.06903/) (2022)
- [It's Much Easier for Neural Networks to learn Game of Life Dynamics with the Right Activation Function: Polynomial Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2606.23587/) (2026)
- [Learning grammar with a divide-and-concur neural network](https://mlsanity.com/paper/2201.07341/) (2022)
- [The Geometry of Logic: Stratification Induces Semantic Structure and Robust Reasoning](https://mlsanity.com/paper/2609.32927/) (2026)
- [Where Does Randomness Matter in Neural Cellular Automata?](https://mlsanity.com/paper/2609.36797/) (2026)

## Abstract (arXiv)

Efforts to improve the learning abilities of neural networks have focused mostly on the role of optimization methods rather than on weight initializations. Recent findings, however, suggest that neural networks rely on lucky random initial weights of subnetworks called "lottery tickets" that converge quickly to a solution. To investigate how weight initializations affect performance, we examine small convolutional networks that are trained to predict n steps of the two-dimensional cellular automaton Conway's Game of Life, the update rules of which can be implemented efficiently in a 2n+1 layer convolutional network. We find that networks of this architecture trained on this task rarely converge. Rather, networks require substantially more parameters to consistently converge. In addition, near-minimal architectures are sensitive to tiny changes in parameters: changing the sign of a single weight can cause the network to fail to learn. Finally, we observe a critical value d_0 such that training minimal networks with examples in which cells are alive with probability d_0 dramatically increases the chance of convergence to a solution. We conclude that training convolutional neural networks to learn the input/output function represented by n steps of Game of Life exhibits many characteristics predicted by the lottery ticket hypothesis, namely, that the size of the networks required to learn this function are often significantly larger than the minimal network required to implement the function.