# Von Neumann Networks

arXiv: https://arxiv.org/abs/2605.05780 · HTML: https://mlsanity.com/paper/2605.05780/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 65 references that are in the map.
- First posted on arXiv 7 May 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation](https://mlsanity.com/paper/1901.02985/) (2019)
- [On the Turing Completeness of Modern Neural Network Architectures](https://mlsanity.com/paper/1901.03429/) (2019)
- [Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data](https://mlsanity.com/paper/2407.04491/) (2024)
- [Training Quantized Nets: A Deeper Understanding](https://mlsanity.com/paper/1706.02379/) (2017)
- [Neural Architecture Transfer](https://mlsanity.com/paper/2005.05859/) (2020)
- [Cellular automata as convolutional neural networks](https://mlsanity.com/paper/1809.02942/) (2018)
- [Lenia - Biology of Artificial Life](https://mlsanity.com/paper/1812.05433/) (2018)
- [Convolutional Neural Fabrics](https://mlsanity.com/paper/1606.02492/) (2016)
- [Lenia and Expanded Universe](https://mlsanity.com/paper/2005.03742/) (2020)
- [μNCA: Texture Generation with Ultra-Compact Neural Cellular Automata](https://mlsanity.com/paper/2111.13545/) (2021)
- [Mesh Neural Cellular Automata](https://mlsanity.com/paper/2311.02820/) (2023)
- [Differentiable Logic Cellular Automata: From Game of Life to Pattern Generation](https://mlsanity.com/paper/2506.04912/) (2025)
- [Neural Green's Functions](https://mlsanity.com/paper/2511.01924/) (2025)

## Abstract (arXiv)

In the mid-twentieth century, mathematician and polymath John von Neumann created a computational system on an array of cells as a simple model of the human brain, where each cell had one of a finite set of roles or states that he predicted would be modelled by a diffusion process. In this work, we show that such a system, when developed in a modern deep learning setting, enables the construction of an artificial neuron having specialized roles that can be learnt. We refer to this neuron as the Von Neumann neuron, and the resulting neural network from such neurons result in a self-engineered design whose architecture is only dependent on the structure and locations of its inputs and outputs on this cellular array. The mathematical framework for these Von Neumann Networks (VNNs) is also constructed and shows that they are based on the extension of neural operators and the learning of Green's functions with convolutions on a cellular topology having a diffusion signature. We also prove that these VNNs are part of a more general computational system called Cellular Machines that are computationally universal. Initial experiments show that VNN based multi-layered perceptrons outperform their equivalent deep learning variant on basic tasks, while being more parameter efficient and are capable of learning new types of tasks. This includes the ability to solve for and construct an extension of the Von Neumann (hardware) architecture common to all modern computers to cells and suggests new opportunities that could be explored.