# Constructing Organism Networks from Collaborative Self-Replicators

arXiv: https://arxiv.org/abs/2212.10078 · HTML: https://mlsanity.com/paper/2212.10078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 12 of 12)

- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Using Fast Weights to Attend to the Recent Past](https://mlsanity.com/paper/1610.06258/) (2016)
- [Optimizing Reusable Knowledge for Continual Learning via Metalearning](https://mlsanity.com/paper/2106.05390/) (2021)
- [Learning from Randomly Initialized Neural Network Features](https://mlsanity.com/paper/2202.06438/) (2022)
- [Continual Learning with Deep Artificial Neurons](https://mlsanity.com/paper/2011.07035/) (2020)
- [Neural Network Quine](https://mlsanity.com/paper/1803.05859/) (2018)

## Abstract (arXiv)

We introduce organism networks, which function like a single neural network but are composed of several neural particle networks; while each particle network fulfils the role of a single weight application within the organism network, it is also trained to self-replicate its own weights. As organism networks feature vastly more parameters than simpler architectures, we perform our initial experiments on an arithmetic task as well as on simplified MNIST-dataset classification as a collective. We observe that individual particle networks tend to specialise in either of the tasks and that the ones fully specialised in the secondary task may be dropped from the network without hindering the computational accuracy of the primary task. This leads to the discovery of a novel pruning-strategy for sparse neural networks