# Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning

arXiv: https://arxiv.org/abs/2406.09787 · HTML: https://mlsanity.com/paper/2406.09787/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 14 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [A Survey on Deep Graph Generation: Methods and Applications](https://mlsanity.com/paper/2203.06714/) (2022)
- [Meta-Learning through Hebbian Plasticity in Random Networks](https://mlsanity.com/paper/2007.02686/) (2020)
- [evosax: JAX-based Evolution Strategies](https://mlsanity.com/paper/2212.04180/) (2022)
- [Collective Intelligence for Deep Learning: A Survey of Recent Developments](https://mlsanity.com/paper/2111.14377/) (2021)
- [Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks](https://mlsanity.com/paper/1703.10371/) (2017)
- [HyperNCA: Growing Developmental Networks with Neural Cellular Automata](https://mlsanity.com/paper/2204.11674/) (2022)
- [Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs](https://mlsanity.com/paper/2307.08197/) (2023)
- [Meta-Learning Bidirectional Update Rules](https://mlsanity.com/paper/2104.04657/) (2021)
- [Neural networks grown and self-organized by noise](https://mlsanity.com/paper/1906.01039/) (2019)
- [Structurally Flexible Neural Networks: Evolving the Building Blocks for General Agents](https://mlsanity.com/paper/2404.15193/) (2024)
- [Neural Circuit Architectural Priors for Embodied Control](https://mlsanity.com/paper/2201.05242/) (2022)
- [Self-building Neural Networks](https://mlsanity.com/paper/2304.01086/) (2023)

## Built on by (most cited first, 4 of 4)

- [Hypernetworks That Evolve Themselves](https://mlsanity.com/paper/2512.16406/) (2025)
- [Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity](https://mlsanity.com/paper/2512.12713/) (2025)
- [Distilling a Modular Reservoir Through a Genomic Bottleneck](https://mlsanity.com/paper/2606.28380/) (2026)
- [Architecture Generalization with MetaNCA](https://mlsanity.com/paper/2607.07743/) (2026)

## Abstract (arXiv)

Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of the nervous systems. Artificial neural networks, on the other hand, have been mainly designed as static, fully connected structures that can be notoriously brittle in the face of changing environments and novel inputs. Building on previous works on Neural Developmental Programs (NDPs), we propose a class of self-organizing neural networks capable of synaptic and structural plasticity in an activity and reward-dependent manner which we call Lifelong Neural Developmental Program (LNDP). We present an instance of such a network built on the graph transformer architecture and propose a mechanism for pre-experience plasticity based on the spontaneous activity of sensory neurons. Our results demonstrate the ability of the model to learn from experiences in different control tasks starting from randomly connected or empty networks. We further show that structural plasticity is advantageous in environments necessitating fast adaptation or with non-stationary rewards.