# Correspondence between neuroevolution and gradient descent

arXiv: https://arxiv.org/abs/2008.06643 · HTML: https://mlsanity.com/paper/2008.06643/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [Learning stochastic dynamics and predicting emergent behavior using transformers](https://mlsanity.com/paper/2202.08708/) (2022)
- [Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems](https://mlsanity.com/paper/2309.05337/) (2023)
- [Training neural networks using Metropolis Monte Carlo and an adaptive variant](https://mlsanity.com/paper/2205.07408/) (2022)
- [Training neural network ensembles via trajectory sampling](https://mlsanity.com/paper/2209.11116/) (2022)
- [Remapping and navigation of an embedding space via error minimization: a fundamental organizational principle of cognition in natural and artificial systems](https://mlsanity.com/paper/2601.14096/) (2026)
- [Minibatch training of neural network ensembles via trajectory sampling](https://mlsanity.com/paper/2306.13442/) (2023)
- [Interlocking-free Selective Rationalization Through Genetic-based Learning](https://mlsanity.com/paper/2412.10312/) (2024)
- [Physics-Informed Evolution: An Evolutionary Framework for Solving Quantum Control Problems Involving the Schr"odinger Equation](https://mlsanity.com/paper/2502.05228/) (2025)
- [Introducing Echo Networks for Computational Neuroevolution](https://mlsanity.com/paper/2604.08204/) (2026)
- [Direct From Darwin: Deriving Advanced Optimizers From Evolutionary First Principles](https://mlsanity.com/paper/2605.05284/) (2026)
- [Distilling a Modular Reservoir Through a Genomic Bottleneck](https://mlsanity.com/paper/2606.28380/) (2026)

## Abstract (arXiv)

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations,for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.