# Impacts of Darwinian Evolution on Pre-trained Deep Neural Networks

arXiv: https://arxiv.org/abs/2408.05563 · HTML: https://mlsanity.com/paper/2408.05563/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 10 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Evolving Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1710.10741/) (2017)
- [Knowledge Fusion By Evolving Weights of Language Models](https://mlsanity.com/paper/2406.12208/) (2024)
- [CADE: Cosine Annealing Differential Evolution for Spiking Neural Network](https://mlsanity.com/paper/2406.02349/) (2024)
- [A Tunnel Gaussian Process Model for Learning Interpretable Flight's Landing Parameters](https://mlsanity.com/paper/2011.09335/) (2020)

## Built on by (most cited first, 3 of 3)

- [Parameter Competition Balancing for Model Merging](https://mlsanity.com/paper/2410.02396/) (2024)
- [Knowledge Fusion of Large Language Models Via Modular SkillPacks](https://mlsanity.com/paper/2505.18502/) (2025)
- [Dynamic Model Merging Made Slim](https://mlsanity.com/paper/2605.18904/) (2026)

## Abstract (arXiv)

Darwinian evolution of the biological brain is documented through multiple lines of evidence, although the modes of evolutionary changes remain unclear. Drawing inspiration from the evolved neural systems (e.g., visual cortex), deep learning models have demonstrated superior performance in visual tasks, among others. While the success of training deep neural networks has been relying on back-propagation (BP) and its variants to learn representations from data, BP does not incorporate the evolutionary processes that govern biological neural systems. This work proposes a neural network optimization framework based on evolutionary theory. Specifically, BP-trained deep neural networks for visual recognition tasks obtained from the ending epochs are considered the primordial ancestors (initial population). Subsequently, the population evolved with differential evolution. Extensive experiments are carried out to examine the relationships between Darwinian evolution and neural network optimization, including the correspondence between datasets, environment, models, and living species. The empirical results show that the proposed framework has positive impacts on the network, with reduced over-fitting and an order of magnitude lower time complexity compared to BP. Moreover, the experiments show that the proposed framework performs well on deep neural networks and big datasets.