# Evolving Deep Neural Networks

arXiv: https://arxiv.org/abs/1703.00548 · HTML: https://mlsanity.com/paper/1703.00548/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Mar 2017.

## Built on by (most cited first, 30 of 95)

- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment](https://mlsanity.com/paper/2103.16196/) (2021)
- [Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search](https://mlsanity.com/paper/2101.11342/) (2021)
- [Generalizing Few-Shot NAS with Gradient Matching](https://mlsanity.com/paper/2203.15207/) (2022)
- [SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models](https://mlsanity.com/paper/2411.02433/) (2024)
- [SoK: Dataset Copyright Auditing in Machine Learning Systems](https://mlsanity.com/paper/2410.16618/) (2024)
- [Computer Vision and Deep Learning for Fish Classification in Underwater Habitats: A Survey](https://mlsanity.com/paper/2203.06951/) (2022)
- [GI-NAS: Boosting Gradient Inversion Attacks Through Adaptive Neural Architecture Search](https://mlsanity.com/paper/2405.20725/) (2024)
- [Memory-Efficient Differentiable Transformer Architecture Search](https://mlsanity.com/paper/2105.14669/) (2021)
- [Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks](https://mlsanity.com/paper/2308.06960/) (2023)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [Neural Exploratory Landscape Analysis for Meta-Black-Box-Optimization](https://mlsanity.com/paper/2408.10672/) (2024)
- [RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving](https://mlsanity.com/paper/2108.08019/) (2021)
- [Automated Reinforcement Learning: An Overview](https://mlsanity.com/paper/2201.05000/) (2022)
- [Construction of Hierarchical Neural Architecture Search Spaces based on Context-free Grammars](https://mlsanity.com/paper/2211.01842/) (2022)
- [Dynamic Neural Network for Multi-Task Learning Searching across Diverse Network Topologies](https://mlsanity.com/paper/2303.06856/) (2023)
- [Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally](https://mlsanity.com/paper/2405.19816/) (2024)
- [Ensemble neuroevolution based approach for multivariate time series anomaly detection](https://mlsanity.com/paper/2108.03585/) (2021)
- [Feature Programming for Multivariate Time Series Prediction](https://mlsanity.com/paper/2306.06252/) (2023)
- [Neural Architecture Search via Bregman Iterations](https://mlsanity.com/paper/2106.02479/) (2021)
- [AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks](https://mlsanity.com/paper/2109.08958/) (2021)
- [einspace: Searching for Neural Architectures from Fundamental Operations](https://mlsanity.com/paper/2405.20838/) (2024)
- [Visionary: Vision architecture discovery for robot learning](https://mlsanity.com/paper/2103.14633/) (2021)
- [MONCAE: Multi-Objective Neuroevolution of Convolutional Autoencoders](https://mlsanity.com/paper/2106.11914/) (2021)
- [CONetV2: Efficient Auto-Channel Size Optimization for CNNs](https://mlsanity.com/paper/2110.06830/) (2021)
- [Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems](https://mlsanity.com/paper/2111.01584/) (2021)
- [Evolving Neural Selection with Adaptive Regularization](https://mlsanity.com/paper/2204.01662/) (2022)
- [Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness](https://mlsanity.com/paper/2302.10253/) (2023)
- [Directed Structural Adaptation to Overcome Statistical Conflicts and Enable Continual Learning](https://mlsanity.com/paper/2412.04190/) (2024)

## Abstract (arXiv)

The success of deep learning depends on finding an architecture to fit the task. As deep learning has scaled up to more challenging tasks, the architectures have become difficult to design by hand. This paper proposes an automated method, CoDeepNEAT, for optimizing deep learning architectures through evolution. By extending existing neuroevolution methods to topology, components, and hyperparameters, this method achieves results comparable to best human designs in standard benchmarks in object recognition and language modeling. It also supports building a real-world application of automated image captioning on a magazine website. Given the anticipated increases in available computing power, evolution of deep networks is promising approach to constructing deep learning applications in the future.