# Evolutionary Neural AutoML for Deep Learning

arXiv: https://arxiv.org/abs/1902.06827 · HTML: https://mlsanity.com/paper/1902.06827/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2019.

## Built on by (most cited first, 20 of 20)

- [Multi-Objective Hyperparameter Optimization in Machine Learning -- An Overview](https://mlsanity.com/paper/2206.07438/) (2022)
- [A survey on multi-objective hyperparameter optimization algorithms for Machine Learning](https://mlsanity.com/paper/2111.13755/) (2021)
- [Self-supervised Cross-silo Federated Neural Architecture Search](https://mlsanity.com/paper/2101.11896/) (2021)
- [Machine Learning for the Detection and Identification of Internet of Things (IoT) Devices: A Survey](https://mlsanity.com/paper/2101.10181/) (2021)
- [An AutoML-based Approach to Multimodal Image Sentiment Analysis](https://mlsanity.com/paper/2102.08092/) (2021)
- [AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks](https://mlsanity.com/paper/2109.08958/) (2021)
- [MedMAE: A Self-Supervised Backbone for Medical Imaging Tasks](https://mlsanity.com/paper/2407.14784/) (2024)
- [Evolving GAN Formulations for Higher Quality Image Synthesis](https://mlsanity.com/paper/2102.08578/) (2021)
- [CityGPT: Towards Urban IoT Learning, Analysis and Interaction with Multi-Agent System](https://mlsanity.com/paper/2405.14691/) (2024)
- [Hybrid Self-Attention NEAT: A novel evolutionary approach to improve the NEAT algorithm](https://mlsanity.com/paper/2112.03670/) (2021)
- [FedPop: Federated Population-based Hyperparameter Tuning](https://mlsanity.com/paper/2308.08634/) (2023)
- [Evolutionary Computation for the Design and Enrichment of General-Purpose Artificial Intelligence Systems: Survey and Prospects](https://mlsanity.com/paper/2407.08745/) (2024)
- [Tricks and Plugins to GBM on Images and Sequences](https://mlsanity.com/paper/2203.00761/) (2022)
- [Concurrent Neural Tree and Data Preprocessing AutoML for Image Classification](https://mlsanity.com/paper/2205.13033/) (2022)
- [Asynchronous Evolution of Deep Neural Network Architectures](https://mlsanity.com/paper/2308.04102/) (2023)
- [Towards Automated Machine Learning Research](https://mlsanity.com/paper/2409.05258/) (2024)
- [Efficient Deep Learning Board: Training Feedback Is Not All You Need](https://mlsanity.com/paper/2410.14743/) (2024)
- [The Generalist Brain Module: Module Repetition in Neural Networks in Light of the Minicolumn Hypothesis](https://mlsanity.com/paper/2507.12473/) (2025)
- [Tricks and Plug-ins for Gradient Boosting in Image Classification](https://mlsanity.com/paper/2507.22842/) (2025)
- [MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging](https://mlsanity.com/paper/2511.04016/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) have produced state-of-the-art results in many benchmarks and problem domains. However, the success of DNNs depends on the proper configuration of its architecture and hyperparameters. Such a configuration is difficult and as a result, DNNs are often not used to their full potential. In addition, DNNs in commercial applications often need to satisfy real-world design constraints such as size or number of parameters. To make configuration easier, automatic machine learning (AutoML) systems for deep learning have been developed, focusing mostly on optimization of hyperparameters. This paper takes AutoML a step further. It introduces an evolutionary AutoML framework called LEAF that not only optimizes hyperparameters but also network architectures and the size of the network. LEAF makes use of both state-of-the-art evolutionary algorithms (EAs) and distributed computing frameworks. Experimental results on medical image classification and natural language analysis show that the framework can be used to achieve state-of-the-art performance. In particular, LEAF demonstrates that architecture optimization provides a significant boost over hyperparameter optimization, and that networks can be minimized at the same time with little drop in performance. LEAF therefore forms a foundation for democratizing and improving AI, as well as making AI practical in future applications.