# Large-Scale Evolution of Image Classifiers

arXiv: https://arxiv.org/abs/1703.01041 · HTML: https://mlsanity.com/paper/1703.01041/

## Key facts

- Cited by 348 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2017.

## Built on by (most cited first, 30 of 348)

- [Primer: Searching for Efficient Transformers for Language Modeling](https://mlsanity.com/paper/2109.08668/) (2021)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [Multiview Transformers for Video Recognition](https://mlsanity.com/paper/2201.04288/) (2022)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Neural Prompt Search](https://mlsanity.com/paper/2206.04673/) (2022)
- [EvoPrompting: Language Models for Code-Level Neural Architecture Search](https://mlsanity.com/paper/2302.14838/) (2023)
- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [Deep Model Reassembly](https://mlsanity.com/paper/2210.17409/) (2022)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition](https://mlsanity.com/paper/2102.01063/) (2021)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [Exploring Relational Context for Multi-Task Dense Prediction](https://mlsanity.com/paper/2104.13874/) (2021)
- [GradMax: Growing Neural Networks using Gradient Information](https://mlsanity.com/paper/2201.05125/) (2022)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients](https://mlsanity.com/paper/2301.11300/) (2023)
- [Evolving Reinforcement Learning Algorithms](https://mlsanity.com/paper/2101.03958/) (2021)
- [Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts](https://mlsanity.com/paper/2002.04013/) (2020)
- [AutoSNN: Towards Energy-Efficient Spiking Neural Networks](https://mlsanity.com/paper/2201.12738/) (2022)
- [Automated Machine Learning on Graphs: A Survey](https://mlsanity.com/paper/2103.00742/) (2021)
- [The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems](https://mlsanity.com/paper/2105.03354/) (2021)
- [HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers](https://mlsanity.com/paper/2106.06560/) (2021)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [Pooling Architecture Search for Graph Classification](https://mlsanity.com/paper/2108.10587/) (2021)
- [AutoDropout: Learning Dropout Patterns to Regularize Deep Networks](https://mlsanity.com/paper/2101.01761/) (2021)
- [AdvRush: Searching for Adversarially Robust Neural Architectures](https://mlsanity.com/paper/2108.01289/) (2021)
- [Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search](https://mlsanity.com/paper/2101.11342/) (2021)
- [Rethinking Graph Neural Architecture Search from Message-passing](https://mlsanity.com/paper/2103.14282/) (2021)
- [NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search](https://mlsanity.com/paper/2206.09166/) (2022)

## Abstract (arXiv)

Neural networks have proven effective at solving difficult problems but designing their architectures can be challenging, even for image classification problems alone. Our goal is to minimize human participation, so we employ evolutionary algorithms to discover such networks automatically. Despite significant computational requirements, we show that it is now possible to evolve models with accuracies within the range of those published in the last year. Specifically, we employ simple evolutionary techniques at unprecedented scales to discover models for the CIFAR-10 and CIFAR-100 datasets, starting from trivial initial conditions and reaching accuracies of 94.6% (95.6% for ensemble) and 77.0%, respectively. To do this, we use novel and intuitive mutation operators that navigate large search spaces; we stress that no human participation is required once evolution starts and that the output is a fully-trained model. Throughout this work, we place special emphasis on the repeatability of results, the variability in the outcomes and the computational requirements.