# Simple And Efficient Architecture Search for Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1711.04528 · HTML: https://mlsanity.com/paper/1711.04528/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Nov 2017.

## Built on by (most cited first, 30 of 40)

- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [AutoFlow: Automated Workflow Generation for Large Language Model Agents](https://mlsanity.com/paper/2407.12821/) (2024)
- [AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench](https://mlsanity.com/paper/2507.02554/) (2025)
- [GradMax: Growing Neural Networks using Gradient Information](https://mlsanity.com/paper/2201.05125/) (2022)
- [Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks](https://mlsanity.com/paper/2102.08574/) (2021)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [The Elastic Lottery Ticket Hypothesis](https://mlsanity.com/paper/2103.16547/) (2021)
- [Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization](https://mlsanity.com/paper/2105.01015/) (2021)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [BM-NAS: Bilevel Multimodal Neural Architecture Search](https://mlsanity.com/paper/2104.09379/) (2021)
- [Neural Architecture Search for Dense Prediction Tasks in Computer Vision](https://mlsanity.com/paper/2202.07242/) (2022)
- [A Survey on Computationally Efficient Neural Architecture Search](https://mlsanity.com/paper/2206.01520/) (2022)
- [AIRA_2: Overcoming Bottlenecks in AI Research Agents](https://mlsanity.com/paper/2603.26499/) (2026)
- [iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization](https://mlsanity.com/paper/2108.11014/) (2021)
- [AutoLossGen: Automatic Loss Function Generation for Recommender Systems](https://mlsanity.com/paper/2204.13160/) (2022)
- [Equivariance-aware Architectural Optimization of Neural Networks](https://mlsanity.com/paper/2210.05484/) (2022)
- [Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation](https://mlsanity.com/paper/2306.12700/) (2023)
- [IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Experts](https://mlsanity.com/paper/2502.18530/) (2025)
- [SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science](https://mlsanity.com/paper/2503.23314/) (2025)
- [Contrastive Self-supervised Neural Architecture Search](https://mlsanity.com/paper/2102.10557/) (2021)
- [Manas: Mining Software Repositories to Assist AutoML](https://mlsanity.com/paper/2112.03395/) (2021)
- [Input Invex Neural Network](https://mlsanity.com/paper/2106.08748/) (2021)
- [Multi-headed Neural Ensemble Search](https://mlsanity.com/paper/2107.04369/) (2021)
- [CONetV2: Efficient Auto-Channel Size Optimization for CNNs](https://mlsanity.com/paper/2110.06830/) (2021)
- [Distilling the Undistillable: Learning from a Nasty Teacher](https://mlsanity.com/paper/2210.11728/) (2022)
- [Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks](https://mlsanity.com/paper/2304.10749/) (2023)
- [A Novel Framework for Neural Architecture Search in the Hill Climbing Domain](https://mlsanity.com/paper/2102.12985/) (2021)
- [Noisy Heuristics NAS: A Network Morphism based Neural Architecture Search using Heuristics](https://mlsanity.com/paper/2207.04467/) (2022)
- [Meta-Learning of NAS for Few-shot Learning in Medical Image Applications](https://mlsanity.com/paper/2203.08951/) (2022)
- [Brain-inspired Evolutionary Architectures for Spiking Neural Networks](https://mlsanity.com/paper/2309.05263/) (2023)

## Abstract (arXiv)

Neural networks have recently had a lot of success for many tasks. However, neural network architectures that perform well are still typically designed manually by experts in a cumbersome trial-and-error process. We propose a new method to automatically search for well-performing CNN architectures based on a simple hill climbing procedure whose operators apply network morphisms, followed by short optimization runs by cosine annealing. Surprisingly, this simple method yields competitive results, despite only requiring resources in the same order of magnitude as training a single network. E.g., on CIFAR-10, our method designs and trains networks with an error rate below 6% in only 12 hours on a single GPU; training for one day reduces this error further, to almost 5%.