# Trends in Neural Architecture Search: Towards the Acceleration of Search

arXiv: https://arxiv.org/abs/2108.08474 · HTML: https://mlsanity.com/paper/2108.08474/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [SMASH: One-Shot Model Architecture Search through HyperNetworks](https://mlsanity.com/paper/1708.05344/) (2017)
- [Graph HyperNetworks for Neural Architecture Search](https://mlsanity.com/paper/1810.05749/) (2018)
- [Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications](https://mlsanity.com/paper/2008.02608/) (2020)
- [Auction-based Charging Scheduling with Deep Learning Framework for Multi-Drone Networks](https://mlsanity.com/paper/2001.03611/) (2020)

## Built on by (most cited first, 3 of 3)

- [Two-stage architectural fine-tuning with neural architecture search using early-stopping in image classification](https://mlsanity.com/paper/2202.08604/) (2022)
- [Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification](https://mlsanity.com/paper/2206.02098/) (2022)
- [Quantum Circuit Structure Optimization for Quantum Reinforcement Learning](https://mlsanity.com/paper/2507.00589/) (2025)

## Abstract (arXiv)

In modern deep learning research, finding optimal (or near optimal) neural network models is one of major research directions and it is widely studied in many applications. In this paper, the main research trends of neural architecture search (NAS) are classified as neuro-evolutionary algorithms, reinforcement learning based algorithms, and one-shot architecture search approaches. Furthermore, each research trend is introduced and finally all the major three trends are compared. Lastly, the future research directions of NAS research trends are discussed.