# Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimization

arXiv: https://arxiv.org/abs/2403.17012 · HTML: https://mlsanity.com/paper/2403.17012/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 11 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search](https://mlsanity.com/paper/1911.12126/) (2019)
- [DARTS+: Improved Differentiable Architecture Search with Early Stopping](https://mlsanity.com/paper/1909.06035/) (2019)
- [A Novel Training Protocol for Performance Predictors of Evolutionary Neural Architecture Search Algorithms](https://mlsanity.com/paper/2008.13187/) (2020)
- [A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities](https://mlsanity.com/paper/2205.10358/) (2022)
- [Searching Toward Pareto-Optimal Device-Aware Neural Architectures](https://mlsanity.com/paper/1808.09830/) (2018)
- [Model-based Reinforcement Learning for Service Mesh Fault Resiliency in a Web Application-level](https://mlsanity.com/paper/2110.13621/) (2021)
- [Sentiment analysis with adaptive multi-head attention in Transformer](https://mlsanity.com/paper/2310.14505/) (2023)
- [Superkernel Neural Architecture Search for Image Denoising](https://mlsanity.com/paper/2004.08870/) (2020)
- [A Novel Framework for Neural Architecture Search in the Hill Climbing Domain](https://mlsanity.com/paper/2102.12985/) (2021)
- [FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation](https://mlsanity.com/paper/2312.00102/) (2023)
- [Joint Detection Algorithm for Multiple Cognitive Users in Spectrum Sensing](https://mlsanity.com/paper/2311.18599/) (2023)
- [Sample-based Dynamic Hierarchical Transformer with Layer and Head Flexibility via Contextual Bandit](https://mlsanity.com/paper/2312.03038/) (2023)
- [Optimizing the Passenger Flow for Airport Security Check](https://mlsanity.com/paper/2312.05259/) (2023)

## Built on by (most cited first, 3 of 3)

- [Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment](https://mlsanity.com/paper/2501.03265/) (2025)
- [Bilevel Optimization for Neural Architecture Search](https://mlsanity.com/paper/2606.29582/) (2026)
- [LP-NAS: Linear Programming-based Neural Architecture Search](https://mlsanity.com/paper/2608.14472/) (2026)

## Abstract (arXiv)

The paper provides a comprehensive overview of Neural Architecture Search (NAS), emphasizing its evolution from manual design to automated, computationally-driven approaches. It covers the inception and growth of NAS, highlighting its application across various domains, including medical imaging and natural language processing. The document details the shift from expert-driven design to algorithm-driven processes, exploring initial methodologies like reinforcement learning and evolutionary algorithms. It also discusses the challenges of computational demands and the emergence of efficient NAS methodologies, such as Differentiable Architecture Search and hardware-aware NAS. The paper further elaborates on NAS's application in computer vision, NLP, and beyond, demonstrating its versatility and potential for optimizing neural network architectures across different tasks. Future directions and challenges, including computational efficiency and the integration with emerging AI domains, are addressed, showcasing NAS's dynamic nature and its continued evolution towards more sophisticated and efficient architecture search methods.