# You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimization

arXiv: https://arxiv.org/abs/1811.01567 · HTML: https://mlsanity.com/paper/1811.01567/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2018.

## Built on by (most cited first, 23 of 23)

- [Federated Class-Incremental Learning](https://mlsanity.com/paper/2203.11473/) (2022)
- [A Comprehensive Survey on Hardware-Aware Neural Architecture Search](https://mlsanity.com/paper/2101.09336/) (2021)
- [Neural Architecture Search for Spiking Neural Networks](https://mlsanity.com/paper/2201.10355/) (2022)
- [AutoMTL: A Programming Framework for Automating Efficient Multi-Task Learning](https://mlsanity.com/paper/2110.13076/) (2021)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices](https://mlsanity.com/paper/2303.09730/) (2023)
- [Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgetting](https://mlsanity.com/paper/2501.15356/) (2025)
- [GP-NAS-ensemble: a model for NAS Performance Prediction](https://mlsanity.com/paper/2301.09231/) (2023)
- [Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising](https://mlsanity.com/paper/2306.08836/) (2023)
- [Enhanced Gradient for Differentiable Architecture Search](https://mlsanity.com/paper/2103.12529/) (2021)
- [Neural Architecture Search via Bregman Iterations](https://mlsanity.com/paper/2106.02479/) (2021)
- [Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks](https://mlsanity.com/paper/2305.14065/) (2023)
- [einspace: Searching for Neural Architectures from Fundamental Operations](https://mlsanity.com/paper/2405.20838/) (2024)
- [Pareto-aware Neural Architecture Generation for Diverse Computational Budgets](https://mlsanity.com/paper/2210.07634/) (2022)
- [Integrating Large Circular Kernels into CNNs through Neural Architecture Search](https://mlsanity.com/paper/2107.02451/) (2021)
- [LISSNAS: Locality-based Iterative Search Space Shrinkage for Neural Architecture Search](https://mlsanity.com/paper/2307.03110/) (2023)
- [Multiple Population Alternate Evolution Neural Architecture Search](https://mlsanity.com/paper/2403.07035/) (2024)
- [Fruit Classification System with Deep Learning and Neural Architecture Search](https://mlsanity.com/paper/2406.01869/) (2024)
- [Meta knowledge assisted Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2504.21545/) (2025)
- [Exploring Complicated Search Spaces with Interleaving-Free Sampling](https://mlsanity.com/paper/2112.02488/) (2021)
- [Multi-trial Neural Architecture Search with Lottery Tickets](https://mlsanity.com/paper/2203.04300/) (2022)
- [PredNAS: A Universal and Sample Efficient Neural Architecture Search Framework](https://mlsanity.com/paper/2210.14460/) (2022)
- [CrossNAS: A Cross-Layer Neural Architecture Search Framework for PIM Systems](https://mlsanity.com/paper/2505.22868/) (2025)

## Abstract (arXiv)

Recently Neural Architecture Search (NAS) has aroused great interest in both academia and industry, however it remains challenging because of its huge and non-continuous search space. Instead of applying evolutionary algorithm or reinforcement learning as previous works, this paper proposes a Direct Sparse Optimization NAS (DSO-NAS) method. In DSO-NAS, we provide a novel model pruning view to NAS problem. In specific, we start from a completely connected block, and then introduce scaling factors to scale the information flow between operations. Next, we impose sparse regularizations to prune useless connections in the architecture. Lastly, we derive an efficient and theoretically sound optimization method to solve it. Our method enjoys both advantages of differentiability and efficiency, therefore can be directly applied to large datasets like ImageNet. Particularly, On CIFAR-10 dataset, DSO-NAS achieves an average test error 2.84%, while on the ImageNet dataset DSO-NAS achieves 25.4% test error under 600M FLOPs with 8 GPUs in 18 hours.