TopoNAS: Boosting Search Efficiency of Gradient-based NAS via Topological Simplification
Not on the map. Why?
Abstract (arXiv)
Improving search efficiency serves as one of the crucial objectives of Neural Architecture Search (NAS). However, many current approaches ignore the universality of the search strategy and fail to reduce the computational redundancy during the search process, especially in one-shot NAS architectures. Besides, current NAS methods show invalid reparameterization in non-linear search space, leading to poor efficiency in common search spaces like DARTS. In this paper, we propose TopoNAS, a model-agnostic approach for gradient-based one-shot NAS that significantly reduces searching time and memory usage by topological simplification of searchable paths. Firstly, we model the non-linearity in search spaces to reveal the parameterization difficulties. To improve the search efficiency, we present a topological simplification method and iteratively apply module-sharing strategies to simplify the topological structure of searchable paths. In addition, a kernel normalization technique is also proposed to preserve the search accuracy. Experimental results on the NASBench201 benchmark with various search spaces demonstrate the effectiveness of our method. It proves the proposed TopoNAS enhances the performance of various architectures in terms of search efficiency while maintaining a high level of accuracy. The project page is available at https://xdedss.github.io/topo_simplification.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 44 of its 55 references that are in the map. First posted on arXiv 2 Aug 2024.
Builds on
44 of 55 references.
References · 44
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
- 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
- 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
- 2017Progressive Neural Architecture Searchcited by 483
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
- 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
32 more
- 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
- 2017Large-Scale Evolution of Image Classifierscited by 348
- 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
- 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
- 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
- 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
- 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
- 2020On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practicecited by 181
- 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
- 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
- 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
- 2019Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hourscited by 87
- 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
- 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
- 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 63
- 2017Evolving Deep Convolutional Neural Networks for Image Classificationcited by 54
- 2017DiracNets: Training Very Deep Neural Networks Without Skip-Connectionscited by 47
- 2019Customizable Architecture Search for Semantic Segmentationcited by 41
- 2018DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architecturescited by 40
- 2019SqueezeNAS: Fast neural architecture search for faster semantic segmentationcited by 31
- 2019AutoDispNet: Improving Disparity Estimation With AutoMLcited by 29
- 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
- 2019SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detectioncited by 25
- 2021OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detectioncited by 19
- 2019Efficient Neural Architecture Transformation Searchin Channel-Level for Object Detectioncited by 16
- 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
- 2021RepNAS: Searching for Efficient Re-parameterizing Blockscited by 11
- 2022ASFD: Automatic and Scalable Face Detectorcited by 9
- 2022EAutoDet: Efficient Architecture Search for Object Detectioncited by 7
- 2023Efficient Evaluation Methods for Neural Architecture Search: A Surveycited by 7
- 2023β-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Searchcited by 7
Built on by
Citing papers per year: 2025: 1.
1 paper.