mlsanityData to 14 Oct 2026

Half Search Space is All You Need

arXiv 2505.13586 ↗First posted 19 May 2025cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural Architecture Search (NAS) is a powerful tool for automating architecture design. One-Shot NAS techniques, such as DARTS, have gained substantial popularity due to their combination of search efficiency with simplicity of implementation. By design, One-Shot methods have high GPU memory requirements during the search. To mitigate this issue, we propose to prune the search space in an efficient automatic manner to reduce memory consumption and search time while preserving the search accuracy. Specifically, we utilise Zero-Shot NAS to efficiently remove low-performing architectures from the search space before applying One-Shot NAS to the pruned search space. Experimental results on the DARTS search space show that our approach reduces memory consumption by 81% compared to the baseline One-Shot setup while achieving the same level of accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

23 of 44 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  3. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  4. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  5. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  6. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  7. 2020Neural Architecture Search without Trainingcited by 244
  8. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  9. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  10. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  11. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  12. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
11 more
  1. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  2. 2020Surrogate NAS Benchmarks: Going Beyond the Limited Search Spaces of Tabular NAS Benchmarkscited by 67
  3. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  4. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
  5. 2020Evaluating Efficient Performance Estimators of Neural Architecturescited by 55
  6. 2021KNAS: Green Neural Architecture Searchcited by 33
  7. 2020Towards NNGP-guided Neural Architecture Searchcited by 24
  8. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  9. 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
  10. 2022FreeREA: Training-Free Evolution-based Architecture Searchcited by 10
  11. 2023Entropic Score metric: Decoupling Topology and Size in Training-free NAScited by 2

Built on by

Not cited yet.