mlsanityData to 14 Oct 2026

IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance

arXiv 2312.12648 ↗First posted 19 Dec 2023cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Among existing Neural Architecture Search methods, DARTS is known for its efficiency and simplicity. This approach applies continuous relaxation of network representation to construct a weight-sharing supernet and enables the identification of excellent subnets in just a few GPU days. However, performance collapse in DARTS results in deteriorating architectures filled with parameter-free operations and remains a great challenge to the robustness. To resolve this problem, we reveal that the fundamental reason is the biased estimation of the candidate importance in the search space through theoretical and experimental analysis, and more precisely select operations via information-based measurements. Furthermore, we demonstrate that the excessive concern over the supernet and inefficient utilization of data in bi-level optimization also account for suboptimal results. We adopt a more realistic objective focusing on the performance of subnets and simplify it with the help of the information-based measurements. Finally, we explain theoretically why progressively shrinking the width of the supernet is necessary and reduce the approximation error of optimal weights in DARTS. Our proposed method, named IS-DARTS, comprehensively improves DARTS and resolves the aforementioned problems. Extensive experiments on NAS-Bench-201 and DARTS-based search space demonstrate the effectiveness of IS-DARTS.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 32 of its 35 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

32 of 35 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  5. 2014Going Deeper with Convolutionscited by 4,241
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  9. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  10. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  11. 2017Progressive Neural Architecture Searchcited by 487
  12. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
20 more
  1. 2017Large-Scale Evolution of Image Classifierscited by 348
  2. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  3. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  4. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  5. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  6. 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
  7. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  8. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  9. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  10. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  11. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  12. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  13. 2020DrNAS: Dirichlet Neural Architecture Searchcited by 73
  14. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  15. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  16. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  17. 2022DropNAS: Grouped Operation Dropout for Differentiable Architecture Searchcited by 12
  18. 2021iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretizationcited by 10
  19. 2023Operation-level Progressive Differentiable Architecture Searchcited by 3
  20. 2023Robustifying DARTS by Eliminating Information Bypass Leakage via Explicit Sparse Regularizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 2; 2026: 1.

5 papers.

  1. 2025SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Modelscited by 7
  2. 2023Efficient Architecture Search via Bi-level Data Pruningcited by 2
  3. 2024HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioningcited by 0
  4. 2025NAS-LoRA: Empowering Parameter-Efficient Fine-Tuning for Visual Foundation Models with Searchable Adaptationcited by 0
  5. 2026Bi-EZP: LLM-Guided Bilevel Program Evolution for Ensemble Zero-Cost Proxy Discoverycited by 0