mlsanityData to 7 Oct 2026

OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength

arXiv 2409.14433 ↗First posted 22 Sep 2024cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Differentiable architecture search (DARTS) has emerged as a promising technique for effective neural architecture search, and it mainly contains two steps to find the high-performance architecture: First, the DARTS supernet that consists of mixed operations will be optimized via gradient descent. Second, the final architecture will be built by the selected operations that contribute the most to the supernet. Although DARTS improves the efficiency of NAS, it suffers from the well-known degeneration issue which can lead to deteriorating architectures. Existing works mainly attribute the degeneration issue to the failure of its supernet optimization, while little attention has been paid to the selection method. In this paper, we cease to apply the widely-used magnitude-based selection method and propose a novel criterion based on operation strength that estimates the importance of an operation by its effect on the final loss. We show that the degeneration issue can be effectively addressed by using the proposed criterion without any modification of supernet optimization, indicating that the magnitude-based selection method can be a critical reason for the instability of DARTS. The experiments on NAS-Bench-201 and DARTS search spaces show the effectiveness of our method.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 58 of its 83 references that are in the map. First posted on arXiv 22 Sep 2024.

Builds on

58 of 83 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  4. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  5. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  6. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  11. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
38 more
  1. 2019Importance Estimation for Neural Network Pruningcited by 531
  2. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  3. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  4. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  5. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  6. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  7. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  8. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  9. 2020Neural Architecture Search without Trainingcited by 241
  10. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  11. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  12. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  13. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  14. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  15. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  16. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  17. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 121
  18. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  19. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  20. 2020Convolutional Networks with Dense Connectivitycited by 98
  21. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  22. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  23. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 86
  24. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  25. 2020DrNAS: Dirichlet Neural Architecture Searchcited by 73
  26. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  27. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  28. 2020Progressive Skeletonization: Trimming more fat from a network at initializationcited by 64
  29. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 63
  30. 2016Highway and Residual Networks learn Unrolled Iterative Estimationcited by 52
  31. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  32. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  33. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  34. 2021CondenseNet V2: Sparse Feature Reactivation for Deep Networkscited by 34
  35. 2021Neural Architecture Search with Random Labelscited by 33
  36. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  37. 2019BlockSwap: Fisher-guided Block Substitution for Network Compression on a Budgetcited by 26
  38. 2020DOTS: Decoupling Operation and Topology in Differentiable Architecture Searchcited by 23

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Bilevel Optimization for Neural Architecture Searchcited by 2
  2. 2025A Novel Trustworthy Video Summarization Algorithm Through a Mixture of LoRA Expertscited by 0
  3. 2026Designing Compact Neural Architectures via Neuron Gating and Mixed Activationcited by 0
  4. 2026LP-NAS: Linear Programming-based Neural Architecture Searchcited by 0