mlsanityData to 7 Oct 2026

MS-DARTS: Mean-Shift Based Differentiable Architecture Search

arXiv 2108.09996 ↗First posted 23 Aug 2021cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Differentiable Architecture Search (DARTS) is an effective continuous relaxation-based network architecture search (NAS) method with low search cost. It has attracted significant attentions in Auto-ML research and becomes one of the most useful paradigms in NAS. Although DARTS can produce superior efficiency over traditional NAS approaches with better control of complex parameters, oftentimes it suffers from stabilization issues in producing deteriorating architectures when discretizing the continuous architecture. We observed considerable loss of validity causing dramatic decline in performance at this final discretization step of DARTS. To address this issue, we propose a Mean-Shift based DARTS (MS-DARTS) to improve stability based on sampling and perturbation. Our approach can improve bot the stability and accuracy of DARTS, by smoothing the loss landscape and sampling architecture parameters within a suitable bandwidth. We investigate the convergence of our mean-shift approach, together with the effects of bandwidth selection that affects stability and accuracy. Evaluations performed on CIFAR-10, CIFAR-100, and ImageNet show that MS-DARTS archives higher performance over other state-of-the-art NAS methods with reduced search cost.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 23 Aug 2021.

Builds on

38 of 49 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2014Going Deeper with Convolutionscited by 4,231
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  8. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  9. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  10. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  11. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  12. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
26 more
  1. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  2. 2017Progressive Neural Architecture Searchcited by 483
  3. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  4. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  5. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  6. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  7. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  8. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  9. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  10. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  11. 2018Neural Architecture Optimizationcited by 164
  12. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  13. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  14. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  15. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  16. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81
  17. 2014On Using Very Large Target Vocabulary for Neural Machine Translationcited by 80
  18. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  19. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  20. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  21. 2019XNAS: Neural Architecture Search with Expert Advicecited by 35
  22. 2016Convolutional Neural Fabricscited by 32
  23. 2019Efficient Neural Architecture Search via Proximal Iterationscited by 31
  24. 2020Discretization-Aware Architecture Searchcited by 14
  25. 2020Latency-Aware Differentiable Neural Architecture Searchcited by 10
  26. 2018MaskConnect: Connectivity Learning by Gradient Descentcited by 9

Built on by

Not cited yet.