mlsanityData to 7 Oct 2026

Single-DARTS: Towards Stable Architecture Search

arXiv 2108.08128 ↗First posted 18 Aug 2021cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Differentiable architecture search (DARTS) marks a milestone in Neural Architecture Search (NAS), boasting simplicity and small search costs. However, DARTS still suffers from frequent performance collapse, which happens when some operations, such as skip connections, zeroes and poolings, dominate the architecture. In this paper, we are the first to point out that the phenomenon is attributed to bi-level optimization. We propose Single-DARTS which merely uses single-level optimization, updating network weights and architecture parameters simultaneously with the same data batch. Even single-level optimization has been previously attempted, no literature provides a systematic explanation on this essential point. Replacing the bi-level optimization, Single-DARTS obviously alleviates performance collapse as well as enhances the stability of architecture search. Experiment results show that Single-DARTS achieves state-of-the-art performance on mainstream search spaces. For instance, on NAS-Benchmark-201, the searched architectures are nearly optimal ones. We also validate that the single-level optimization framework is much more stable than the bi-level one. We hope that this simple yet effective method will give some insights on differential architecture search. The code is available at https://github.com/PencilAndBike/Single-DARTS.git.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

33 of 40 references.

References · 33

  1. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  5. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  6. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  7. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  8. 2017Progressive Neural Architecture Searchcited by 483
  9. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  10. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  11. 2017Large-Scale Evolution of Image Classifierscited by 348
  12. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
21 more
  1. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  2. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  3. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  4. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  5. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  6. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  7. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  8. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
  9. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  10. 2017Neural Optimizer Search with Reinforcement Learningcited by 82
  11. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  12. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  13. 2020AutoGAN-Distiller: Searching to Compress Generative Adversarial Networkscited by 50
  14. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  15. 2019Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Searchcited by 23
  16. 2019sharpDARTS: Faster and More Accurate Differentiable Architecture Searchcited by 19
  17. 2020Cyclic Differentiable Architecture Searchcited by 18
  18. 2020GOLD-NAS: Gradual, One-Level, Differentiablecited by 15
  19. 2019Hierarchical Neural Architecture Search via Operator Clusteringcited by 5
  20. 2020Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processescited by 5
  21. 2019Single-Path NAS: Device-Aware Efficient ConvNet Designcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 0; 2024: 1.

3 papers.

  1. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  2. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  3. 2024Soft-Label Integration for Robust Toxicity Classificationcited by 1