mlsanityData to 14 Oct 2026

DARTS-PRIME: Regularization and Scheduling Improve Constrained Optimization in Differentiable NAS

arXiv 2106.11655 ↗First posted 22 Jun 2021cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Differentiable Architecture Search (DARTS) is a recent neural architecture search (NAS) method based on a differentiable relaxation. Due to its success, numerous variants analyzing and improving parts of the DARTS framework have recently been proposed. By considering the problem as a constrained bilevel optimization, we present and analyze DARTS-PRIME, a variant including improvements to architectural weight update scheduling and regularization towards discretization. We propose a dynamic schedule based on per-minibatch network information to make architecture updates more informed, as well as proximity regularization to promote well-separated discretization. Our results in multiple domains show that DARTS-PRIME improves both performance and reliability, comparable to state-of-the-art in differentiable NAS.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 22 Jun 2021.

Builds on

21 of 30 references.

References · 21

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  6. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  7. 2017Progressive Neural Architecture Searchcited by 487
  8. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  9. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  10. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  11. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  12. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
9 more
  1. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 94
  2. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  3. 2019Efficient Neural Architecture Search via Proximal Iterationscited by 31
  4. 2020Theory-Inspired Path-Regularized Differential Network Architecture Searchcited by 25
  5. 2019ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruningcited by 13
  6. 2018Progressive Weight Pruning of Deep Neural Networks using ADMMcited by 11
  7. 2016Reducing the Model Order of Deep Neural Networks Using Information Theorycited by 10
  8. 2018StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNscited by 8
  9. 2016Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026High-order Knowledge Based Network Controllability Robustness Prediction: A Hypergraph Neural Network Approachcited by 0