# DARTS-PRIME: Regularization and Scheduling Improve Constrained Optimization in Differentiable NAS

arXiv: https://arxiv.org/abs/2106.11655 · HTML: https://mlsanity.com/paper/2106.11655/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 22 Jun 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [Searching for A Robust Neural Architecture in Four GPU Hours](https://mlsanity.com/paper/1910.04465/) (2019)
- [Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation](https://mlsanity.com/paper/1904.12760/) (2019)
- [Understanding and Robustifying Differentiable Architecture Search](https://mlsanity.com/paper/1909.09656/) (2019)
- [Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search](https://mlsanity.com/paper/1911.12126/) (2019)
- [DARTS+: Improved Differentiable Architecture Search with Early Stopping](https://mlsanity.com/paper/1909.06035/) (2019)
- [Faster gaze prediction with dense networks and Fisher pruning](https://mlsanity.com/paper/1801.05787/) (2018)
- [DARTS-: Robustly Stepping out of Performance Collapse Without Indicators](https://mlsanity.com/paper/2009.01027/) (2020)
- [Efficient Neural Architecture Search via Proximal Iterations](https://mlsanity.com/paper/1905.13577/) (2019)
- [Theory-Inspired Path-Regularized Differential Network Architecture Search](https://mlsanity.com/paper/2006.16537/) (2020)
- [ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning](https://mlsanity.com/paper/1905.00136/) (2019)
- [Progressive Weight Pruning of Deep Neural Networks using ADMM](https://mlsanity.com/paper/1810.07378/) (2018)
- [Reducing the Model Order of Deep Neural Networks Using Information Theory](https://mlsanity.com/paper/1605.04859/) (2016)
- [StructADMM: A Systematic, High-Efficiency Framework of Structured Weight Pruning for DNNs](https://mlsanity.com/paper/1807.11091/) (2018)
- [Alternating Direction Method of Multipliers for Sparse Convolutional Neural Networks](https://mlsanity.com/paper/1611.01590/) (2016)

## Built on by (most cited first, 1 of 1)

- [High-order Knowledge Based Network Controllability Robustness Prediction: A Hypergraph Neural Network Approach](https://mlsanity.com/paper/2603.02265/) (2026)

## Abstract (arXiv)

Differentiable Architecture Search (DARTS) is a recent neural architecture search (NAS) method based on a differentiable relaxation. Due to its success, numerous variants analyzing and improving parts of the DARTS framework have recently been proposed. By considering the problem as a constrained bilevel optimization, we present and analyze DARTS-PRIME, a variant including improvements to architectural weight update scheduling and regularization towards discretization. We propose a dynamic schedule based on per-minibatch network information to make architecture updates more informed, as well as proximity regularization to promote well-separated discretization. Our results in multiple domains show that DARTS-PRIME improves both performance and reliability, comparable to state-of-the-art in differentiable NAS.