# Efficient Neural Architecture Search via Proximal Iterations

arXiv: https://arxiv.org/abs/1905.13577 · HTML: https://mlsanity.com/paper/1905.13577/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 30 of 31)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Rethinking Architecture Selection in Differentiable NAS](https://mlsanity.com/paper/2108.04392/) (2021)
- [DC-BENCH: Dataset Condensation Benchmark](https://mlsanity.com/paper/2207.09639/) (2022)
- [Search to aggregate neighborhood for graph neural network](https://mlsanity.com/paper/2104.06608/) (2021)
- [NASI: Label- and Data-agnostic Neural Architecture Search at Initialization](https://mlsanity.com/paper/2109.00817/) (2021)
- [Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Search](https://mlsanity.com/paper/2101.11342/) (2021)
- [Generalizing Few-Shot NAS with Gradient Matching](https://mlsanity.com/paper/2203.15207/) (2022)
- [Core-set Sampling for Efficient Neural Architecture Search](https://mlsanity.com/paper/2107.06869/) (2021)
- [AutoGEL: An Automated Graph Neural Network with Explicit Link Information](https://mlsanity.com/paper/2112.01064/) (2021)
- [Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search](https://mlsanity.com/paper/2201.09785/) (2022)
- [Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey](https://mlsanity.com/paper/2505.01821/) (2025)
- [Searching to Sparsify Tensor Decomposition for N-ary Relational Data](https://mlsanity.com/paper/2104.10625/) (2021)
- [Memory-Efficient Differentiable Transformer Architecture Search](https://mlsanity.com/paper/2105.14669/) (2021)
- [Bilinear Scoring Function Search for Knowledge Graph Learning](https://mlsanity.com/paper/2107.00184/) (2021)
- [Efficient Relation-aware Scoring Function Search for Knowledge Graph Embedding](https://mlsanity.com/paper/2104.10880/) (2021)
- [Equivariance-aware Architectural Optimization of Neural Networks](https://mlsanity.com/paper/2210.05484/) (2022)
- [Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics](https://mlsanity.com/paper/2108.11939/) (2021)
- [AutoAC: Towards Automated Attribute Completion for Heterogeneous Graph Neural Network](https://mlsanity.com/paper/2301.03049/) (2023)
- [Neural Architecture Search for Speech Emotion Recognition](https://mlsanity.com/paper/2203.16928/) (2022)
- [Differentiable Meta Multigraph Search with Partial Message Propagation on Heterogeneous Information Networks](https://mlsanity.com/paper/2211.14752/) (2022)
- [ColdNAS: Search to Modulate for User Cold-Start Recommendation](https://mlsanity.com/paper/2306.03387/) (2023)
- [Long-range Meta-path Search on Large-scale Heterogeneous Graphs](https://mlsanity.com/paper/2307.08430/) (2023)
- [Robustifying and Boosting Training-Free Neural Architecture Search](https://mlsanity.com/paper/2403.07591/) (2024)
- [CONet: Channel Optimization for Convolutional Neural Networks](https://mlsanity.com/paper/2108.06822/) (2021)
- [Guided Evolutionary Neural Architecture Search With Efficient Performance Estimation](https://mlsanity.com/paper/2208.06475/) (2022)
- [Efficient Search of Comprehensively Robust Neural Architectures via Multi-fidelity Evaluation](https://mlsanity.com/paper/2305.07308/) (2023)
- [DARTS-PRIME: Regularization and Scheduling Improve Constrained Optimization in Differentiable NAS](https://mlsanity.com/paper/2106.11655/) (2021)
- [Meta-multigraph Search: Rethinking Meta-structure on Heterogeneous Information Networks](https://mlsanity.com/paper/2304.11574/) (2023)
- [MS-DARTS: Mean-Shift Based Differentiable Architecture Search](https://mlsanity.com/paper/2108.09996/) (2021)
- [Delve into the Performance Degradation of Differentiable Architecture Search](https://mlsanity.com/paper/2109.13466/) (2021)

## Abstract (arXiv)

Neural architecture search (NAS) recently attracts much research attention because of its ability to identify better architectures than handcrafted ones. However, many NAS methods, which optimize the search process in a discrete search space, need many GPU days for convergence. Recently, DARTS, which constructs a differentiable search space and then optimizes it by gradient descent, can obtain high-performance architecture and reduces the search time to several days. However, DARTS is still slow as it updates an ensemble of all operations and keeps only one after convergence. Besides, DARTS can converge to inferior architectures due to the strong correlation among operations. In this paper, we propose a new differentiable Neural Architecture Search method based on Proximal gradient descent (denoted as NASP). Different from DARTS, NASP reformulates the search process as an optimization problem with a constraint that only one operation is allowed to be updated during forward and backward propagation. Since the constraint is hard to deal with, we propose a new algorithm inspired by proximal iterations to solve it. Experiments on various tasks demonstrate that NASP can obtain high-performance architectures with 10 times of speedup on the computational time than DARTS.