mlsanityData to 14 Oct 2026

Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation

arXiv 1904.12760 ↗First posted 29 Apr 2019cs.CV cs.LGcited by 230

Not on the map. Why?

Abstract (arXiv)

Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdarts.

Key facts. Cited by 230 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 88; 2022: 51; 2023: 42; 2024: 23; 2025: 16; 2026: 10.

230 papers.

Sort:

By area:

  1. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  2. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 153
  3. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  4. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  5. 2021LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Searchcited by 82
  6. 2022DC-BENCH: Dataset Condensation Benchmarkcited by 67
  7. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  8. 2021Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyondcited by 63
  9. 2021A Value-Function-based Interior-point Method for Non-convex Bi-level Optimizationcited by 60
  10. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
  11. 2021iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradientscited by 51
  12. 2021DiNTS: Differentiable Neural Network Topology Search for 3D Medical Image Segmentationcited by 48
38 more
  1. 2021Searching the Search Space of Vision Transformercited by 41
  2. 2021CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identificationcited by 38
  3. 2021Teachers Do More Than Teach: Compressing Image-to-Image Modelscited by 37
  4. 2021PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillationcited by 36
  5. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  6. 2021Contrastive Neural Architecture Search with Neural Architecture Comparatorscited by 34
  7. 2023Visual Tuningcited by 34
  8. 2021Neural Architecture Search with Random Labelscited by 33
  9. 2021Stronger NAS with Weaker Predictorscited by 28
  10. 2021AdvRush: Searching for Adversarially Robust Neural Architecturescited by 28
  11. 2021NASI: Label- and Data-agnostic Neural Architecture Search at Initializationcited by 28
  12. 2021Towards Improving the Consistency, Efficiency, and Flexibility of Differentiable Neural Architecture Searchcited by 27
  13. 2023Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexitycited by 26
  14. 2022Generalizing Few-Shot NAS with Gradient Matchingcited by 25
  15. 2021Value-Function-based Sequential Minimization for Bi-level Optimizationcited by 24
  16. 2021Core-set Sampling for Efficient Neural Architecture Searchcited by 23
  17. 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
  18. 2023Auto-HeG: Automated Graph Neural Network on Heterophilic Graphscited by 23
  19. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 22
  20. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  21. 2021BN-NAS: Neural Architecture Search with Batch Normalizationcited by 21
  22. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  23. 2021HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Searchcited by 20
  24. 2022Continual Object Detection via Prototypical Task Correlation Guided Gating Mechanismcited by 20
  25. 2021Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentationcited by 19
  26. 2021Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspectivecited by 19
  27. 2021ZARTS: On Zero-order Optimization for Neural Architecture Searchcited by 18
  28. 2022On Redundancy and Diversity in Cell-based Neural Architecture Searchcited by 18
  29. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  30. 2022Unifying and Boosting Gradient-Based Training-Free Neural Architecture Searchcited by 17
  31. 2022Sparse Structure Search for Parameter-Efficient Tuningcited by 17
  32. 2023Near-Optimal Nonconvex-Strongly-Convex Bilevel Optimization with Fully First-Order Oraclescited by 17
  33. 2022Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentationcited by 16
  34. 2023MSINet: Twins Contrastive Search of Multi-Scale Interaction for Object ReIDcited by 16
  35. 2024Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategycited by 16
  36. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  37. 2022Automated Dilated Spatio-Temporal Synchronous Graph Modeling for Traffic Predictioncited by 15
  38. 2021Accelerating Neural Architecture Search via Proxy Datacited by 14

50 most cited of 230 All 230 (JSON)