mlsanityData to 7 Oct 2026

Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

arXiv 1912.10952 ↗First posted 23 Dec 2019cs.CV cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

With the rapid development of neural architecture search (NAS), researchers found powerful network architectures for a wide range of vision tasks. However, it remains unclear if the searched architecture can transfer across different types of tasks as manually designed ones did. This paper puts forward this problem, referred to as NAS in the wild, which explores the possibility of finding the optimal architecture in a proxy dataset and then deploying it to mostly unseen scenarios. We instantiate this setting using a currently popular algorithm named differentiable architecture search (DARTS), which often suffers unsatisfying performance while being transferred across different tasks. We argue that the accuracy drop originates from the formulation that uses a super-network for search but a sub-network for re-training. The different properties of these stages have resulted in a significant optimization gap, and consequently, the architectural parameters "over-fit" the super-network. To alleviate the gap, we present a progressive method that gradually increases the network depth during the search stage, which leads to the Progressive DARTS (P-DARTS) algorithm. With a reduced search cost (7 hours on a single GPU), P-DARTS achieves improved performance on both the proxy dataset (CIFAR10) and a few target problems (ImageNet classification, COCO detection and three ReID benchmarks). Our code is available at urlhttps://github.com/chenxin061/pdarts.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 6; 2023: 9; 2024: 6; 2025: 6; 2026: 2.

39 papers.

Sort:

  1. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  2. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 71
  3. 2024ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Modelscited by 36
  4. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 33
  5. 2021Designing the Topology of Graph Neural Networks: A Novel Feature Fusion Perspectivecited by 19
  6. 2021Trilevel Neural Architecture Search for Efficient Single Image Super-Resolutioncited by 15
  7. 2021FBNetV5: Neural Architecture Search for Multiple Tasks in One Runcited by 15
  8. 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
  9. 2022Efficient Joint-Dimensional Search with Solution Space Regularization for Real-Time Semantic Segmentationcited by 9
  10. 2021Learning from Mistakes -- A Framework for Neural Architecture Searchcited by 6
  11. 2023Improving Differentiable Architecture Search via Self-Distillationcited by 6
  12. 2023OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operatorscited by 6
27 more
  1. 2021Contrastive Self-supervised Neural Architecture Searchcited by 5
  2. 2022NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimensioncited by 5
  3. 2024CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Frameworkcited by 5
  4. 2021NAX: Co-Designing Neural Network and Hardware Architecture for Memristive Xbar based Computing Systemscited by 3
  5. 2021CHASE: Robust Visual Tracking via Cell-Level Differentiable Neural Architecture Searchcited by 3
  6. 2022CR-LSO: Convex Neural Architecture Optimization in the Latent Space of Graph Variational Autoencoder with Input Convex Neural Networkscited by 2
  7. 2023A lightweight network for photovoltaic cell defect detection in electroluminescence images based on neural architecture search and knowledge distillationcited by 2
  8. 2023Carbon-Efficient Neural Architecture Searchcited by 2
  9. 2022One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learningcited by 1
  10. 2023NASiam: Efficient Representation Learning using Neural Architecture Search for Siamese Networkscited by 1
  11. 2023Shrink-Perturb Improves Architecture Mixing during Population Based Training for Neural Architecture Searchcited by 1
  12. 2023Improving Prompt Tuning with Learned Prompting Layerscited by 1
  13. 2024Multi-Objective Neural Architecture Search by Learning Search Space Partitionscited by 1
  14. 2025ZO-DARTS++: An Efficient and Size-Variable Zeroth-Order Neural Architecture Search Algorithmcited by 1
  15. 2025DDS-NAS: Dynamic Data Selection within Neural Architecture Search via On-line Hard Example Mining applied to Image Classificationcited by 1
  16. 2025Zero-Shot Neural Architecture Search with Weighted Response Correlationcited by 1
  17. 2021D-DARTS: Distributed Differentiable Architecture Searchcited by 0
  18. 2022Efficient Re-parameterization Operations Search for Easy-to-Deploy Network Based on Directional Evolutionary Strategycited by 0
  19. 2023Automated Search-Space Generation Neural Architecture Searchcited by 0
  20. 2023ICDARTS: Improving the Stability and Performance of Cyclic DARTScited by 0
  21. 2024Heterogeneous Learning Rate Scheduling for Neural Architecture Search on Long-Tailed Datasetscited by 0
  22. 2024S2HPruner: Soft-to-Hard Distillation Bridges the Discretization Gap in Pruningcited by 0
  23. 2025Neural Architecture Search by Learning a Hierarchical Search Spacecited by 0
  24. 2025FX-DARTS: Designing Topology-unconstrained Architectures with Differentiable Architecture Search and Entropy-based Super-network Shrinkingcited by 0
  25. 2025DNAD: Differentiable Neural Architecture Distillationcited by 0
  26. 2026MIDAS: Mosaic Input-Specific Differentiable Architecture Searchcited by 0
  27. 2026EvoTreeNAD: Genealogy-Guided Evolution for LLM-Driven Neural Architecture Discoverycited by 0