mlsanityData to 14 Oct 2026

OptiProxy-NAS: Optimization Proxy based End-to-End Neural Architecture Search

arXiv 2509.05656 ↗First posted 6 Sep 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) is a hard computationally expensive optimization problem with a discrete, vast, and spiky search space. One of the key research efforts dedicated to this space focuses on accelerating NAS via certain proxy evaluations of neural architectures. Different from the prevalent predictor-based methods using surrogate models and differentiable architecture search via supernetworks, we propose an optimization proxy to streamline the NAS as an end-to-end optimization framework, named OptiProxy-NAS. In particular, using a proxy representation, the NAS space is reformulated to be continuous, differentiable, and smooth. Thereby, any differentiable optimization method can be applied to the gradient-based search of the relaxed architecture parameters. Our comprehensive experiments on 12 NAS tasks of 4 search spaces across three different domains including computer vision, natural language processing, and resource-constrained NAS fully demonstrate the superior search results and efficiency. Further experiments on low-fidelity scenarios verify the flexibility.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 48 of its 66 references that are in the map. First posted on arXiv 6 Sep 2025.

Builds on

48 of 66 references.

References · 48

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  3. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  7. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  8. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  9. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  10. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  11. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  12. 2017Smart "Predict, then Optimize"cited by 339
36 more
  1. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  2. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  3. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  4. 2020Neural Architecture Search without Trainingcited by 244
  5. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  6. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
  7. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  8. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  9. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  10. 2018Neural Architecture Optimizationcited by 167
  11. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  12. 2019Neural Predictor for Neural Architecture Searchcited by 118
  13. 2020BRP-NAS: Prediction-based NAS using GCNscited by 111
  14. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  15. 2021HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmarkcited by 84
  16. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  17. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  18. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  19. 2020NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processingcited by 60
  20. 2020Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?cited by 59
  21. 2020Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gapcited by 50
  22. 2018Automatically designing CNN architectures using genetic algorithm for image classificationcited by 48
  23. 2020A Study on Encodings for Neural Architecture Searchcited by 38
  24. 2023End-to-End Feasible Optimization Proxies for Large-Scale Economic Dispatchcited by 35
  25. 2021Rapid Neural Architecture Search by Learning to Generate Graphs from Datasetscited by 33
  26. 2021Learning Optimization Proxies for Large-Scale Security-Constrained Economic Dispatchcited by 29
  27. 2021Stronger NAS with Weaker Predictorscited by 28
  28. 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
  29. 2020Exploring the Loss Landscape in Neural Architecture Searchcited by 20
  30. 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
  31. 2020Learning to Rank Learning Curvescited by 16
  32. 2021Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generatorcited by 15
  33. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  34. 2024Multi-conditioned Graph Diffusion for Neural Architecture Searchcited by 14
  35. 2021Generative Adversarial Neural Architecture Searchcited by 9
  36. 2023Predict-Then-Optimize by Proxy: Learning Joint Models of Prediction and Optimizationcited by 4

Built on by

Not cited yet.