mlsanityData to 14 Oct 2026

ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies

arXiv 2110.10423 ↗First posted 20 Oct 2021cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Designing neural architectures requires immense manual efforts. This has promoted the development of neural architecture search (NAS) to automate the design. While previous NAS methods achieve promising results but run slowly, zero-cost proxies run extremely fast but are less promising. Therefore, it is of great potential to accelerate NAS via those zero-cost proxies. The existing method has two limitations, which are unforeseeable reliability and one-shot usage. To address the limitations, we present ProxyBO, an efficient Bayesian optimization (BO) framework that utilizes the zero-cost proxies to accelerate neural architecture search. We apply the generalization ability measurement to estimate the fitness of proxies on the task during each iteration and design a novel acquisition function to combine BO with zero-cost proxies based on their dynamic influence. Extensive empirical studies show that ProxyBO consistently outperforms competitive baselines on five tasks from three public benchmarks. Concretely, ProxyBO achieves up to 5.41x and 3.86x speedups over the state-of-the-art approaches REA and BRP-NAS.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 40 of its 68 references that are in the map. First posted on arXiv 20 Oct 2021.

Builds on

40 of 68 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  7. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  8. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  9. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  10. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  11. 2017Progressive Neural Architecture Searchcited by 487
  12. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
28 more
  1. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  2. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 470
  3. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  4. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  5. 2020Neural Architecture Search without Trainingcited by 244
  6. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  7. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  8. 2019The Evolved Transformercited by 180
  9. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 179
  10. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  11. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  12. 2018Neural Architecture Optimizationcited by 167
  13. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  14. 2020BRP-NAS: Prediction-based NAS using GCNscited by 111
  15. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  16. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 94
  17. 2021Rethinking Architecture Selection in Differentiable NAScited by 92
  18. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  19. 2014Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictionscited by 67
  20. 2020EcoNAS: Finding Proxies for Economical Neural Architecture Searchcited by 61
  21. 2020Evaluating Efficient Performance Estimators of Neural Architecturescited by 55
  22. 2021OpenBox: A Generalized Black-box Optimization Servicecited by 46
  23. 2021Rapid Neural Architecture Search by Learning to Generate Graphs from Datasetscited by 33
  24. 2021NASI: Label- and Data-agnostic Neural Architecture Search at Initializationcited by 28
  25. 2021VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decompositioncited by 25
  26. 2019Deep Neural Architecture Search with Deep Graph Bayesian Optimizationcited by 18
  27. 2020MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurementscited by 18
  28. 2019Pre-train and Learn: Preserve Global Information for Graph Neural Networkscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 7; 2025: 3; 2026: 3.

22 papers.

Sort:

By area:

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  2. 2022PaSca: a Graph Neural Architecture Search System under the Scalable Paradigmcited by 37
  3. 2022NAS-Bench-Suite-Zero: Accelerating Research on Zero Cost Proxiescited by 36
  4. 2022Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scalecited by 13
  5. 2024Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Priorcited by 11
  6. 2024Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimizationcited by 9
  7. 2023Efficient Activation Function Optimization through Surrogate Modelingcited by 5
  8. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  9. 2023Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"cited by 3
  10. 2024TG-NAS: Generalizable Zero-Cost Proxies with Operator Description Embedding and Graph Learning for Efficient Neural Architecture Searchcited by 2
  11. 2023Neural Architecture Search: Two Constant Shared Weights Initialisationscited by 1
  12. 2023An Evaluation of Zero-Cost Proxies -- from Neural Architecture Performance to Model Robustnesscited by 1
10 more
  1. 2024SONATA: Self-adaptive Evolutionary Framework for Hardware-aware Neural Architecture Searchcited by 1
  2. 2025Simple Optimizers for Convex Aligned Multi-Objective Optimizationcited by 1
  3. 2025Where to Begin: Efficient Pretraining via Subnetwork Selection and Distillationcited by 1
  4. 2023Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initializationcited by 0
  5. 2024Hard Work Does Not Always Pay Off: Poisoning Attacks on Neural Architecture Searchcited by 0
  6. 2024Manifold Metric: A Loss Landscape Approach for Predicting Model Performancecited by 0
  7. 2025MOCA-HESP: Meta High-dimensional Bayesian Optimization for Combinatorial and Mixed Spaces via Hyper-ellipsoid Partitioningcited by 0
  8. 2026Prediction-powered Neural Architecture Searchcited by 0
  9. 2026Bayesian Optimization in Sequence-to-Architecture Latent Space for Zero-Shot NAScited by 0
  10. 2026DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneckcited by 0