B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search
Not on the map. Why?
Abstract (arXiv)
The early pioneering Neural Architecture Search (NAS) works were multi-trial methods applicable to any general search space. The subsequent works took advantage of the early findings and developed weight-sharing methods that assume a structured search space typically with pre-fixed hyperparameters. Despite the amazing computational efficiency of the weight-sharing NAS algorithms, it is becoming apparent that multi-trial NAS algorithms are also needed for identifying very high-performance architectures, especially when exploring a general search space. In this work, we carefully review the latest multi-trial NAS algorithms and identify the key strategies including Evolutionary Algorithm (EA), Bayesian Optimization (BO), diversification, input and output transformations, and lower fidelity estimation. To accommodate the key strategies into a single framework, we develop B2EA that is a surrogate assisted EA with two BO surrogate models and a mutation step in between. To show that B2EA is robust and efficient, we evaluate three performance metrics over 14 benchmarks with general and cell-based search spaces. Comparisons with state-of-the-art multi-trial algorithms reveal that B2EA is robust and efficient over the 14 benchmarks for three difficulty levels of target performance. The B2EA code is publicly available at urlhttps://github.com/snu-adsl/BBEA.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 46 of its 76 references that are in the map. First posted on arXiv 7 Feb 2022.
Builds on
46 of 76 references.
References · 46
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
- 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
- 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
- 2017Progressive Neural Architecture Searchcited by 483
- 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
- 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
- 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
- 2019AutoML: A Survey of the State-of-the-Artcited by 388
- 2017Population Based Training of Neural Networkscited by 351
34 more
- 2017Large-Scale Evolution of Image Classifierscited by 348
- 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
- 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
- 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
- 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
- 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
- 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
- 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
- 2021Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020cited by 146
- 2016COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Settingcited by 135
- 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
- 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 129
- 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
- 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
- 2021Rethinking Architecture Selection in Differentiable NAScited by 92
- 2020Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimizationcited by 89
- 2021HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmarkcited by 83
- 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
- 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
- 2014Input Warping for Bayesian Optimization of Non-stationary Functionscited by 66
- 2020NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processingcited by 60
- 2019Tabular Benchmarks for Joint Architecture and Hyperparameter Optimizationcited by 57
- 2019Bayesian Optimisation over Multiple Continuous and Categorical Inputscited by 55
- 2019Best Practices for Scientific Research on Neural Architecture Searchcited by 53
- 2015GLASSES: Relieving The Myopia Of Bayesian Optimisationcited by 38
- 2020A Study on Encodings for Neural Architecture Searchcited by 37
- 2021NAS-Bench-x11 and the Power of Learning Curvescited by 20
- 2020Efficient Rollout Strategies for Bayesian Optimizationcited by 19
- 2018Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Searchcited by 17
- 2019Compositionally-Warped Gaussian Processescited by 14
- 2019EENA: Efficient Evolution of Neural Architecturecited by 10
- 2021NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameterscited by 9
- 2020Robust Gaussian Process Regression with a Bias Modelcited by 5
Built on by
Citing papers per year: 2022: 2; 2023: 0; 2024: 1.
3 papers.