mlsanityData to 7 Oct 2026

Neural Ensemble Search via Bayesian Sampling

arXiv 2109.02533 ↗First posted 6 Sep 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Recently, neural architecture search (NAS) has been applied to automate the design of neural networks in real-world applications. A large number of algorithms have been developed to improve the search cost or the performance of the final selected architectures in NAS. Unfortunately, these NAS algorithms aim to select only one single well-performing architecture from their search spaces and thus have overlooked the capability of neural network ensemble (i.e., an ensemble of neural networks with diverse architectures) in achieving improved performance over a single final selected architecture. To this end, we introduce a novel neural ensemble search algorithm, called neural ensemble search via Bayesian sampling (NESBS), to effectively and efficiently select well-performing neural network ensembles from a NAS search space. In our extensive experiments, NESBS algorithm is shown to be able to achieve improved performance over state-of-the-art NAS algorithms while incurring a comparable search cost, thus indicating the superior performance of our NESBS algorithm over these NAS algorithms in practice.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 37 of its 45 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

37 of 45 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
25 more
  1. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  2. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  5. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  6. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  7. 2017Progressive Neural Architecture Searchcited by 483
  8. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  9. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  10. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  11. 2017Snapshot Ensembles: Train 1, get M for freecited by 349
  12. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  13. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  14. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  15. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  16. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  17. 2019FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Searchcited by 129
  18. 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
  19. 2019When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attackscited by 87
  20. 2020DARTS-: Robustly Stepping out of Performance Collapse Without Indicatorscited by 81
  21. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  22. 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
  23. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  24. 2017Message Passing Stein Variational Gradient Descentcited by 47
  25. 2017Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networkscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 0; 2026: 1.

7 papers.

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  2. 2022Sample-Then-Optimize Batch Neural Thompson Samplingcited by 25
  3. 2022Bayesian Optimization under Stochastic Delayed Feedbackcited by 10
  4. 2023Training-Free Neural Active Learning with Initialization-Robustness Guaranteescited by 5
  5. 2024Robustifying and Boosting Training-Free Neural Architecture Searchcited by 4
  6. 2023Bayesian Quadrature for Neural Ensemble Searchcited by 1
  7. 2026SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networkscited by 1