mlsanityData to 7 Oct 2026

Bayesian Quadrature for Neural Ensemble Search

arXiv 2303.08874 ↗First posted 15 Mar 2023stat.ML cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Ensembling can improve the performance of Neural Networks, but existing approaches struggle when the architecture likelihood surface has dispersed, narrow peaks. Furthermore, existing methods construct equally weighted ensembles, and this is likely to be vulnerable to the failure modes of the weaker architectures. By viewing ensembling as approximately marginalising over architectures we construct ensembles using the tools of Bayesian Quadrature -- tools which are well suited to the exploration of likelihood surfaces with dispersed, narrow peaks. Additionally, the resulting ensembles consist of architectures weighted commensurate with their performance. We show empirically -- in terms of test likelihood, accuracy, and expected calibration error -- that our method outperforms state-of-the-art baselines, and verify via ablation studies that its components do so independently.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 39 of its 55 references that are in the map. First posted on arXiv 15 Mar 2023.

Builds on

39 of 55 references.

References · 39

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  11. 2019AutoML: A Survey of the State-of-the-Artcited by 388
  12. 2017Large-Scale Evolution of Image Classifierscited by 348
27 more
  1. 2018Slimmable Neural Networkscited by 301
  2. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  3. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  4. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  5. 2019Universally Slimmable Networks and Improved Training Techniquescited by 219
  6. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
  7. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
  8. 2020BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Modelscited by 187
  9. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  10. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 149
  11. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  12. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  13. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  14. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  15. 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
  16. 2023AutoPEFT: Automatic Configuration Search for Parameter-Efficient Fine-Tuningcited by 49
  17. 2014Sampling for Inference in Probabilistic Models with Fast Bayesian Quadraturecited by 31
  18. 2021Positively Weighted Kernel Quadrature via Subsamplingcited by 25
  19. 2018Evolutionary Architecture Search For Deep Multitask Networkscited by 23
  20. 2022Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombinationcited by 20
  21. 2022Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrappingcited by 19
  22. 2021Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimationcited by 18
  23. 2018Improving Quadrature for Constrained Integrandscited by 12
  24. 2018Diversity regularization in deep ensemblescited by 11
  25. 2021Neural Ensemble Search via Bayesian Samplingcited by 7
  26. 2021Approximate Neural Architecture Search via Operation Distribution Learningcited by 6
  27. 2022Active Learning and Novel Model Calibration Measurements for Automated Visual Inspection in Manufacturingcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Conditional Bayesian Quadraturecited by 8