mlsanityData to 14 Oct 2026

Bayesian posterior approximation with stochastic ensembles

arXiv 2212.08123 ↗First posted 15 Dec 2022cs.LG cs.CV stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distributions and trained to approximate the Bayesian posterior with variational inference. We implement stochastic ensembles based on Monte Carlo dropout, DropConnect and a novel non-parametric version of dropout and evaluate them on a toy problem and CIFAR image classification. For both tasks, we test the quality of the posteriors directly against Hamiltonian Monte Carlo simulations. Our results show that stochastic ensembles provide more accurate posterior estimates than other popular baselines for Bayesian inference.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 38 of its 64 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

38 of 64 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  7. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  8. 2016Deep Networks with Stochastic Depthcited by 979
  9. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  10. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  11. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  12. 2015Weight Uncertainty in Neural Networkscited by 554
26 more
  1. 2020A Survey of Deep Active Learningcited by 554
  2. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  3. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  4. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  5. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  6. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  7. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
  8. 2017Concrete Dropoutcited by 227
  9. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  10. 2015Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inferencecited by 159
  11. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  12. 2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 150
  13. 2021Repulsive Deep Ensembles are Bayesiancited by 104
  14. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  15. 2020Neural Ensemble Search for Uncertainty Estimation and Dataset Shiftcited by 56
  16. 2021Is MC Dropout Bayesian?cited by 35
  17. 2017Dropout Inference in Bayesian Neural Networks with Alpha-divergencescited by 28
  18. 2021Deep Ensembles from a Bayesian Perspectivecited by 27
  19. 2021Diversity Matters When Learning From Ensemblescited by 27
  20. 2020Diverse Ensembles Improve Calibrationcited by 23
  21. 2022Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrappingcited by 19
  22. 2022Improving robustness and calibration in ensembles with diversity regularizationcited by 8
  23. 2022Deep interpretable ensemblescited by 5
  24. 2020Improving Ensemble Robustness by Collaboratively Promoting and Demoting Adversarial Robustnesscited by 4
  25. 2021Greedy Bayesian Posterior Approximation with Deep Ensemblescited by 3
  26. 2022Theoretical Error Analysis of Entropy Approximation for Gaussian Mixturescited by 2

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3; 2025: 1.

5 papers.

  1. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 37
  2. 2024CLAP4CLIP: Continual Learning with Probabilistic Finetuning for Vision-Language Modelscited by 25
  3. 2023Sensitivity-Aware Amortized Bayesian Inferencecited by 20
  4. 2024Bayesian RG Flow in Neural Network Field Theoriescited by 9
  5. 2025PEdger++: Practical Edge Detection via Assembling Cross Informationcited by 0