Data to 14 Oct 2026

Deep Ensembles from a Bayesian Perspective

arXiv 2105.13283 ↗First posted 27 May 2021cs.LG cs.AI stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

Deep ensembles can be considered as the current state-of-the-art for uncertainty quantification in deep learning. While the approach was originally proposed as a non-Bayesian technique, arguments supporting its Bayesian footing have been put forward as well. We show that deep ensembles can be viewed as an approximate Bayesian method by specifying the corresponding assumptions. Our findings lead to an improved approximation which results in an enlarged epistemic part of the uncertainty. Numerical examples suggest that the improved approximation can lead to more reliable uncertainties. Analytical derivations ensure easy calculation of results.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 29 of its 51 references that are in the map. First posted on arXiv 27 May 2021.

Builds on

29 of 51 references.

References · 29

2014Adam: A Method for Stochastic Optimizationcited by 37,240
2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
2016Variational Inference: A Review for Statisticianscited by 1,472
2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
2015Variational Dropout and the Local Reparameterization Trickcited by 490
2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338

17 more

2017DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Carscited by 251
2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 210
2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
2019Explainability Techniques for Graph Convolutional Networkscited by 161
2020Hyperparameter Ensembles for Robustness and Uncertainty Quantificationcited by 150
2018Uncertainty in Neural Networks: Approximately Bayesian Ensemblingcited by 94
2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
2019Evaluating and Calibrating Uncertainty Prediction in Regression Taskscited by 87
2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 61
2019Quality of Uncertainty Quantification for Bayesian Neural Network Inferencecited by 59
2018Testing Deep Neural Networkscited by 36
2020Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithmscited by 34
2015Early Stopping is Nonparametric Variational Inferencecited by 23
2020On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimationcited by 23
2018The Relevance of Bayesian Layer Positioning to Model Uncertainty in Deep Bayesian Active Learningcited by 13
2021Uncertainty Quantification by Ensemble Learning for Computational Optical Form Measurementscited by 11

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 8; 2023: 5; 2024: 6; 2025: 5.

27 papers.

Sort:

By area:

2022Deep Ensembles Work, But Are They Necessary?cited by 63
2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 37
2024On Information-Theoretic Measures of Predictive Uncertaintycited by 16
2024Zero-Shot Uncertainty Quantification using Diffusion Probabilistic Modelscited by 14
2023Finite-time Lyapunov exponents of deep neural networkscited by 12
2025Uncertainty Quantification with the Empirical Neural Tangent Kernelcited by 11
2023Graph Neural Network Interatomic Potential Ensembles with Calibrated Aleatoric and Epistemic Uncertainty on Energy and Forcescited by 10
2024A prediction rigidity formalism for low-cost uncertainties in trained neural networkscited by 8
2021Bayesian Neural Networks: Essentialscited by 7
2022Ensembling over Classifiers: a Bias-Variance Perspectivecited by 6
2022Bayesian Deep Learning with Multilevel Trace-class Neural Networkscited by 5
2022Deep interpretable ensemblescited by 5

15 more

2022Bayesian posterior approximation with stochastic ensemblescited by 5
2021A framework for benchmarking uncertainty in deep regressioncited by 4
2023Deep Gaussian Mixture Ensemblescited by 4
2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
2024DADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safetycited by 3
2025Contextual Similarity Distillation: Ensemble Uncertainties with a Single Modelcited by 3
2022Looking at the posterior: accuracy and uncertainty of neural-network predictionscited by 2
2023Confident Naturalness Explanation (CNE): A Framework to Explain and Assess Patterns Forming Naturalnesscited by 2
2022DBCal: Density Based Calibration of classifier predictions for uncertainty quantificationcited by 1
2021A deep ensemble approach to X-ray polarimetrycited by 0
2023Coherent energy and force uncertainty in deep learning force fieldscited by 0
2024A Priori Uncertainty Quantification of Reacting Turbulence Closure Models using Bayesian Neural Networkscited by 0
2025Evaluating structural uncertainty in accelerated MRI: are voxelwise measures useful surrogates?cited by 0
2025EMoE: Training-Free Expert Disagreement for Uncertainty-Aware Text-to-Image Diffusioncited by 0
2025Quantifying Out-of-Training Uncertainty of Neural-Network based Turbulence Closurescited by 0