Data to 14 Oct 2026

Uncertainty in Neural Networks: Approximately Bayesian Ensembling

arXiv 1810.05546 ↗First posted 12 Oct 2018stat.ML cs.LGcited by 94

Not on the map. Why?

Abstract (arXiv)

Understanding the uncertainty of a neural network's (NN) predictions is essential for many purposes. The Bayesian framework provides a principled approach to this, however applying it to NNs is challenging due to large numbers of parameters and data. Ensembling NNs provides an easily implementable, scalable method for uncertainty quantification, however, it has been criticised for not being Bayesian. This work proposes one modification to the usual process that we argue does result in approximate Bayesian inference; regularising parameters about values drawn from a distribution which can be set equal to the prior. A theoretical analysis of the procedure in a simplified setting suggests the recovered posterior is centred correctly but tends to have an underestimated marginal variance, and overestimated correlation. However, two conditions can lead to exact recovery. We argue that these conditions are partially present in NNs. Empirical evaluations demonstrate it has an advantage over standard ensembling, and is competitive with variational methods.

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 19; 2023: 17; 2024: 20; 2025: 11; 2026: 4.

94 papers.

Sort:

By area:

2022A Survey of Machine Unlearningcited by 312
2021Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimationcited by 71
2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 62
2021Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantificationcited by 57
2022A Review of Uncertainty for Deep Reinforcement Learningcited by 40
2021Strength of Minibatch Noise in SGDcited by 39
2021Uncertainty-Aware Machine Translation Evaluationcited by 35
2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 31
2023Learn to Accumulate Evidence from All Training Samples: Theory and Practicecited by 29
2021Deep Ensembles from a Bayesian Perspectivecited by 27
2023A Primer on Bayesian Neural Networks: Review and Debatescited by 27
2022Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Roboticscited by 24

38 more

2021PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physicscited by 21
2023Towards Efficient MCMC Sampling in Bayesian Neural Networks by Exploiting Symmetrycited by 21
2022UncertaINR: Uncertainty Quantification of End-to-End Implicit Neural Representations for Computed Tomographycited by 19
2022Sampling-based inference for large linear models, with application to linearised Laplacecited by 18
2024Uncertainty Quantification Metrics for Deep Regressioncited by 16
2021How to Evaluate Uncertainty Estimates in Machine Learning for Regression?cited by 15
2024Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Modelscited by 15
2021Pathologies in priors and inference for Bayesian transformerscited by 13
2023DUCK: Distance-based Unlearning via Centroid Kinematicscited by 13
2024Uncertainty Regularized Evidential Regressioncited by 13
2021Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoringcited by 11
2024A Structured Review of Literature on Uncertainty in Machine Learning & Deep Learningcited by 10
2023A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learningcited by 8
2021Bayesian Inference Forgettingcited by 7
2023Uncertainty Quantification in Machine Learning for Biosignal Applications -- A Reviewcited by 7
2021Model-Value Inconsistency as a Signal for Epistemic Uncertaintycited by 6
2021PI3NN: Out-of-distribution-aware prediction intervals from three neural networkscited by 5
2021Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection?cited by 5
2022Analytic Mutual Information in Bayesian Neural Networkscited by 5
2023Deep Evidential Learning for Bayesian Quantile Regressioncited by 5
2024Low-Budget Simulation-Based Inference with Bayesian Neural Networkscited by 5
2021Uncertainty-Aware Boosted Ensembling in Multi-Modal Settingscited by 4
2021A framework for benchmarking uncertainty in deep regressioncited by 4
2022Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detectioncited by 4
2023Deep Gaussian Mixture Ensemblescited by 4
2024Uncertainty Quantification on Graph Learning: A Surveycited by 4
2021Quantifying sources of uncertainty in drug discovery predictions with probabilistic modelscited by 3
2021Greedy Bayesian Posterior Approximation with Deep Ensemblescited by 3
2021Quasi-Bayesian Dual Instrumental Variable Regressioncited by 3
2021Which Model to Trust: Assessing the Influence of Models on the Performance of Reinforcement Learning Algorithms for Continuous Control Taskscited by 3
2022Confident Neural Network Regression with Bootstrapped Deep Ensemblescited by 3
2022Do autoencoders need a bottleneck for anomaly detection?cited by 3
2022Distributional Actor-Critic Ensemble for Uncertainty-Aware Continuous Controlcited by 3
2022Sample-based Uncertainty Quantification with a Single Deterministic Neural Networkcited by 3
2022Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelinescited by 3
2024On Uncertainty Quantification for Near-Bayes Optimal Algorithmscited by 3
2024Error-Driven Uncertainty Aware Trainingcited by 3
2024DADEE: Well-calibrated uncertainty quantification in neural networks for barriers-based robot safetycited by 3

50 most cited of 94 All 94 (JSON)