mlsanityData to 7 Oct 2026

Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrapping

arXiv 2206.03633 ↗First posted 8 Jun 2022cs.LG cs.AI stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

In machine learning, an agent needs to estimate uncertainty to efficiently explore and adapt and to make effective decisions. A common approach to uncertainty estimation maintains an ensemble of models. In recent years, several approaches have been proposed for training ensembles, and conflicting views prevail with regards to the importance of various ingredients of these approaches. In this paper, we aim to address the benefits of two ingredients -- prior functions and bootstrapping -- which have come into question. We show that prior functions can significantly improve an ensemble agent's joint predictions across inputs and that bootstrapping affords additional benefits if the signal-to-noise ratio varies across inputs. Our claims are justified by both theoretical and experimental results.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

23 of 40 references.

References · 23

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  7. 2016Deep Exploration via Bootstrapped DQNcited by 518
  8. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  9. 2017A Tutorial on Thompson Samplingcited by 420
  10. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 218
  11. 2021Epistemic Neural Networkscited by 120
  12. 2021Repulsive Deep Ensembles are Bayesiancited by 103
11 more
  1. 2020Bayesian Deep Ensembles via the Neural Tangent Kernelcited by 90
  2. 2017Ensemble Samplingcited by 62
  3. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  4. 2020Hypermodels for Explorationcited by 37
  5. 2019Maximizing Overall Diversity for Improved Uncertainty Estimates in Deep Ensemblescited by 30
  6. 2021From Predictions to Decisions: The Importance of Joint Predictive Distributionscited by 28
  7. 2021Diversity Matters When Learning From Ensemblescited by 27
  8. 2022An Analysis of Ensemble Samplingcited by 23
  9. 2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21
  10. 2018Diversity regularization in deep ensemblescited by 11
  11. 2022Evaluating High-Order Predictive Distributions in Deep Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 7; 2025: 3; 2026: 1.

19 papers.

Sort:

By area:

  1. 2024To Believe or Not to Believe Your LLMcited by 120
  2. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  3. 2023Leveraging Demonstrations to Improve Online Learning: Quality Matterscited by 13
  4. 2023Using AI Uncertainty Quantification to Improve Human Decision-Makingcited by 11
  5. 2023Probabilistic Inference in Reinforcement Learning Done Rightcited by 11
  6. 2023Efficient Exploration via Epistemic-Risk-Seeking Policy Optimizationcited by 10
  7. 2023Neural Exploitation and Exploration of Contextual Banditscited by 7
  8. 2024Online Bandit Learning with Offline Preference Data for Improved RLHFcited by 6
  9. 2022Bayesian posterior approximation with stochastic ensemblescited by 5
  10. 2024Satisficing Exploration for Deep Reinforcement Learningcited by 4
  11. 2024Fully Heteroscedastic Count Regression with Deep Double Poisson Networkscited by 3
  12. 2025Best Policy Learning from Trajectory Preference Feedbackcited by 3
7 more
  1. 2025Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?cited by 3
  2. 2023Bayesian Quadrature for Neural Ensemble Searchcited by 1
  3. 2023Likelihood-ratio-based confidence intervals for neural networkscited by 1
  4. 2024Epinet for Content Cold Startcited by 1
  5. 2025Pointwise confidence estimation in the non-linear ℓ2-regularized least squarescited by 1
  6. 2026Sharp analysis of linear ensemble samplingcited by 1
  7. 2024Trends, Challenges, and Future Directions in Deep Learning for Glaucoma: A Systematic Reviewcited by 0