# Accurate Uncertainty Estimation and Decomposition in Ensemble Learning

arXiv: https://arxiv.org/abs/1911.04061 · HTML: https://mlsanity.com/paper/1911.04061/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2019.

## Built on by (most cited first, 30 of 42)

- [Classifier Calibration: A survey on how to assess and improve predicted class probabilities](https://mlsanity.com/paper/2112.10327/) (2021)
- [Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification](https://mlsanity.com/paper/2203.10192/) (2022)
- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Valid prediction intervals for regression problems](https://mlsanity.com/paper/2107.00363/) (2021)
- [From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation](https://mlsanity.com/paper/2402.10727/) (2024)
- [DOTA: Distributional Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2409.19375/) (2024)
- [Drone-assisted Road Gaussian Splatting with Cross-view Uncertainty](https://mlsanity.com/paper/2408.15242/) (2024)
- [Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation](https://mlsanity.com/paper/2107.04212/) (2021)
- [MixedNUTS: Training-Free Accuracy-Robustness Balance via Nonlinearly Mixed Classifiers](https://mlsanity.com/paper/2402.02263/) (2024)
- [Using AI Uncertainty Quantification to Improve Human Decision-Making](https://mlsanity.com/paper/2309.10852/) (2023)
- [Dense Uncertainty Estimation](https://mlsanity.com/paper/2110.06427/) (2021)
- [Clarifying Trust of Materials Property Predictions using Neural Networks with Distribution-Specific Uncertainty Quantification](https://mlsanity.com/paper/2302.02595/) (2023)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning](https://mlsanity.com/paper/2209.15449/) (2022)
- [Benchmarking Uncertainty Quantification on Biosignal Classification Tasks under Dataset Shift](https://mlsanity.com/paper/2112.09196/) (2021)
- [Model-agnostic variable importance for predictive uncertainty: an entropy-based approach](https://mlsanity.com/paper/2310.12842/) (2023)
- [SiBBlInGS: Similarity-driven Building-Block Inference using Graphs across States](https://mlsanity.com/paper/2306.04817/) (2023)
- [On the Calibration of Uncertainty Estimation in LiDAR-based Semantic Segmentation](https://mlsanity.com/paper/2308.02248/) (2023)
- [Selective Learning: Towards Robust Calibration with Dynamic Regularization](https://mlsanity.com/paper/2402.08384/) (2024)
- [Modeling Mask Uncertainty in Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2112.15362/) (2021)
- [Estimating Regression Predictive Distributions with Sample Networks](https://mlsanity.com/paper/2211.13724/) (2022)
- [Industry Scale Semi-Supervised Learning for Natural Language Understanding](https://mlsanity.com/paper/2103.15871/) (2021)
- [Tailored Uncertainty Estimation for Deep Learning Systems](https://mlsanity.com/paper/2204.13963/) (2022)
- [Estimating Uncertainty in Landslide Segmentation Models](https://mlsanity.com/paper/2311.11138/) (2023)
- [Generative Data Mining with Longtail-Guided Diffusion](https://mlsanity.com/paper/2502.01980/) (2025)
- [Generalization-Memorization Machines](https://mlsanity.com/paper/2207.03976/) (2022)
- [Surrogate uncertainty estimation for your time series forecasting black-box: learn when to trust](https://mlsanity.com/paper/2302.02834/) (2023)
- [Sampling-based Uncertainty Estimation for an Instance Segmentation Network](https://mlsanity.com/paper/2305.14977/) (2023)
- [h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective](https://mlsanity.com/paper/2506.17968/) (2025)
- [Uncertainty-Aware Segmentation Quality Prediction via Deep Learning Bayesian Modeling: Comprehensive Evaluation and Interpretation on Skin Cancer and Liver Segmentation](https://mlsanity.com/paper/2508.01460/) (2025)

## Abstract (arXiv)

Ensemble learning is a standard approach to building machine learning systems that capture complex phenomena in real-world data. An important aspect of these systems is the complete and valid quantification of model uncertainty. We introduce a Bayesian nonparametric ensemble (BNE) approach that augments an existing ensemble model to account for different sources of model uncertainty. BNE augments a model's prediction and distribution functions using Bayesian nonparametric machinery. It has a theoretical guarantee in that it robustly estimates the uncertainty patterns in the data distribution, and can decompose its overall predictive uncertainty into distinct components that are due to different sources of noise and error. We show that our method achieves accurate uncertainty estimates under complex observational noise, and illustrate its real-world utility in terms of uncertainty decomposition and model bias detection for an ensemble in predict air pollution exposures in Eastern Massachusetts, USA.