mlsanityData to 7 Oct 2026

Uncertainty in Gradient Boosting via Ensembles

arXiv 2006.10562 ↗First posted 18 Jun 2020cs.LG stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

For many practical, high-risk applications, it is essential to quantify uncertainty in a model's predictions to avoid costly mistakes. While predictive uncertainty is widely studied for neural networks, the topic seems to be under-explored for models based on gradient boosting. However, gradient boosting often achieves state-of-the-art results on tabular data. This work examines a probabilistic ensemble-based framework for deriving uncertainty estimates in the predictions of gradient boosting classification and regression models. We conducted experiments on a range of synthetic and real datasets and investigated the applicability of ensemble approaches to gradient boosting models that are themselves ensembles of decision trees. Our analysis shows that ensembles of gradient boosting models successfully detect anomalous inputs while having limited ability to improve the predicted total uncertainty. Importantly, we also propose a concept of a virtual ensemble to get the benefits of an ensemble via only one gradient boosting model, which significantly reduces complexity.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 14; 2023: 12; 2024: 12; 2025: 7; 2026: 4.

56 papers.

Sort:

By area:

  1. 2023Decomposing Uncertainty for Large Language Models through Input Clarification Ensemblingcited by 113
  2. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 100
  3. 2024Uncertainty quantification in fine-tuned LLMs using LoRA ensemblescited by 35
  4. 2021Fed-ensemble: Improving Generalization through Model Ensembling in Federated Learningcited by 23
  5. 2023Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Meancited by 18
  6. 2022Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Treescited by 16
  7. 2022When to intervene? Prescriptive Process Monitoring Under Uncertainty and Resource Constraintscited by 11
  8. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  9. 2022Instance-Based Uncertainty Estimation for Gradient-Boosted Regression Treescited by 9
  10. 2022On the Calibration of Probabilistic Classifier Setscited by 8
  11. 2024SeqNAS: Neural Architecture Search for Event Sequence Classificationcited by 8
  12. 2022Uncertainty-Aware Mixed-Variable Machine Learning for Materials Designcited by 6
38 more
  1. 2022Benchmarking Machine Learning Robustness in Covid-19 Genome Sequence Classificationcited by 6
  2. 2024Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMscited by 5
  3. 2021Uncertainty-Aware Boosted Ensembling in Multi-Modal Settingscited by 4
  4. 2022On Uncertainty Estimation by Tree-based Surrogate Models in Sequential Model-based Optimizationcited by 4
  5. 2022Gradient Boosting Performs Gaussian Process Inferencecited by 4
  6. 2023Prescriptive Process Monitoring Under Resource Constraints: A Reinforcement Learning Approachcited by 4
  7. 2022Confident Sinkhorn Allocation for Pseudo-Labelingcited by 3
  8. 2024Regularized KL-Divergence for Well-Defined Function-Space Variational Inference in Bayesian neural networkscited by 3
  9. 2025MorphBoost: Self-Organizing Universal Gradient Boosting with Adaptive Tree Morphingcited by 3
  10. 2021Evaluating Predictive Uncertainty and Robustness to Distributional Shift Using Real World Datacited by 2
  11. 2022Towards Clear Expectations for Uncertainty Estimationcited by 2
  12. 2023Surrogate uncertainty estimation for your time series forecasting black-box: learn when to trustcited by 2
  13. 2023Machine-Learning Solutions for the Analysis of Single-Particle Diffusion Trajectoriescited by 2
  14. 2024SEMF: Supervised Expectation-Maximization Framework for Predicting Intervalscited by 2
  15. 2024Diffusion Boosted Treescited by 2
  16. 2024Advancing Molecular Machine Learning Representations with Stereoelectronics-Infused Molecular Graphscited by 2
  17. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
  18. 2025TabResFlow: A Normalizing Spline Flow Model for Probabilistic Univariate Tabular Regressioncited by 2
  19. 2025Statistical Inference for Gradient Boosting Regressioncited by 2
  20. 2021Look Before You Leap! Designing a Human-Centered AI System for Change Risk Assessmentcited by 1
  21. 2021Detecting and Mitigating Test-time Failure Risks via Model-agnostic Uncertainty Learningcited by 1
  22. 2021More layers! End-to-end regression and uncertainty on tabular data with deep learningcited by 1
  23. 2022TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regressioncited by 1
  24. 2022Uncertainty Aware Trader-Company Method: Interpretable Stock Price Prediction Capturing Uncertaintycited by 1
  25. 2023SE(3)-Invariant Multiparameter Persistent Homology for Chiral-Sensitive Molecular Property Predictioncited by 1
  26. 2024ProbSAINT: Probabilistic Tabular Regression for Used Car Pricingcited by 1
  27. 2024Gradient Boosting Reinforcement Learningcited by 1
  28. 2024Position Specific Scoring Is All You Need? Revisiting Protein Sequence Classification Taskscited by 1
  29. 2025Unveiling the Role of Data Uncertainty in Tabular Deep Learningcited by 1
  30. 2026A Ground-Truth Framework for Uncertainty Disentanglement with Posterior Riskcited by 1
  31. 2022Probabilistic Models for Manufacturing Lead Timescited by 0
  32. 2022Uncertainty in Extreme Multi-label Classificationcited by 0
  33. 2023Variational Boosted Soft Treescited by 0
  34. 2023Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unitcited by 0
  35. 2023Identifying contributors to supply chain outcomes in a multi-echelon setting: a decentralised approachcited by 0
  36. 2023Correcting sampling biases via importance reweighting for spatial modelingcited by 0
  37. 2023Improving Opioid Use Disorder Risk Modelling through Behavioral and Genetic Feature Integrationcited by 0
  38. 2023Model Uncertainty based Active Learning on Tabular Data using Boosted Treescited by 0

50 most cited of 56 All 56 (JSON)