mlsanityData to 7 Oct 2026

Using stacking to average Bayesian predictive distributions

arXiv 1704.02030 ↗First posted 6 Apr 2017stat.ME stat.COcited by 39

Not on the map. Why?

Abstract (arXiv)

The widely recommended procedure of Bayesian model averaging is flawed in the M-open setting in which the true data-generating process is not one of the candidate models being fit. We take the idea of stacking from the point estimation literature and generalize to the combination of predictive distributions, extending the utility function to any proper scoring rule, using Pareto smoothed importance sampling to efficiently compute the required leave-one-out posterior distributions and regularization to get more stability. We compare stacking of predictive distributions to several alternatives: stacking of means, Bayesian model averaging (BMA), pseudo-BMA using AIC-type weighting, and a variant of pseudo-BMA that is stabilized using the Bayesian bootstrap. Based on simulations and real-data applications, we recommend stacking of predictive distributions, with BB-pseudo-BMA as an approximate alternative when computation cost is an issue.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 7; 2023: 7; 2024: 10; 2025: 7; 2026: 2.

39 papers.

Sort:

  1. 2022Forecast combinations: an over 50-year reviewcited by 34
  2. 2022Bayesian Active Learning for Scanning Probe Microscopy: from Gaussian Processes to Hypothesis Learningcited by 16
  3. 2021Bayesian hierarchical stacking: Some models are (somewhere) usefulcited by 14
  4. 2021A Tree-based Model Averaging Approach for Personalized Treatment Effect Estimation from Heterogeneous Data Sourcescited by 14
  5. 2022Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximationscited by 12
  6. 2022A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecastingcited by 12
  7. 2025Simulation-based Bayesian inference under model misspecificationcited by 9
  8. 2021Fast and Scalable Spike and Slab Variable Selection in High-Dimensional Gaussian Processescited by 7
  9. 2021The Implications of the No-Free-Lunch Theorems for Meta-inductioncited by 6
  10. 2022Meta-Uncertainty in Bayesian Model Comparisoncited by 5
  11. 2024Dynamic Online Ensembles of Basis Expansionscited by 5
  12. 2025Ensemble Survival Analysis for Preclinical Cognitive Decline Prediction in Alzheimer's Disease Using Longitudinal Biomarkerscited by 4
27 more
  1. 2023Ensemble learning for blending gridded satellite and gauge-measured precipitation datacited by 3
  2. 2023Simulation-based stackingcited by 3
  3. 2024Ensemble learning for uncertainty estimation with application to the correction of satellite precipitation productscited by 3
  4. 2025Stability of Mean-Field Variational Inferencecited by 3
  5. 2023Local Bayesian Dirichlet mixing of imperfect modelscited by 2
  6. 2021A Subspace-based Approach for Dimensionality Reduction and Important Variable Selectioncited by 1
  7. 2024Fusion of Gaussian Processes Predictions with Monte Carlo Samplingcited by 1
  8. 2024Model orthogonalization and Bayesian forecast mixing via Principal Component Analysiscited by 1
  9. 2024Towards a Probabilistic Fusion Approach for Robust Battery Prognosticscited by 1
  10. 2024MODL: Multilearner Online Deep Learningcited by 1
  11. 2026FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Modelscited by 1
  12. 2021Bayesian Model Averaging for Data Driven Decision Making when Causality is Partially Knowncited by 0
  13. 2022Multi-Model Probabilistic Programmingcited by 0
  14. 2022Robust leave-one-out cross-validation for high-dimensional Bayesian modelscited by 0
  15. 2023Bayesian Analysis for Over-parameterized Linear Model via Effective Spectracited by 0
  16. 2023Automating Model Comparison in Factor Graphscited by 0
  17. 2023On the meaning of uncertainty for ethical AI: philosophy and practicecited by 0
  18. 2023Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Supportcited by 0
  19. 2024BayesBlend: Easy Model Blending using Pseudo-Bayesian Model Averaging, Stacking and Hierarchical Stacking in Pythoncited by 0
  20. 2024Addressing the Inconsistency in Bayesian Deep Learning via Generalized Laplace Approximationcited by 0
  21. 2024Combinations of distributional regression algorithms with application in uncertainty estimation of corrected satellite precipitation productscited by 0
  22. 2024Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategycited by 0
  23. 2025Understanding the Trade-offs in Accuracy and Uncertainty Quantification: Architecture and Inference Choices in Bayesian Neural Networkscited by 0
  24. 2025On Local Posterior Structure in Deep Ensemblescited by 0
  25. 2025Measuring the right thing: justifying metrics in AI impact assessmentscited by 0
  26. 2025Bayesian Ensembling: Insights from Online Optimization and Empirical Bayescited by 0
  27. 2026A Divergence-Based Method for Weighting and Averaging Model Predictionscited by 0