mlsanityData to 14 Oct 2026

Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support

arXiv 2310.14888 ↗First posted 23 Oct 2023cs.LG cs.PLcited by 0

Not on the map. Why?

Abstract (arXiv)

The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path. We show that making predictions with this full posterior implicitly performs a Bayesian model averaging (BMA) over paths. This is potentially problematic, as BMA weights can be unstable due to model misspecification or inference approximations, leading to sub-optimal predictions in turn. To remedy this issue, we propose alternative mechanisms for path weighting: one based on stacking and one based on ideas from PAC-Bayes. We show how both can be implemented as a cheap post-processing step on top of existing inference engines. In our experiments, we find them to be more robust and lead to better predictions compared to the default BMA weights.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 96 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

31 of 96 references.

References · 31

  1. 2015Importance Weighted Autoencoderscited by 456
  2. 2018Pyro: Deep Universal Probabilistic Programmingcited by 380
  3. 2013Black Box Variational Inferencecited by 319
  4. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  5. 2017A Conceptual Introduction to Hamiltonian Monte Carlocited by 209
  6. 2015Practical Bayesian model evaluation using leave-one-out cross-validation and WAICcited by 110
  7. 2018An Introduction to Probabilistic Programmingcited by 63
  8. 2015Pareto Smoothed Importance Samplingcited by 42
  9. 2017Using stacking to average Bayesian predictive distributionscited by 39
  10. 2013Automated Variational Inference in Probabilistic Programmingcited by 34
  11. 2016Inference Compilation and Universal Probabilistic Programmingcited by 32
  12. 2017Deep Probabilistic Programmingcited by 24
19 more
  1. 2015A New Approach to Probabilistic Programming Inferencecited by 22
  2. 2014Venture: a higher-order probabilistic programming platform with programmable inferencecited by 20
  3. 2020Automatic Differentiation Variational Inference with Mixturescited by 19
  4. 2015Layered Adaptive Importance Samplingcited by 18
  5. 2019Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scalecited by 18
  6. 2016Design and Implementation of Probabilistic Programming Language Anglicancited by 15
  7. 2021Bayesian hierarchical stacking: Some models are (somewhere) usefulcited by 14
  8. 2015A Lambda-Calculus Foundation for Universal Probabilistic Programmingcited by 12
  9. 2016Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraintscited by 12
  10. 2022ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programscited by 12
  11. 2018Automated learning with a probabilistic programming language: Birchcited by 11
  12. 2019Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Supportcited by 11
  13. 2020Holes in Bayesian Statisticscited by 7
  14. 2020Reproducible Model Selection Using Bagged Posteriorscited by 7
  15. 2016Interacting Particle Markov Chain Monte Carlocited by 6
  16. 2016Composing inference algorithms as program transformationscited by 4
  17. 2019Attention for Inference Compilationcited by 3
  18. 2020Symbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysiscited by 1
  19. 2022Higher-Order Generalization Bounds: Learning Deep Probabilistic Programs via PAC-Bayes Objectivescited by 1

Built on by

Not cited yet.