Understanding and mitigating difficulties in posterior predictive evaluation
Not on the map. Why?
Abstract (arXiv)
Predictive posterior densities (PPDs) are of interest in approximate Bayesian inference. Typically, these are estimated by simple Monte Carlo (MC) averages using samples from the approximate posterior. We observe that the signal-to-noise ratio (SNR) of such estimators can be extremely low. An analysis for exact inference reveals SNR decays exponentially as there is an increase in (a) the mismatch between training and test data, (b) the dimensionality of the latent space, or (c) the size of the test data relative to the training data. Further analysis extends these results to approximate inference. To remedy the low SNR problem, we propose replacing simple MC sampling with importance sampling using a proposal distribution optimized at test time on a variational proxy for the SNR and demonstrate that this yields greatly improved estimates.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 70 references that are in the map. First posted on arXiv 30 May 2024.
Builds on
45 of 70 references.
References · 45
- 2021Convex optimizationcited by 2,515
- 2015Variational Inference with Normalizing Flowscited by 2,170
- 2016Density estimation using Real NVPcited by 2,042
- 2016Variational Inference: A Review for Statisticianscited by 1,472
- 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
- 2015Importance Weighted Autoencoderscited by 456
- 2018Pyro: Deep Universal Probabilistic Programmingcited by 380
- 2013Black Box Variational Inferencecited by 319
- 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 308
- 2019Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyrocited by 191
- 2016Automatic Differentiation Variational Inferencecited by 159
- 2015Practical Bayesian model evaluation using leave-one-out cross-validation and WAICcited by 110
33 more
- 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 98
- 2017Filtering Variational Objectivescited by 91
- 2018Tighter Variational Bounds are Not Necessarily Bettercited by 76
- 2019A Systematic Comparison of Bayesian Deep Learning Robustness in Diabetic Retinopathy Taskscited by 62
- 2019Generalized Variational Inference: Three arguments for deriving new Posteriorscited by 57
- 2018Importance Weighting and Variational Inferencecited by 55
- 2018Doubly Reparameterized Gradient Estimators for Monte Carlo Objectivescited by 41
- 2016On the Quantitative Analysis of Decoder-Based Generative Modelscited by 40
- 2022A Variational Perspective on Generative Flow Networkscited by 39
- 2020Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimizationcited by 37
- 2021Challenges and Opportunities in High-dimensional Variational Inferencecited by 35
- 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
- 2019Provable Smoothness Guarantees for Black-Box Variational Inferencecited by 29
- 2022A review of predictive uncertainty estimation with machine learningcited by 29
- 2023Provable convergence guarantees for black-box variational inferencecited by 29
- 2020Automatic structured variational inferencecited by 22
- 2023On the Convergence of Black-Box Variational Inferencecited by 22
- 2019Provable Gradient Variance Guarantees for Black-Box Variational Inferencecited by 21
- 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
- 2021Amortized Variational Inference for Simple Hierarchical Modelscited by 19
- 2019Variational Bayesian Decision-making for Continuous Utilitiescited by 16
- 2021Generalized Doubly Reparameterized Gradient Estimatorscited by 14
- 2023Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?cited by 10
- 2021Automatic variational inference with cascading flowscited by 9
- 2021Post-hoc loss-calibration for Bayesian neural networkscited by 9
- 2016Overdispersed Black-Box Variational Inferencecited by 8
- 2022Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradientscited by 6
- 2023Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inferencecited by 6
- 2023Sample Average Approximation for Black-Box VIcited by 6
- 2022Loss-calibrated expectation propagation for approximate Bayesian decision-makingcited by 5
- 2019Correcting Predictions for Approximate Bayesian Inferencecited by 4
- 2022Bayesian Forecasting in Economics and Finance: A Modern Reviewcited by 4
- 2023Rethinking Variational Inference for Probabilistic Programs with Stochastic Supportcited by 2
Built on by
Citing papers per year: 2024: 1.
1 paper.