# Bernstein Flows for Flexible Posteriors in Variational Bayes

arXiv: https://arxiv.org/abs/2202.05650 · HTML: https://mlsanity.com/paper/2202.05650/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 11 Feb 2022.

## Builds on (most cited first, 26 of 26)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Sylvester Normalizing Flows for Variational Inference](https://mlsanity.com/paper/1803.05649/) (2018)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Yes, but Did It Work?: Evaluating Variational Inference](https://mlsanity.com/paper/1802.02538/) (2018)
- [Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization](https://mlsanity.com/paper/2006.10343/) (2020)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors](https://mlsanity.com/paper/2006.12335/) (2020)
- [Validated Variational Inference via Practical Posterior Error Bounds](https://mlsanity.com/paper/1910.04102/) (2019)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [Robust, Accurate Stochastic Optimization for Variational Inference](https://mlsanity.com/paper/2009.00666/) (2020)
- [Most Likely Transformations](https://mlsanity.com/paper/1508.06749/) (2015)
- [Deep transformation models: Tackling complex regression problems with neural network based transformation models](https://mlsanity.com/paper/2004.00464/) (2020)
- [Multivariate Conditional Transformation Models](https://mlsanity.com/paper/1906.03151/) (2019)
- [Probabilistic Time Series Forecasts with Autoregressive Transformation Models](https://mlsanity.com/paper/2110.08248/) (2021)
- [Bayesian Conditional Transformation Models](https://mlsanity.com/paper/2012.11016/) (2020)
- [Robust normalizing flows using Bernstein-type polynomials](https://mlsanity.com/paper/2102.03509/) (2021)

## Built on by (most cited first, 3 of 3)

- [Distributional Gradient Boosting Machines](https://mlsanity.com/paper/2204.00778/) (2022)
- [Foundation Posteriors for Approximate Probabilistic Inference](https://mlsanity.com/paper/2205.09735/) (2022)
- [Bayesian Semi-structured Subspace Inference](https://mlsanity.com/paper/2401.12950/) (2024)

## Abstract (arXiv)

Variational inference (VI) is a technique to approximate difficult to compute posteriors by optimization. In contrast to MCMC, VI scales to many observations. In the case of complex posteriors, however, state-of-the-art VI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method, flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art VI methods including normalizing flow based VI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI outperforms other VI methods. Further, we develop with BF-VI a Bayesian model for the semi-structured Melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate for the first time how the use of VI in semi-structured models.