# Transformation Models for Flexible Posteriors in Variational Bayes

arXiv: https://arxiv.org/abs/2106.00528 · HTML: https://mlsanity.com/paper/2106.00528/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Multiplicative Normalizing Flows for Variational Bayesian Neural Networks](https://mlsanity.com/paper/1703.01961/) (2017)
- [Stacking for Non-mixing Bayesian Computations: The Curse and Blessing of Multimodal Posteriors](https://mlsanity.com/paper/2006.12335/) (2020)
- [Deep Conditional Transformation Models](https://mlsanity.com/paper/2010.07860/) (2020)
- [Most Likely Transformations](https://mlsanity.com/paper/1508.06749/) (2015)
- [Deep transformation models: Tackling complex regression problems with neural network based transformation models](https://mlsanity.com/paper/2004.00464/) (2020)
- [Bayesian Conditional Transformation Models](https://mlsanity.com/paper/2012.11016/) (2020)
- [Robust normalizing flows using Bernstein-type polynomials](https://mlsanity.com/paper/2102.03509/) (2021)
- [Deep and interpretable regression models for ordinal outcomes](https://mlsanity.com/paper/2010.08376/) (2020)

## Abstract (arXiv)

The main challenge in Bayesian models is to determine the posterior for the model parameters. Already, in models with only one or few parameters, the analytical posterior can only be determined in special settings. In Bayesian neural networks, variational inference is widely used to approximate difficult-to-compute posteriors by variational distributions. Usually, Gaussians are used as variational distributions (Gaussian-VI) which limits the quality of the approximation due to their limited flexibility. Transformation models on the other hand are flexible enough to fit any distribution. Here we present transformation model-based variational inference (TM-VI) and demonstrate that it allows to accurately approximate complex posteriors in models with one parameter and also works in a mean-field fashion for multi-parameter models like neural networks.