# A practical tutorial on Variational Bayes

arXiv: https://arxiv.org/abs/2103.01327 · HTML: https://mlsanity.com/paper/2103.01327/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 40 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Concentration of tempered posteriors and of their variational approximations](https://mlsanity.com/paper/1706.09293/) (2017)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations](https://mlsanity.com/paper/1906.02914/) (2019)
- [Convergence Rates of Variational Posterior Distributions](https://mlsanity.com/paper/1712.02519/) (2017)
- [Gaussian variational approximation with a factor covariance structure](https://mlsanity.com/paper/1701.03208/) (2017)
- [Gaussian variational approximation with sparse precision matrices](https://mlsanity.com/paper/1605.05622/) (2016)
- [Variational Bayes with Intractable Likelihood](https://mlsanity.com/paper/1503.08621/) (2015)
- [Variational Bayes on Manifolds](https://mlsanity.com/paper/1908.03097/) (2019)
- [Recurrent Conditional Heteroskedasticity](https://mlsanity.com/paper/2010.13061/) (2020)
- [Copula-type Estimators for Flexible Multivariate Density Modeling using Mixtures](https://mlsanity.com/paper/1306.3033/) (2013)

## Built on by (most cited first, 10 of 10)

- [Machine Learning in Aerodynamic Shape Optimization](https://mlsanity.com/paper/2202.07141/) (2022)
- [Bayesian Learning for Neural Networks: an algorithmic survey](https://mlsanity.com/paper/2211.11865/) (2022)
- [Bayesian Bilinear Neural Network for Predicting the Mid-price Dynamics in Limit-Order Book Markets](https://mlsanity.com/paper/2203.03613/) (2022)
- [Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning](https://mlsanity.com/paper/2205.11568/) (2022)
- [Quantum Speedup of Natural Gradient for Variational Bayes](https://mlsanity.com/paper/2106.05807/) (2021)
- [Risk-averse Contextual Multi-armed Bandit Problem with Linear Payoffs](https://mlsanity.com/paper/2206.12463/) (2022)
- [Manifold Gaussian Variational Bayes on the Precision Matrix](https://mlsanity.com/paper/2210.14598/) (2022)
- [Enhancing Dynamical System Modeling through Interpretable Machine Learning Augmentations: A Case Study in Cathodic Electrophoretic Deposition](https://mlsanity.com/paper/2401.08414/) (2024)
- [Maximum a Posteriori Inference for Factor Graphs via Benders' Decomposition](https://mlsanity.com/paper/2410.19131/) (2024)
- [Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors](https://mlsanity.com/paper/2410.21862/) (2024)

## Abstract (arXiv)

This tutorial gives a quick introduction to Variational Bayes (VB), also called Variational Inference or Variational Approximation, from a practical point of view. The paper covers a range of commonly used VB methods and an attempt is made to keep the materials accessible to the wide community of data analysis practitioners. The aim is that the reader can quickly derive and implement their first VB algorithm for Bayesian inference with their data analysis problem. An end-user software package in Matlab together with the documentation can be found at https://vbayeslab.github.io/VBLabDocs/