# Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning

arXiv: https://arxiv.org/abs/2205.11568 · HTML: https://mlsanity.com/paper/2205.11568/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 19 of 19)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [Practical Deep Learning with Bayesian Principles](https://mlsanity.com/paper/1906.02506/) (2019)
- [Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam](https://mlsanity.com/paper/1806.04854/) (2018)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Temporal Attention augmented Bilinear Network for Financial Time-Series Data Analysis](https://mlsanity.com/paper/1712.00975/) (2017)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [Handling the Positive-Definite Constraint in the Bayesian Learning Rule](https://mlsanity.com/paper/2002.10060/) (2020)
- [Automatic Variational Inference in Stan](https://mlsanity.com/paper/1506.03431/) (2015)
- [Gaussian variational approximation with sparse precision matrices](https://mlsanity.com/paper/1605.05622/) (2016)
- [An Introduction to Variational Inference](https://mlsanity.com/paper/2108.13083/) (2021)
- [A practical tutorial on Variational Bayes](https://mlsanity.com/paper/2103.01327/) (2021)
- [Variational Bayes on Manifolds](https://mlsanity.com/paper/1908.03097/) (2019)
- [Variational Bayes with Synthetic Likelihood](https://mlsanity.com/paper/1608.03069/) (2016)

## Built on by (most cited first, 3 of 3)

- [Bayesian Learning for Neural Networks: an algorithmic survey](https://mlsanity.com/paper/2211.11865/) (2022)
- [Manifold Gaussian Variational Bayes on the Precision Matrix](https://mlsanity.com/paper/2210.14598/) (2022)
- [Variational Inference for GARCH-family Models](https://mlsanity.com/paper/2310.03435/) (2023)

## Abstract (arXiv)

We develop an optimization algorithm suitable for Bayesian learning in complex models. Our approach relies on natural gradient updates within a general black-box framework for efficient training with limited model-specific derivations. It applies within the class of exponential-family variational posterior distributions, for which we extensively discuss the Gaussian case for which the updates have a rather simple form. Our Quasi Black-box Variational Inference (QBVI) framework is readily applicable to a wide class of Bayesian inference problems and is of simple implementation as the updates of the variational posterior do not involve gradients with respect to the model parameters, nor the prescription of the Fisher information matrix. We develop QBVI under different hypotheses for the posterior covariance matrix, discuss details about its robust and feasible implementation, and provide a number of real-world applications to demonstrate its effectiveness.