# On the Convergence of Stochastic Variational Inference in Bayesian Networks

arXiv: https://arxiv.org/abs/1507.04505 · HTML: https://mlsanity.com/paper/1507.04505/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2015.

## Built on by (most cited first, 1 of 1)

- [Variational Bayes Made Easy](https://mlsanity.com/paper/2304.14251/) (2023)

## Abstract (arXiv)

We highlight a pitfall when applying stochastic variational inference to general Bayesian networks. For global random variables approximated by an exponential family distribution, natural gradient steps, commonly starting from a unit length step size, are averaged to convergence. This useful insight into the scaling of initial step sizes is lost when the approximation factorizes across a general Bayesian network, and care must be taken to ensure practical convergence. We experimentally investigate how much of the baby (well-scaled steps) is thrown out with the bath water (exact gradients).