# Stochastic gradient variational Bayes for gamma approximating distributions

arXiv: https://arxiv.org/abs/1509.01631 · HTML: https://mlsanity.com/paper/1509.01631/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2015.

## Built on by (most cited first, 9 of 9)

- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [DeepStability: A Study of Unstable Numerical Methods and Their Solutions in Deep Learning](https://mlsanity.com/paper/2202.03493/) (2022)
- [A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Prediction](https://mlsanity.com/paper/2110.03446/) (2021)
- [Viking: Variational Bayesian Variance Tracking](https://mlsanity.com/paper/2104.10777/) (2021)
- [A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck](https://mlsanity.com/paper/2207.13529/) (2022)
- [Enhancing Uncertainty Estimation and Interpretability via Bayesian Non-negative Decision Layer](https://mlsanity.com/paper/2505.22199/) (2025)
- [Low Complexity Approximate Bayesian Logistic Regression for Sparse Online Learning](https://mlsanity.com/paper/2101.12113/) (2021)
- [Likelihood-based inference and forecasting for trawl processes: a stochastic optimization approach](https://mlsanity.com/paper/2308.16092/) (2023)
- [PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM](https://mlsanity.com/paper/2607.24583/) (2026)

## Abstract (arXiv)

While stochastic variational inference is relatively well known for scaling inference in Bayesian probabilistic models, related methods also offer ways to circumnavigate the approximation of analytically intractable expectations. The key challenge in either setting is controlling the variance of gradient estimates: recent work has shown that for continuous latent variables, particularly multivariate Gaussians, this can be achieved by using the gradient of the log posterior. In this paper we apply the same idea to gamma distributed latent variables given gamma variational distributions, enabling straightforward "black box" variational inference in models where sparsity and non-negativity are appropriate. We demonstrate the method on a recently proposed gamma process model for network data, as well as a novel sparse factor analysis. We outperform generic sampling algorithms and the approach of using Gaussian variational distributions on transformed variables.