# A trust-region method for stochastic variational inference with applications to streaming data

arXiv: https://arxiv.org/abs/1505.07649 · HTML: https://mlsanity.com/paper/1505.07649/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2015.

## Built on by (most cited first, 9 of 9)

- [A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Models](https://mlsanity.com/paper/2209.11533/) (2022)
- [Batch and match: black-box variational inference with a score-based divergence](https://mlsanity.com/paper/2402.14758/) (2024)
- [Partitioned Variational Inference: A Framework for Probabilistic Federated Learning](https://mlsanity.com/paper/2202.12275/) (2022)
- [Optimization Guarantees for Square-Root Natural-Gradient Variational Inference](https://mlsanity.com/paper/2507.07853/) (2025)
- [Why GANs are overkill for NLP](https://mlsanity.com/paper/2205.09838/) (2022)
- [VISA: Variational Inference with Sequential Sample-Average Approximations](https://mlsanity.com/paper/2403.09429/) (2024)
- [Large-scale Score-based Variational Posterior Inference for Bayesian Deep Neural Networks](https://mlsanity.com/paper/2602.05873/) (2026)
- [Orthogonal Polynomials Approximation Algorithm (OPAA):a functional analytic approach to estimating probability densities](https://mlsanity.com/paper/2211.08594/) (2022)
- [Recursive Entropic Variational Inference for Nonlinear State-Space Models](https://mlsanity.com/paper/2511.15409/) (2025)

## Abstract (arXiv)

Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradient step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.