# VISA: Variational Inference with Sequential Sample-Average Approximations

arXiv: https://arxiv.org/abs/2403.09429 · HTML: https://mlsanity.com/paper/2403.09429/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 26 of 26)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [A Stochastic Quasi-Newton Method for Large-Scale Optimization](https://mlsanity.com/paper/1401.7020/) (2014)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Elements of Sequential Monte Carlo](https://mlsanity.com/paper/1903.04797/) (2019)
- [An Introduction to Probabilistic Programming](https://mlsanity.com/paper/1809.10756/) (2018)
- [Markovian Score Climbing: Variational Inference with KL(p||q)](https://mlsanity.com/paper/2003.10374/) (2020)
- [Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives](https://mlsanity.com/paper/1810.04152/) (2018)
- [Covariances, Robustness, and Variational Bayes](https://mlsanity.com/paper/1709.02536/) (2017)
- [Automated Variational Inference in Probabilistic Programming](https://mlsanity.com/paper/1301.1299/) (2013)
- [Pathfinder: Parallel quasi-Newton variational inference](https://mlsanity.com/paper/2108.03782/) (2021)
- [Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box](https://mlsanity.com/paper/2304.05527/) (2023)
- [Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes](https://mlsanity.com/paper/1506.04088/) (2015)
- [Generalized Doubly Reparameterized Gradient Estimators](https://mlsanity.com/paper/2101.11046/) (2021)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [A trust-region method for stochastic variational inference with applications to streaming data](https://mlsanity.com/paper/1505.07649/) (2015)
- [Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport](https://mlsanity.com/paper/2202.01841/) (2022)
- [Fast Black-box Variational Inference through Stochastic Trust-Region Optimization](https://mlsanity.com/paper/1706.02375/) (2017)
- [Amortized Population Gibbs Samplers with Neural Sufficient Statistics](https://mlsanity.com/paper/1911.01382/) (2019)
- [Learning Proposals for Probabilistic Programs with Inference Combinators](https://mlsanity.com/paper/2103.00668/) (2021)
- [Sample Average Approximation for Black-Box VI](https://mlsanity.com/paper/2304.06803/) (2023)
- [Bayesian Optimization for Probabilistic Programs](https://mlsanity.com/paper/1707.04314/) (2017)
- [Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow](https://mlsanity.com/paper/1805.10469/) (2018)
- [Sparse Bayesian Lasso via a Variable-Coefficient ℓ1 Penalty](https://mlsanity.com/paper/2211.05089/) (2022)
- [Approximate Variational Inference Based on a Finite Sample of Gaussian Latent Variables](https://mlsanity.com/paper/1906.04507/) (2019)

## Built on by (most cited first, 1 of 1)

- [Control, Optimal Transport and Neural Differential Equations in Supervised Learning](https://mlsanity.com/paper/2503.15105/) (2025)

## Abstract (arXiv)

We present variational inference with sequential sample-average approximation (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.