mlsanityData to 7 Oct 2026

Sample Average Approximation for Black-Box VI

arXiv 2304.06803 ↗First posted 13 Apr 2023cs.LG math.OC stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

We present a novel approach for black-box VI that bypasses the difficulties of stochastic gradient ascent, including the task of selecting step-sizes. Our approach involves using a sequence of sample average approximation (SAA) problems. SAA approximates the solution of stochastic optimization problems by transforming them into deterministic ones. We use quasi-Newton methods and line search to solve each deterministic optimization problem and present a heuristic policy to automate hyperparameter selection. Our experiments show that our method simplifies the VI problem and achieves faster performance than existing methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 22 of its 61 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

22 of 61 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019An Introduction to Variational Autoencoderscited by 997
  4. 2015Importance Weighted Autoencoderscited by 454
  5. 2013Black Box Variational Inferencecited by 317
  6. 2016Automatic Differentiation Variational Inferencecited by 159
  7. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  8. 2014A Stochastic Quasi-Newton Method for Large-Scale Optimizationcited by 68
  9. 2018Importance Weighting and Variational Inferencecited by 55
  10. 2018A Swiss Army Infinitesimal Jackknifecited by 51
  11. 2017Covariances, Robustness, and Variational Bayescited by 39
  12. 2020Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimizationcited by 37
10 more
  1. 2018Pathwise Derivatives Beyond the Reparameterization Trickcited by 36
  2. 2013Automated Variational Inference in Probabilistic Programmingcited by 31
  3. 2021Pathfinder: Parallel quasi-Newton variational inferencecited by 26
  4. 2013PEGASUS: A Policy Search Method for Large MDPs and POMDPscited by 21
  5. 2018Using Large Ensembles of Control Variates for Variational Inferencecited by 21
  6. 2023Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Boxcited by 17
  7. 2015Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayescited by 14
  8. 2019Divide and Couple: Using Monte Carlo Variational Objectives for Posterior Approximationcited by 14
  9. 2021Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametricscited by 4
  10. 2023U-Statistics for Importance-Weighted Variational Inferencecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Batch and match: black-box variational inference with a score-based divergencecited by 22
  2. 2024VISA: Variational Inference with Sequential Sample-Average Approximationscited by 1
  3. 2024Understanding and mitigating difficulties in posterior predictive evaluationcited by 1
  4. 2026Large-scale Score-based Variational Posterior Inference for Bayesian Deep Neural Networkscited by 1
  5. 2025Online Robust Planning under Model Uncertainty: A Sample-Based Approachcited by 0
  6. 2025AutoGD: Automatic Learning Rate Selection for Gradient Descentcited by 0