mlsanityData to 14 Oct 2026

Efficient Semi-Implicit Variational Inference

arXiv 2101.06070 ↗First posted 15 Jan 2021cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose CI-VI an efficient and scalable solver for semi-implicit variational inference (SIVI). Our method, first, maps SIVI's evidence lower bound (ELBO) to a form involving a nonlinear functional nesting of expected values and then develops a rigorous optimiser capable of correctly handling bias inherent to nonlinear nested expectations using an extrapolation-smoothing mechanism coupled with gradient sketching. Our theoretical results demonstrate convergence to a stationary point of the ELBO in general non-convex settings typically arising when using deep network models and an order of O(t^-frac45) gradient-bias-vanishing rate. We believe these results generalise beyond the specific nesting arising from SIVI to other forms. Finally, in a set of experiments, we demonstrate the effectiveness of our algorithm in approximating complex posteriors on various data-sets including those from natural language processing.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 15 Jan 2021.

Builds on

30 of 48 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2015Variational Inference with Normalizing Flowscited by 2,170
  5. 2016Variational Inference: A Review for Statisticianscited by 1,472
  6. 2019On the Convergence of Adam and Beyondcited by 841
  7. 2018SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimatorcited by 287
  8. 2017How to Escape Saddle Points Efficientlycited by 285
  9. 2017A Conceptual Introduction to Hamiltonian Monte Carlocited by 209
  10. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 191
  11. 2016Learning in Implicit Generative Modelscited by 125
  12. 2014Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functionscited by 95
18 more
  1. 2015Hierarchical Variational Modelscited by 89
  2. 2017On Nesting Monte Carlo Estimatorscited by 83
  3. 2016Auxiliary Deep Generative Modelscited by 72
  4. 2016Variance Reduction for Faster Non-Convex Optimizationcited by 72
  5. 2017Gradient Estimators for Implicit Modelscited by 58
  6. 2018Semi-Implicit Variational Inferencecited by 55
  7. 2017Variants of RMSProp and Adagrad with Logarithmic Regret Boundscited by 50
  8. 2016Accelerating Stochastic Composition Optimizationcited by 46
  9. 2017Variational Inference using Implicit Distributionscited by 39
  10. 2018Unbiased Implicit Variational Inferencecited by 33
  11. 2015The Variational Gaussian Processcited by 30
  12. 2015Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayescited by 14
  13. 2015Variational Gaussian Copula Inferencecited by 12
  14. 2019Multi-View Reinforcement Learningcited by 12
  15. 2020Compositional ADAM: An Adaptive Compositional Solvercited by 8
  16. 2018Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problemcited by 4
  17. 2016A Distributed Newton Method for Large Scale Consensus Optimizationcited by 2
  18. 2019Derivative-Free & Order-Robust Optimisationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 2; 2025: 1; 2026: 2.

9 papers.

  1. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 214
  2. 2023Semi-Implicit Variational Inference via Score Matchingcited by 19
  3. 2023Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Accelerationcited by 14
  4. 2024Kernel Semi-Implicit Variational Inferencecited by 8
  5. 2022VFDS: Variational Foresight Dynamic Selection in Bayesian Neural Networks for Efficient Human Activity Recognitioncited by 1
  6. 2026A Kernel Approach for Semi-implicit Variational Inferencecited by 1
  7. 2022Bayesian Neural Network Inference via Implicit Models and the Posterior Predictive Distributioncited by 0
  8. 2025Continuous Semi-Implicit Modelscited by 0
  9. 2026Implicit Variational Rejection Samplingcited by 0