# Semi-Implicit Variational Inference

arXiv: https://arxiv.org/abs/1805.11183 · HTML: https://mlsanity.com/paper/1805.11183/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 30 of 55)

- [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities](https://mlsanity.com/paper/2302.01018/) (2023)
- [Probabilistic Conformal Prediction Using Conditional Random Samples](https://mlsanity.com/paper/2206.06584/) (2022)
- [Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step](https://mlsanity.com/paper/2410.14919/) (2024)
- [Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders](https://mlsanity.com/paper/2202.09671/) (2022)
- [SeeGera: Self-supervised Semi-implicit Graph Variational Auto-encoders with Masking](https://mlsanity.com/paper/2301.12458/) (2023)
- [Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models](https://mlsanity.com/paper/2409.11219/) (2024)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)
- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling](https://mlsanity.com/paper/2305.18375/) (2023)
- [Beta Diffusion](https://mlsanity.com/paper/2309.07867/) (2023)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [Latent State Marginalization as a Low-cost Approach for Improving Exploration](https://mlsanity.com/paper/2210.00999/) (2022)
- [Star-Shaped Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2302.05259/) (2023)
- [Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation](https://mlsanity.com/paper/2406.01561/) (2024)
- [Neural Markov Jump Processes](https://mlsanity.com/paper/2305.19744/) (2023)
- [Few-Step Diffusion via Score identity Distillation](https://mlsanity.com/paper/2505.12674/) (2025)
- [Efficient Semi-Implicit Variational Inference](https://mlsanity.com/paper/2101.06070/) (2021)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Structured Dropout Variational Inference for Bayesian Neural Networks](https://mlsanity.com/paper/2102.07927/) (2021)
- [Particle Semi-Implicit Variational Inference](https://mlsanity.com/paper/2407.00649/) (2024)
- [On Divergence Measures for Training GFlowNets](https://mlsanity.com/paper/2410.09355/) (2024)
- [Implicit Variational Inference for High-Dimensional Posteriors](https://mlsanity.com/paper/2310.06643/) (2023)
- [Wasserstein Gradient Flow over Variational Parameter Space for Variational Inference](https://mlsanity.com/paper/2310.16705/) (2023)
- [CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Models](https://mlsanity.com/paper/2512.07890/) (2025)
- [Interpolating between sampling and variational inference with infinite stochastic mixtures](https://mlsanity.com/paper/2110.09618/) (2021)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2206.05111/) (2022)
- [Diffusion Boosted Trees](https://mlsanity.com/paper/2406.01813/) (2024)
- [BayesCNS: A Unified Bayesian Approach to Address Cold Start and Non-Stationarity in Search Systems at Scale](https://mlsanity.com/paper/2410.02126/) (2024)

## Abstract (arXiv)

Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generated via reparameterization. Not only does SIVI expand the variational family to incorporate highly flexible variational distributions, including implicit ones that have no analytic density functions, but also sandwiches the evidence lower bound (ELBO) between a lower bound and an upper bound, and further derives an asymptotically exact surrogate ELBO that is amenable to optimization via stochastic gradient ascent. With a substantially expanded variational family and a novel optimization algorithm, SIVI is shown to closely match the accuracy of MCMC in inferring the posterior in a variety of Bayesian inference tasks.