# Particle Semi-Implicit Variational Inference

arXiv: https://arxiv.org/abs/2407.00649 · HTML: https://mlsanity.com/paper/2407.00649/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 62 references that are in the map.
- First posted on arXiv 30 Jun 2024.

## Builds on (most cited first, 28 of 28)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Maximum Mean Discrepancy Gradient Flow](https://mlsanity.com/paper/1906.04370/) (2019)
- [Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks](https://mlsanity.com/paper/1512.07666/) (2015)
- [Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference](https://mlsanity.com/paper/1703.09194/) (2017)
- [Variational inference via Wasserstein gradient flows](https://mlsanity.com/paper/2205.15902/) (2022)
- [Kernel Stein Discrepancy Descent](https://mlsanity.com/paper/2105.09994/) (2021)
- [Posterior Collapse and Latent Variable Non-identifiability](https://mlsanity.com/paper/2301.00537/) (2023)
- [Semi-Implicit Variational Inference](https://mlsanity.com/paper/1805.11183/) (2018)
- [Can Push-forward Generative Models Fit Multimodal Distributions?](https://mlsanity.com/paper/2206.14476/) (2022)
- [Variational Inference using Implicit Distributions](https://mlsanity.com/paper/1702.08235/) (2017)
- [Escaping Saddle Points with Adaptive Gradient Methods](https://mlsanity.com/paper/1901.09149/) (2019)
- [Unbiased Implicit Variational Inference](https://mlsanity.com/paper/1808.02078/) (2018)
- [MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splitting](https://mlsanity.com/paper/2306.08777/) (2023)
- [The Generalized Reparameterization Gradient](https://mlsanity.com/paper/1610.02287/) (2016)
- [Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2301.01766/) (2023)
- [Particle algorithms for maximum likelihood training of latent variable models](https://mlsanity.com/paper/2204.12965/) (2022)
- [Automatic Differentiation Variational Inference with Mixtures](https://mlsanity.com/paper/2003.01687/) (2020)
- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Sampling with Mollified Interaction Energy Descent](https://mlsanity.com/paper/2210.13400/) (2022)
- [Stochastic Backpropagation through Mixture Density Distributions](https://mlsanity.com/paper/1607.05690/) (2016)
- [Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities](https://mlsanity.com/paper/2403.02004/) (2024)
- [Momentum Particle Maximum Likelihood](https://mlsanity.com/paper/2312.07335/) (2023)
- [Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems](https://mlsanity.com/paper/2312.01127/) (2023)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)

## Built on by (most cited first, 7 of 7)

- [Training Latent Diffusion Models with Interacting Particle Algorithms](https://mlsanity.com/paper/2505.12412/) (2025)
- [Semi-Implicit Functional Gradient Flow for Efficient Sampling](https://mlsanity.com/paper/2410.17935/) (2024)
- [Particle Dynamics for Latent-Variable Energy-Based Models](https://mlsanity.com/paper/2510.15447/) (2025)
- [Revisiting Unbiased Implicit Variational Inference](https://mlsanity.com/paper/2506.03839/) (2025)
- [Semi-Implicit Variational Inference via Kernelized Path Gradient Descent](https://mlsanity.com/paper/2506.05088/) (2025)
- [A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Models](https://mlsanity.com/paper/2509.19276/) (2025)
- [Implicit Variational Rejection Sampling](https://mlsanity.com/paper/2606.14235/) (2026)

## Abstract (arXiv)

Semi-implicit variational inference (SIVI) enriches the expressiveness of variational families by utilizing a kernel and a mixing distribution to hierarchically define the variational distribution. Existing SIVI methods parameterize the mixing distribution using implicit distributions, leading to intractable variational densities. As a result, directly maximizing the evidence lower bound (ELBO) is not possible, so they resort to one of the following: optimizing bounds on the ELBO, employing costly inner-loop Markov chain Monte Carlo runs, or solving minimax objectives. In this paper, we propose a novel method for SIVI called Particle Variational Inference (PVI) which employs empirical measures to approximate the optimal mixing distributions characterized as the minimizer of a free energy functional. PVI arises naturally as a particle approximation of a Euclidean--Wasserstein gradient flow and, unlike prior works, it directly optimizes the ELBO whilst making no parametric assumption about the mixing distribution. Our empirical results demonstrate that PVI performs favourably compared to other SIVI methods across various tasks. Moreover, we provide a theoretical analysis of the behaviour of the gradient flow of a related free energy functional: establishing the existence and uniqueness of solutions as well as propagation of chaos results.