mlsanityData to 7 Oct 2026

Particle Semi-Implicit Variational Inference

arXiv 2407.00649 ↗First posted 30 Jun 2024stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Semi-implicit variational inference (SIVI) enriches the expressiveness of variational families by utilizing a kernel and a mixing distribution to hierarchically define the variational distribution. Existing SIVI methods parameterize the mixing distribution using implicit distributions, leading to intractable variational densities. As a result, directly maximizing the evidence lower bound (ELBO) is not possible, so they resort to one of the following: optimizing bounds on the ELBO, employing costly inner-loop Markov chain Monte Carlo runs, or solving minimax objectives. In this paper, we propose a novel method for SIVI called Particle Variational Inference (PVI) which employs empirical measures to approximate the optimal mixing distributions characterized as the minimizer of a free energy functional. PVI arises naturally as a particle approximation of a Euclidean--Wasserstein gradient flow and, unlike prior works, it directly optimizes the ELBO whilst making no parametric assumption about the mixing distribution. Our empirical results demonstrate that PVI performs favourably compared to other SIVI methods across various tasks. Moreover, we provide a theoretical analysis of the behaviour of the gradient flow of a related free energy functional: establishing the existence and uniqueness of solutions as well as propagation of chaos results.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 62 references that are in the map. First posted on arXiv 30 Jun 2024.

Builds on

28 of 62 references.

References · 28

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2016Variational Inference: A Review for Statisticianscited by 1,466
  3. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  4. 2016Automatic Differentiation Variational Inferencecited by 159
  5. 2015Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measurescited by 138
  6. 2019Maximum Mean Discrepancy Gradient Flowcited by 123
  7. 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
  8. 2017Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inferencecited by 97
  9. 2022Variational inference via Wasserstein gradient flowscited by 84
  10. 2021Kernel Stein Discrepancy Descentcited by 57
  11. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  12. 2018Semi-Implicit Variational Inferencecited by 55
16 more
  1. 2022Can Push-forward Generative Models Fit Multimodal Distributions?cited by 41
  2. 2017Variational Inference using Implicit Distributionscited by 39
  3. 2019Escaping Saddle Points with Adaptive Gradient Methodscited by 34
  4. 2018Unbiased Implicit Variational Inferencecited by 33
  5. 2023MMD-FUSE: Learning and Combining Kernels for Two-Sample Testing Without Data Splittingcited by 33
  6. 2016The Generalized Reparameterization Gradientcited by 30
  7. 2022Particle algorithms for maximum likelihood training of latent variable modelscited by 26
  8. 2023Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flowcited by 25
  9. 2020Automatic Differentiation Variational Inference with Mixturescited by 19
  10. 2023Semi-Implicit Variational Inference via Score Matchingcited by 19
  11. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  12. 2016Stochastic Backpropagation through Mixture Density Distributionscited by 14
  13. 2024Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalitiescited by 13
  14. 2023Momentum Particle Maximum Likelihoodcited by 12
  15. 2023Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problemscited by 11
  16. 2024Kernel Semi-Implicit Variational Inferencecited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 1.

7 papers.

  1. 2025Training Latent Diffusion Models with Interacting Particle Algorithmscited by 4
  2. 2024Semi-Implicit Functional Gradient Flow for Efficient Samplingcited by 1
  3. 2025Revisiting Unbiased Implicit Variational Inferencecited by 0
  4. 2025Semi-Implicit Variational Inference via Kernelized Path Gradient Descentcited by 0
  5. 2025A Gradient Flow Approach to Solving Inverse Problems with Latent Diffusion Modelscited by 0
  6. 2025Particle Dynamics for Latent-Variable Energy-Based Modelscited by 0
  7. 2026Implicit Variational Rejection Samplingcited by 0