# Variational Inference using Implicit Distributions

arXiv: https://arxiv.org/abs/1702.08235 · HTML: https://mlsanity.com/paper/1702.08235/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2017.

## Built on by (most cited first, 30 of 39)

- [Estimating Epistemic and Aleatoric Uncertainty with a Single Model](https://mlsanity.com/paper/2402.03478/) (2024)
- [Causal Confusion and Reward Misidentification in Preference-Based Reward Learning](https://mlsanity.com/paper/2204.06601/) (2022)
- [Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs](https://mlsanity.com/paper/2310.02619/) (2023)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders](https://mlsanity.com/paper/2202.09671/) (2022)
- [GATSBI: Generative Adversarial Training for Simulation-Based Inference](https://mlsanity.com/paper/2203.06481/) (2022)
- [Is MC Dropout Bayesian?](https://mlsanity.com/paper/2110.04286/) (2021)
- [The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models](https://mlsanity.com/paper/2204.10759/) (2022)
- [Adversarial and Contrastive Variational Autoencoder for Sequential Recommendation](https://mlsanity.com/paper/2103.10693/) (2021)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)
- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Variational inference with a quantum computer](https://mlsanity.com/paper/2103.06720/) (2021)
- [Learning to Jump: Thinning and Thickening Latent Counts for Generative Modeling](https://mlsanity.com/paper/2305.18375/) (2023)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)
- [Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility](https://mlsanity.com/paper/2207.04895/) (2022)
- [Neural Markov Jump Processes](https://mlsanity.com/paper/2305.19744/) (2023)
- [Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows](https://mlsanity.com/paper/2205.07918/) (2022)
- [Efficient Semi-Implicit Variational Inference](https://mlsanity.com/paper/2101.06070/) (2021)
- [DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision](https://mlsanity.com/paper/2502.08005/) (2025)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Gromov-Wasserstein Autoencoders](https://mlsanity.com/paper/2209.07007/) (2022)
- [Particle Semi-Implicit Variational Inference](https://mlsanity.com/paper/2407.00649/) (2024)
- [Generative Bayesian Inference with GANs](https://mlsanity.com/paper/2208.12113/) (2022)
- [Neural Operator Variational Inference based on Regularized Stein Discrepancy for Deep Gaussian Processes](https://mlsanity.com/paper/2309.12658/) (2023)
- [Implicit Variational Inference for High-Dimensional Posteriors](https://mlsanity.com/paper/2310.06643/) (2023)
- [Symmetric Equilibrium Learning of VAEs](https://mlsanity.com/paper/2307.09883/) (2023)
- [Learning beyond sensations: how dreams organize neuronal representations](https://mlsanity.com/paper/2308.01830/) (2023)
- [Quantifying Assistive Robustness Via the Natural-Adversarial Frontier](https://mlsanity.com/paper/2310.10610/) (2023)

## Abstract (arXiv)

Generative adversarial networks (GANs) have given us a great tool to fit implicit generative models to data. Implicit distributions are ones we can sample from easily, and take derivatives of samples with respect to model parameters. These models are highly expressive and we argue they can prove just as useful for variational inference (VI) as they are for generative modelling. Several papers have proposed GAN-like algorithms for inference, however, connections to the theory of VI are not always well understood. This paper provides a unifying review of existing algorithms establishing connections between variational autoencoders, adversarially learned inference, operator VI, GAN-based image reconstruction, and more. Secondly, the paper provides a framework for building new algorithms: depending on the way the variational bound is expressed we introduce prior-contrastive and joint-contrastive methods, and show practical inference algorithms based on either density ratio estimation or denoising.