# Learning to Draw Samples with Amortized Stein Variational Gradient Descent

arXiv: https://arxiv.org/abs/1707.06626 · HTML: https://mlsanity.com/paper/1707.06626/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2017.

## Built on by (most cited first, 28 of 28)

- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Tractable Regularization of Probabilistic Circuits](https://mlsanity.com/paper/2106.02264/) (2021)
- [DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow](https://mlsanity.com/paper/2403.14966/) (2024)
- [Particle-based Variational Inference with Preconditioned Functional Gradient Flow](https://mlsanity.com/paper/2211.13954/) (2022)
- [Sliced Wasserstein Variational Inference](https://mlsanity.com/paper/2207.13177/) (2022)
- [S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic](https://mlsanity.com/paper/2405.00987/) (2024)
- [Sampling with Mirrored Stein Operators](https://mlsanity.com/paper/2106.12506/) (2021)
- [Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent](https://mlsanity.com/paper/2409.08469/) (2024)
- [Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent](https://mlsanity.com/paper/2305.14076/) (2023)
- [Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes](https://mlsanity.com/paper/2302.04534/) (2023)
- [Coin Sampling: Gradient-Based Bayesian Inference without Learning Rates](https://mlsanity.com/paper/2301.11294/) (2023)
- [Multi-Discriminator Sobolev Defense-GAN Against Adversarial Attacks for End-to-End Speech Systems](https://mlsanity.com/paper/2103.08086/) (2021)
- [Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference](https://mlsanity.com/paper/2105.05489/) (2021)
- [Stein Variational Gradient Descent with Multiple Kernel](https://mlsanity.com/paper/2107.09338/) (2021)
- [Finite-Particle Convergence Rates for Conservative and Non-Conservative Drifting Models](https://mlsanity.com/paper/2605.22795/) (2026)
- [A Semi-Bayesian Nonparametric Estimator of the Maximum Mean Discrepancy Measure: Applications in Goodness-of-Fit Testing and Generative Adversarial Networks](https://mlsanity.com/paper/2303.02637/) (2023)
- [Moreau-Yoshida Variational Transport: A General Framework For Solving Regularized Distributional Optimization Problems](https://mlsanity.com/paper/2307.16358/) (2023)
- [Uniform-in-time Propagation-of-Chaos for Stein Variational Gradient Descent](https://mlsanity.com/paper/2607.00149/) (2026)
- [Negotiated Reasoning: On Provably Addressing Relative Over-Generalization](https://mlsanity.com/paper/2306.05353/) (2023)
- [ELBOing Stein: Variational Bayes with Stein Mixture Inference](https://mlsanity.com/paper/2410.22948/) (2024)
- [RSD-GAN: Regularized Sobolev Defense GAN Against Speech-to-Text Adversarial Attacks](https://mlsanity.com/paper/2207.06858/) (2022)
- [Towards Task-Prioritized Policy Composition](https://mlsanity.com/paper/2209.09536/) (2022)
- [Posterior sampling with CNN-based, Plug-and-Play regularization with applications to Post-Stack Seismic Inversion](https://mlsanity.com/paper/2212.14595/) (2022)
- [Revisiting Unbiased Implicit Variational Inference](https://mlsanity.com/paper/2506.03839/) (2025)
- [Semi-Implicit Variational Inference via Kernelized Path Gradient Descent](https://mlsanity.com/paper/2506.05088/) (2025)
- [Branching Stein Variational Gradient Descent for sampling multimodal distributions](https://mlsanity.com/paper/2506.13916/) (2025)
- [Refinement-based Flow Policy Optimization](https://mlsanity.com/paper/2609.15123/) (2026)
- [Amortised Inference through One-Step Implicit Sampling](https://mlsanity.com/paper/2610.12679/) (2026)

## Abstract (arXiv)

We propose a simple algorithm to train stochastic neural networks to draw samples from given target distributions for probabilistic inference. Our method is based on iteratively adjusting the neural network parameters so that the output changes along a Stein variational gradient direction (Liu & Wang, 2016) that maximally decreases the KL divergence with the target distribution. Our method works for any target distribution specified by their unnormalized density function, and can train any black-box architectures that are differentiable in terms of the parameters we want to adapt. We demonstrate our method with a number of applications, including variational autoencoder (VAE) with expressive encoders to model complex latent space structures, and hyper-parameter learning of MCMC samplers that allows Bayesian inference to adaptively improve itself when seeing more data.