# Metropolis-Hastings view on variational inference and adversarial training

arXiv: https://arxiv.org/abs/1810.07151 · HTML: https://mlsanity.com/paper/1810.07151/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2018.

## Built on by (most cited first, 6 of 6)

- [Optimal Preconditioning and Fisher Adaptive Langevin Sampling](https://mlsanity.com/paper/2305.14442/) (2023)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [AutoStep: Locally adaptive involutive MCMC](https://mlsanity.com/paper/2410.18929/) (2024)
- [Semi-Empirical Objective Functions for MCMC Proposal Optimization](https://mlsanity.com/paper/2106.02104/) (2021)
- [Variational Rejection Particle Filtering](https://mlsanity.com/paper/2103.15343/) (2021)
- [Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems](https://mlsanity.com/paper/2610.02377/) (2026)

## Abstract (arXiv)

A significant part of MCMC methods can be considered as the Metropolis-Hastings (MH) algorithm with different proposal distributions. From this point of view, the problem of constructing a sampler can be reduced to the question - how to choose a proposal for the MH algorithm? To address this question, we propose to learn an independent sampler that maximizes the acceptance rate of the MH algorithm, which, as we demonstrate, is highly related to the conventional variational inference. For Bayesian inference, the proposed method compares favorably against alternatives to sample from the posterior distribution. Under the same approach, we step beyond the scope of classical MCMC methods and deduce the Generative Adversarial Networks (GANs) framework from scratch, treating the generator as the proposal and the discriminator as the acceptance test. On real-world datasets, we improve Frechet Inception Distance and Inception Score, using different GANs as a proposal distribution for the MH algorithm. In particular, we demonstrate improvements of recently proposed BigGAN model on ImageNet.