# Differentiating Metropolis-Hastings to Optimize Intractable Densities

arXiv: https://arxiv.org/abs/2306.07961 · HTML: https://mlsanity.com/paper/2306.07961/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 13 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Improved Contrastive Divergence Training of Energy Based Models](https://mlsanity.com/paper/2012.01316/) (2020)
- [Score-Based Diffusion meets Annealed Importance Sampling](https://mlsanity.com/paper/2208.07698/) (2022)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Designing Perceptual Puzzles by Differentiating Probabilistic Programs](https://mlsanity.com/paper/2204.12301/) (2022)
- [ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs](https://mlsanity.com/paper/2212.06386/) (2022)
- [Storchastic: A Framework for General Stochastic Automatic Differentiation](https://mlsanity.com/paper/2104.00428/) (2021)
- [Maximal couplings of the Metropolis-Hastings algorithm](https://mlsanity.com/paper/2010.08573/) (2020)
- [Acting as Inverse Inverse Planning](https://mlsanity.com/paper/2305.16913/) (2023)

## Built on by (most cited first, 6 of 6)

- [Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics](https://mlsanity.com/paper/2308.16680/) (2023)
- [On Feynman--Kac training of partial Bayesian neural networks](https://mlsanity.com/paper/2310.19608/) (2023)
- [A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing](https://mlsanity.com/paper/2607.16183/) (2026)
- [Gradient estimators for parameter inference in discrete stochastic kinetic models](https://mlsanity.com/paper/2604.02121/) (2026)
- [Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Method](https://mlsanity.com/paper/2601.13542/) (2026)
- [Learning Graph Energies for Finite-Time Generation](https://mlsanity.com/paper/2610.19946/) (2026)

## Abstract (arXiv)

We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers, allowing us to differentiate through probabilistic inference, even if the model has discrete components within it. Our approach fuses recent advances in stochastic automatic differentiation with traditional Markov chain coupling schemes, providing an unbiased and low-variance gradient estimator. This allows us to apply gradient-based optimization to objectives expressed as expectations over intractable target densities. We demonstrate our approach by finding an ambiguous observation in a Gaussian mixture model and by maximizing the specific heat in an Ising model.