# Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows

arXiv: https://arxiv.org/abs/2207.08219 · HTML: https://mlsanity.com/paper/2207.08219/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 17 Jul 2022.

## Builds on (most cited first, 25 of 25)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Parallel WaveNet: Fast High-Fidelity Speech Synthesis](https://mlsanity.com/paper/1711.10433/) (2017)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Neural Importance Sampling](https://mlsanity.com/paper/1808.03856/) (2018)
- [Flow-based generative models for Markov chain Monte Carlo in lattice field theory](https://mlsanity.com/paper/1904.12072/) (2019)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Equivariant flow-based sampling for lattice gauge theory](https://mlsanity.com/paper/2003.06413/) (2020)
- [Solving Statistical Mechanics Using Variational Autoregressive Networks](https://mlsanity.com/paper/1809.10606/) (2018)
- [Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference](https://mlsanity.com/paper/1703.09194/) (2017)
- [Sampling using SU(N) gauge equivariant flows](https://mlsanity.com/paper/2008.05456/) (2020)
- [VarGrad: A Low-Variance Gradient Estimator for Variational Inference](https://mlsanity.com/paper/2010.10436/) (2020)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives](https://mlsanity.com/paper/1810.04152/) (2018)
- [Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and Optimization](https://mlsanity.com/paper/2006.10343/) (2020)
- [Pathwise Derivatives Beyond the Reparameterization Trick](https://mlsanity.com/paper/1806.01851/) (2018)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Flow-based sampling for multimodal and extended-mode distributions in lattice field theory](https://mlsanity.com/paper/2107.00734/) (2021)
- [Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms](https://mlsanity.com/paper/1610.05683/) (2016)
- [On the Difficulty of Unbiased Alpha Divergence Minimization](https://mlsanity.com/paper/2010.09541/) (2020)
- [Generalized Doubly Reparameterized Gradient Estimators](https://mlsanity.com/paper/2101.11046/) (2021)
- [Path-Gradient Estimators for Continuous Normalizing Flows](https://mlsanity.com/paper/2206.09016/) (2022)
- [Empirical Evaluation of Biased Methods for Alpha Divergence Minimization](https://mlsanity.com/paper/2105.06587/) (2021)
- [Machine Learning of Thermodynamic Observables in the Presence of Mode Collapse](https://mlsanity.com/paper/2111.11303/) (2021)
- [Comment on "Solving Statistical Mechanics Using VANs": Introducing saVANt - VANs Enhanced by Importance and MCMC Sampling](https://mlsanity.com/paper/1903.11048/) (2019)

## Built on by (most cited first, 23 of 23)

- [Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization](https://mlsanity.com/paper/2310.02679/) (2023)
- [Learning Interpolations between Boltzmann Densities](https://mlsanity.com/paper/2301.07388/) (2023)
- [Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories](https://mlsanity.com/paper/2302.14082/) (2023)
- [Dynamical Measure Transport and Neural PDE Solvers for Sampling](https://mlsanity.com/paper/2407.07873/) (2024)
- [Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability](https://mlsanity.com/paper/2310.13402/) (2023)
- [Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows](https://mlsanity.com/paper/2310.20090/) (2023)
- [Energy based diffusion generator for efficient sampling of Boltzmann distributions](https://mlsanity.com/paper/2401.02080/) (2024)
- [Stable Training of Normalizing Flows for High-dimensional Variational Inference](https://mlsanity.com/paper/2402.16408/) (2024)
- [Practical applications of machine-learned flows on gauge fields](https://mlsanity.com/paper/2404.11674/) (2024)
- [Mitigating mode collapse in normalizing flows by annealing with an adaptive schedule: Application to parameter estimation](https://mlsanity.com/paper/2505.03652/) (2025)
- [Conditional Normalizing Flows for Active Learning of Coarse-Grained Molecular Representations](https://mlsanity.com/paper/2402.01195/) (2024)
- [Fast and Unified Path Gradient Estimators for Normalizing Flows](https://mlsanity.com/paper/2403.15881/) (2024)
- [Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixtures](https://mlsanity.com/paper/2602.12923/) (2026)
- [Path Gradients after Flow Matching](https://mlsanity.com/paper/2505.10139/) (2025)
- [Efficient Training of Boltzmann Generators Using Off-Policy Log-Dispersion Regularization](https://mlsanity.com/paper/2602.03729/) (2026)
- [Training normalizing flows with computationally intensive target probability distributions](https://mlsanity.com/paper/2308.13294/) (2023)
- [Multi-Lattice Sampling of Quantum Field Theories via Neural Operator-based Flows](https://mlsanity.com/paper/2401.00828/) (2024)
- [Mixed Noise and Posterior Estimation with Conditional DeepGEM](https://mlsanity.com/paper/2402.02964/) (2024)
- [Preferential Normalizing Flows](https://mlsanity.com/paper/2410.08710/) (2024)
- [NeuMC -- a package for neural sampling for lattice field theories](https://mlsanity.com/paper/2503.11482/) (2025)
- [Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies](https://mlsanity.com/paper/2604.23716/) (2026)
- [Mode Coverage in Normalizing Flow Boltzmann Generators via Log-Ratio Variation](https://mlsanity.com/paper/2609.09473/) (2026)
- [Scaling Online Optimal Transport Minibatches for Flow Matching](https://mlsanity.com/paper/2610.18838/) (2026)

## Abstract (arXiv)

We propose an algorithm to estimate the path-gradient of both the reverse and forward Kullback-Leibler divergence for an arbitrary manifestly invertible normalizing flow. The resulting path-gradient estimators are straightforward to implement, have lower variance, and lead not only to faster convergence of training but also to better overall approximation results compared to standard total gradient estimators. We also demonstrate that path-gradient training is less susceptible to mode-collapse. In light of our results, we expect that path-gradient estimators will become the new standard method to train normalizing flows for variational inference.