# A unified view of likelihood ratio and reparameterization gradients and an optimal importance sampling scheme

arXiv: https://arxiv.org/abs/1910.06419 · HTML: https://mlsanity.com/paper/1910.06419/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2019.

## Built on by (most cited first, 2 of 2)

- [A unified view of likelihood ratio and reparameterization gradients](https://mlsanity.com/paper/2105.14900/) (2021)
- [Low-Variance Gradient Estimation in Unrolled Computation Graphs with ES-Single](https://mlsanity.com/paper/2304.11153/) (2023)

## Abstract (arXiv)

Reparameterization (RP) and likelihood ratio (LR) gradient estimators are used throughout machine and reinforcement learning; however, they are usually explained as simple mathematical tricks without providing any insight into their nature. We use a first principles approach to explain LR and RP, and show a connection between the two via the divergence theorem. The theory motivated us to derive optimal importance sampling schemes to reduce LR gradient variance. Our newly derived distributions have analytic probability densities and can be directly sampled from. The improvement for Gaussian target distributions was modest, but for other distributions such as a Beta distribution, our method could lead to arbitrarily large improvements, and was crucial to obtain competitive performance in evolution strategies experiments.