# Reparameterization Gradient for Non-differentiable Models

arXiv: https://arxiv.org/abs/1806.00176 · HTML: https://mlsanity.com/paper/1806.00176/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2018.

## Built on by (most cited first, 11 of 11)

- [A unified view of likelihood ratio and reparameterization gradients](https://mlsanity.com/paper/2105.14900/) (2021)
- [Model-free Policy Learning with Reward Gradients](https://mlsanity.com/paper/2103.05147/) (2021)
- [Zero Grads: Learning Local Surrogate Losses for Non-Differentiable Graphics](https://mlsanity.com/paper/2308.05739/) (2023)
- [Probabilistic Programming with Programmable Variational Inference](https://mlsanity.com/paper/2406.15742/) (2024)
- [ωPAP Spaces: Reasoning Denotationally About Higher-Order, Recursive Probabilistic and Differentiable Programs](https://mlsanity.com/paper/2302.10636/) (2023)
- [Revisiting the Gumbel-Softmax in MADDPG](https://mlsanity.com/paper/2302.11793/) (2023)
- [Safe Reinforcement Learning using Finite-Horizon Gradient-based Estimation](https://mlsanity.com/paper/2412.11138/) (2024)
- [Fast and Correct Gradient-Based Optimisation for Probabilistic Programming via Smoothing](https://mlsanity.com/paper/2301.03415/) (2023)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing](https://mlsanity.com/paper/2402.11752/) (2024)
- [Does "Do Differentiable Simulators Give Better Policy Gradients?'' Give Better Policy Gradients?](https://mlsanity.com/paper/2604.18161/) (2026)

## Abstract (arXiv)

We present a new algorithm for stochastic variational inference that targets at models with non-differentiable densities. One of the key challenges in stochastic variational inference is to come up with a low-variance estimator of the gradient of a variational objective. We tackle the challenge by generalizing the reparameterization trick, one of the most effective techniques for addressing the variance issue for differentiable models, so that the trick works for non-differentiable models as well. Our algorithm splits the space of latent variables into regions where the density of the variables is differentiable, and their boundaries where the density may fail to be differentiable. For each differentiable region, the algorithm applies the standard reparameterization trick and estimates the gradient restricted to the region. For each potentially non-differentiable boundary, it uses a form of manifold sampling and computes the direction for variational parameters that, if followed, would increase the boundary's contribution to the variational objective. The sum of all the estimates becomes the gradient estimate of our algorithm. Our estimator enjoys the reduced variance of the reparameterization gradient while remaining unbiased even for non-differentiable models. The experiments with our preliminary implementation confirm the benefit of reduced variance and unbiasedness.