# Straight-Through Estimator as Projected Wasserstein Gradient Flow

arXiv: https://arxiv.org/abs/1910.02176 · HTML: https://mlsanity.com/paper/1910.02176/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2019.

## Built on by (most cited first, 7 of 7)

- [A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification](https://mlsanity.com/paper/2205.07877/) (2022)
- [Encoding Time-Series Explanations through Self-Supervised Model Behavior Consistency](https://mlsanity.com/paper/2306.02109/) (2023)
- [AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging](https://mlsanity.com/paper/2211.07818/) (2022)
- [High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator](https://mlsanity.com/paper/2510.10693/) (2025)
- [Straight-Through meets Sparse Recovery: the Support Exploration Algorithm](https://mlsanity.com/paper/2301.13584/) (2023)
- [AdaTriplet-RA: Domain Matching via Adaptive Triplet and Reinforced Attention for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2211.08894/) (2022)
- [Interpretable Classification via a Rule Network with Selective Logical Operators](https://mlsanity.com/paper/2408.11918/) (2024)

## Abstract (arXiv)

The Straight-Through (ST) estimator is a widely used technique for back-propagating gradients through discrete random variables. However, this effective method lacks theoretical justification. In this paper, we show that ST can be interpreted as the simulation of the projected Wasserstein gradient flow (pWGF). Based on this understanding, a theoretical foundation is established to justify the convergence properties of ST. Further, another pWGF estimator variant is proposed, which exhibits superior performance on distributions with infinite support,e.g., Poisson distributions. Empirically, we show that ST and our proposed estimator, while applied to different types of discrete structures (including both Bernoulli and Poisson latent variables), exhibit comparable or even better performances relative to other state-of-the-art methods. Our results uncover the origin of the widespread adoption of the ST estimator and represent a helpful step towards exploring alternative gradient estimators for discrete variables.