mlsanityData to 7 Oct 2026

Gradient Estimation with Discrete Stein Operators

arXiv 2202.09497 ↗First posted 19 Feb 2022stat.ML cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 33 of its 73 references that are in the map. First posted on arXiv 19 Feb 2022.

Builds on

33 of 73 references.

References · 33

  1. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  6. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  7. 2015Importance Weighted Autoencoderscited by 454
  8. 2013Black Box Variational Inferencecited by 317
  9. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 281
  10. 2015Gradient Estimation Using Stochastic Computation Graphscited by 120
  11. 2014Neural Variational Inference and Learning in Belief Networkscited by 116
  12. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
21 more
  1. 2021Implicit MLE: Backpropagating Through Discrete Exponential Family Distributionscited by 97
  2. 2015Measuring Sample Quality with Stein's Methodcited by 94
  3. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  4. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 84
  5. 2016Variational inference for Monte Carlo objectivescited by 83
  6. 2017REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelscited by 78
  7. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  8. 2014Control functionals for Monte Carlo integrationcited by 59
  9. 2019Minimum Stein Discrepancy Estimatorscited by 57
  10. 2015MuProp: Unbiased Backpropagation for Stochastic Neural Networkscited by 36
  11. 2018Pathwise Derivatives Beyond the Reparameterization Trickcited by 36
  12. 2020Estimating Gradients for Discrete Random Variables by Sampling without Replacementcited by 29
  13. 2020DisARM: An Antithetic Gradient Estimator for Binary Latent Variablescited by 27
  14. 2020Probabilistic Best Subset Selection via Gradient-Based Optimizationcited by 17
  15. 2021A unified view of likelihood ratio and reparameterization gradientscited by 12
  16. 2020The reproducing Stein kernel approach for post-hoc corrected samplingcited by 10
  17. 2020Scalable Control Variates for Monte Carlo Methods via Stochastic Optimizationcited by 10
  18. 2021Coupled Gradient Estimators for Discrete Latent Variablescited by 9
  19. 2021ARMS: Antithetic-REINFORCE-Multi-Sample Gradient for Binary Variablescited by 8
  20. 2021Double Control Variates for Gradient Estimation in Discrete Latent Variable Modelscited by 8
  21. 2019GO Gradient for Expectation-Based Objectivescited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 8; 2024: 6; 2025: 5.

22 papers.

Sort:

By area:

  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  3. 2023Bridging Discrete and Backpropagation: Straight-Through and Beyondcited by 31
  4. 2025Learning-Order Autoregressive Models with Application to Molecular Graph Generationcited by 29
  5. 2023Sparse Backpropagation for MoE Trainingcited by 16
  6. 2022Generalised Bayesian Inference for Discrete Intractable Likelihoodcited by 14
  7. 2023Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trickcited by 13
  8. 2025Discrete Neural Flow Samplers with Locally Equivariant Transformercited by 13
  9. 2024Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Modelcited by 11
  10. 2022A kernel Stein test of goodness of fit for sequential modelscited by 9
  11. 2022Gradient Estimation for Binary Latent Variables via Gradient Variance Clippingcited by 7
  12. 2023Meta-learning Control Variates: Variance Reduction with Limited Datacited by 6
10 more
  1. 2024On Divergence Measures for Training GFlowNetscited by 6
  2. 2023Adaptive Instrument Design for Indirect Experimentscited by 5
  3. 2023When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimalitycited by 4
  4. 2023Bayesian Numerical Integration with Neural Networkscited by 3
  5. 2025Secrets of GFlowNets' Learning Behavior: A Theoretical Studycited by 2
  6. 2023DBsurf: A Discrepancy Based Method for Discrete Stochastic Gradient Estimationcited by 0
  7. 2024What to Do When Your Discrete Optimization Is the Size of a Neural Network?cited by 0
  8. 2024Scalable Monte Carlo for Bayesian Learningcited by 0
  9. 2025Variational phylogenetic inference with products over bipartitionscited by 0
  10. 2025BayesSum: Bayesian Quadrature in Discrete Spacescited by 0