mlsanityData to 14 Oct 2026

Automatic Differentiation of Programs with Discrete Randomness

arXiv 2210.08572 ↗First posted 16 Oct 2022cs.LG cs.MS cs.NAcited by 29

Not on the map. Why?

Abstract (arXiv)

Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded by gradient-based optimization. However, AD systems have been restricted to the subset of programs that have a continuous dependence on parameters. Programs that have discrete stochastic behaviors governed by distribution parameters, such as flipping a coin with probability p of being heads, pose a challenge to these systems because the connection between the result (heads vs tails) and the parameters (p) is fundamentally discrete. In this paper we develop a new reparameterization-based methodology that allows for generating programs whose expectation is the derivative of the expectation of the original program. We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway's Game of Life, and unbiased reverse-mode AD of a particle filter. Our code package is available at https://github.com/gaurav-arya/StochasticAD.jl.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 18 of its 60 references that are in the map. First posted on arXiv 16 Oct 2022.

Builds on

18 of 60 references.

References · 18

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  3. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  4. 2015Automatic differentiation in machine learning: a surveycited by 855
  5. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  6. 2015Gradient Estimation Using Stochastic Computation Graphscited by 121
  7. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 106
  8. 2017REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelscited by 79
  9. 2018Differentiable Particle Filters: End-to-End Learning with Algorithmic Priorscited by 59
  10. 2021Differentiable Particle Filtering via Entropy-Regularized Optimal Transportcited by 59
  11. 2018Don't Unroll Adjoint: Differentiating SSA-Form Programscited by 45
  12. 2022Differentiable Agent-based Epidemiologycited by 21
6 more
  1. 2021Differentiable Particle Filtering without Modifying the Forward Passcited by 18
  2. 2021Storchastic: A Framework for General Stochastic Automatic Differentiationcited by 11
  3. 2021Efficient Learning of the Parameters of Non-Linear Models using Differentiable Resampling in Particle Filterscited by 9
  4. 2022Plots.jl -- a user extendable plotting API for the julia programming languagecited by 4
  5. 2019Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spacescited by 3
  6. 2020Situating Agent-Based Modelling in Population Health Researchcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 9; 2025: 5; 2026: 8.

29 papers.

Sort:

By area:

  1. 2024On the limits of agency in agent-based modelscited by 21
  2. 2023Smoothing Methods for Automatic Differentiation Across Conditional Branchescited by 7
  3. 2023Differentiating Metropolis-Hastings to Optimize Intractable Densitiescited by 6
  4. 2023Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physicscited by 6
  5. 2024An efficient Wasserstein-distance approach for reconstructing jump-diffusion processes using parameterized neural networkscited by 5
  6. 2023On Feynman--Kac training of partial Bayesian neural networkscited by 4
  7. 2024Probabilistic Programming with Programmable Variational Inferencecited by 4
  8. 2024Bayesian calibration of stochastic agent based model via random forestcited by 4
  9. 2024Enhanced SMC2: Leveraging Gradient Information from Differentiable Particle Filters Within Langevin Proposalscited by 4
  10. 2024Generalizing Stochastic Smoothing for Differentiation and Gradient Estimationcited by 4
  11. 2025Learning Individual Behavior in Agent-Based Models with Graph Diffusion Networkscited by 4
  12. 2023Some challenges of calibrating differentiable agent-based modelscited by 2
17 more
  1. 2025Diffusion differentiable resamplingcited by 2
  2. 2026A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computingcited by 2
  3. 2023Auto-Differentiation of Relational Computations for Very Large Scale Machine Learningcited by 1
  4. 2024Automatic Gradient Estimation for Calibrating Crowd Models with Discrete Decision Makingcited by 1
  5. 2024Towards Learning Stochastic Population Models by Gradient Descentcited by 1
  6. 2024Fully Data-driven but Interpretable Human Behavioural Modelling with Differentiable Discrete Choice Modelcited by 1
  7. 2026Gradient estimators for parameter inference in discrete stochastic kinetic modelscited by 1
  8. 2023DynaLay: An Introspective Approach to Dynamic Layer Selection for Deep Networkscited by 0
  9. 2025A Principled Bayesian Framework for Training Binary and Spiking Neural Networkscited by 0
  10. 2025A generalized Wasserstein-2 distance approach for efficient reconstruction of random field models using stochastic neural networkscited by 0
  11. 2025Control of dynamical systems with neural networkscited by 0
  12. 2026Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Methodcited by 0
  13. 2026Dimensional Peeking for Low-Variance Gradients in Zeroth-Order Discrete Optimization via Simulationcited by 0
  14. 2026partialCBDs: Differentiable Causal Block Diagramscited by 0
  15. 2026GradInf: Gradient Estimation as Probabilistic Inferencecited by 0
  16. 2026Scalable Training of Continuous-Time Spiking Neural Networks with Differentiable Spike-Time Discretizationcited by 0
  17. 2026Learning Graph Energies for Finite-Time Generationcited by 0