# Gradient estimators for parameter inference in discrete stochastic kinetic models

arXiv: https://arxiv.org/abs/2604.02121 · HTML: https://mlsanity.com/paper/2604.02121/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator](https://mlsanity.com/paper/2010.04838/) (2020)
- [MadMiner: Machine learning-based inference for particle physics](https://mlsanity.com/paper/1907.10621/) (2019)
- [Automatic Differentiation of Programs with Discrete Randomness](https://mlsanity.com/paper/2210.08572/) (2022)
- [Differentiating Metropolis-Hastings to Optimize Intractable Densities](https://mlsanity.com/paper/2306.07961/) (2023)
- [Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics](https://mlsanity.com/paper/2308.16680/) (2023)
- [Simulation-based inference for Precision Neutrino Physics through Neural Monte Carlo tuning](https://mlsanity.com/paper/2507.23297/) (2025)
- [Exact Discrete Stochastic Simulation with Deep-Learning-Scale Gradient Optimization](https://mlsanity.com/paper/2602.19775/) (2026)
- [Automatic differentiation and the optimization of differential equation models in biology](https://mlsanity.com/paper/2207.04487/) (2022)
- [Flexible Gravitational-Wave Parameter Estimation with Transformers](https://mlsanity.com/paper/2512.02968/) (2025)

## Built on by (most cited first, 1 of 1)

- [Propensity Straight-Through Gradients for Discrete Stochastic Systems](https://mlsanity.com/paper/2608.25631/) (2026)

## Abstract (arXiv)

Stochastic kinetic models are ubiquitous in physics, yet inferring their parameters from experimental data remains challenging. For deterministic models, parameter inference often relies on gradients, which can be obtained efficiently through automatic differentiation (AD). However, AD cannot be applied directly to the Gillespie stochastic simulation algorithm (SSA), since sampling from a discrete set of reactions introduces non-differentiable operations. In this work, we adopt three gradient estimators from machine learning for the Gillespie SSA: the Gumbel-Softmax Straight-Through (GS-ST) estimator, the Score Function estimator, and the Alternative Path estimator. We use the estimators to evaluate gradients of steady-state and time-dependent observables, and compare their performance in representative biophysical systems with relaxation dynamics (bimolecular association) and oscillatory dynamics (repressilator). We find that the GS-ST estimator generally yields well-behaved gradient estimates, but exhibits diverging variance in challenging parameter regimes, which can cause parameter inference to fail. In these cases, other estimators provide more robust, lower variance gradients. Our results demonstrate that gradient-based parameter inference can be effectively combined with the Gillespie SSA, with different estimators offering complementary advantages.