# A hitchhiker's guide to Poisson gradient estimation

arXiv: https://arxiv.org/abs/2602.03896 · HTML: https://mlsanity.com/paper/2602.03896/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 47 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Builds on (most cited first, 14 of 14)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Gradient Estimation Using Stochastic Computation Graphs](https://mlsanity.com/paper/1506.05254/) (2015)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Semi-Amortized Variational Autoencoders](https://mlsanity.com/paper/1802.02550/) (2018)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Poisson Variational Autoencoder](https://mlsanity.com/paper/2405.14473/) (2024)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [Brain-like Variational Inference](https://mlsanity.com/paper/2410.19315/) (2024)
- [Forward chi2 Divergence Based Variational Importance Sampling](https://mlsanity.com/paper/2311.02516/) (2023)
- [A Differentiable Partially Observable Generalized Linear Model with Forward-Backward Message Passing](https://mlsanity.com/paper/2402.01263/) (2024)
- [Negative Binomial Variational Autoencoders for Overdispersed Latent Modeling](https://mlsanity.com/paper/2508.05423/) (2025)
- [Inferring response times of perceptual decisions with Poisson variational autoencoders](https://mlsanity.com/paper/2511.11480/) (2025)

## Built on by (most cited first, 1 of 1)

- [Metabolic cost of information processing in Poisson variational autoencoders](https://mlsanity.com/paper/2602.13421/) (2026)

## Abstract (arXiv)

Poisson-distributed latent variable models are widely used in computational neuroscience, but differentiating through discrete stochastic samples remains challenging. Two approaches address this: *Exponential Arrival Time* (EAT) simulation and *Gumbel-SoftMax* (GSM) relaxation. We provide the first systematic comparison of these methods, along with practical guidance for practitioners. Our main technical contribution is a modification to the EAT method that theoretically guarantees an unbiased first moment (exactly matching the firing rate), and reduces second-moment bias. We evaluate these methods on their distributional fidelity, gradient quality, and performance on two tasks: (1) variational autoencoders with Poisson latents, and (2) partially observable generalized linear models, where latent neural connectivity must be inferred from observed spike trains. Across all metrics, our modified EAT method exhibits better overall performance (often comparable to exact gradients), and substantially higher robustness to hyperparameter choices. These results extend to over-dispersed Negative Binomial latents, where modified EAT again performs best. However, only GSM generalizes to arbitrary non-Poisson distributions, including the under-dispersed regime. Together, our results clarify the trade-offs between these methods and offer concrete recommendations for practitioners working with Poisson latent variable models.