# Unbiased Derivative Estimation for Stationary Mean of Parameterized Markov chains

arXiv: https://arxiv.org/abs/2606.11487 · HTML: https://mlsanity.com/paper/2606.11487/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 7 of 7)

- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Queueing Network Controls via Deep Reinforcement Learning](https://mlsanity.com/paper/2008.01644/) (2020)
- [Statistically efficient thinning of a Markov chain sampler](https://mlsanity.com/paper/1510.07727/) (2015)
- [Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chains](https://mlsanity.com/paper/2010.01845/) (2020)
- [Unbiased Markov Chain Monte Carlo: what, why, and how](https://mlsanity.com/paper/2406.06851/) (2024)
- [Maximizing Influence in an Ising Network: A Mean-Field Optimal Solution](https://mlsanity.com/paper/1608.06850/) (2016)
- [Maximizing Activity in Ising Networks via the TAP Approximation](https://mlsanity.com/paper/1803.00110/) (2018)

## Abstract (arXiv)

We propose a new approach to unbiased estimation of the gradients of the stationary means associated with parametrized families of Markov chains. Our estimators are particularly efficient when the Markov chains have slow mixing rate. Our approach does not require a specific parametrization except for an oracle to evaluate the transition density and its gradient at a given data point without any additional knowledge about the density function itself. It makes our estimator suitable for parametrizations associated with neural networks. The estimator can potentially achieve large improvement in terms of efficiency. Numerical experiments confirm the good performance predicted by the theory.