# Expressive probabilistic sampling in recurrent neural networks

arXiv: https://arxiv.org/abs/2308.11809 · HTML: https://mlsanity.com/paper/2308.11809/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 56 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [Generative Modeling with Denoising Auto-Encoders and Langevin Sampling](https://mlsanity.com/paper/2002.00107/) (2020)
- [The implications of perception as probabilistic inference for correlated neural variability during behavior](https://mlsanity.com/paper/1409.0257/) (2014)
- [Flexible and accurate inference and learning for deep generative models](https://mlsanity.com/paper/1805.11051/) (2018)
- [The Hamiltonian brain: efficient probabilistic inference with excitatory-inhibitory neural circuit dynamics](https://mlsanity.com/paper/1407.0973/) (2014)
- [The multilayer connectome of Caenorhabditis elegans](https://mlsanity.com/paper/1608.08793/) (2016)

## Built on by (most cited first, 1 of 1)

- [Setting up for failure: automatic discovery of the neural mechanisms of cognitive errors](https://mlsanity.com/paper/2512.04808/) (2025)

## Abstract (arXiv)

In sampling-based Bayesian models of brain function, neural activities are assumed to be samples from probability distributions that the brain uses for probabilistic computation. However, a comprehensive understanding of how mechanistic models of neural dynamics can sample from arbitrary distributions is still lacking. We use tools from functional analysis and stochastic differential equations to explore the minimum architectural requirements for textitrecurrent neural circuits to sample from complex distributions. We first consider the traditional sampling model consisting of a network of neurons whose outputs directly represent the samples (sampler-only network). We argue that synaptic current and firing-rate dynamics in the traditional model have limited capacity to sample from a complex probability distribution. We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution. We call such circuits reservoir-sampler networks (RSNs). We propose an efficient training procedure based on denoising score matching that finds recurrent and output weights such that the RSN implements Langevin sampling. We empirically demonstrate our model's ability to sample from several complex data distributions using the proposed neural dynamics and discuss its applicability to developing the next generation of sampling-based brain models.