# Weak Generative Sampler to Efficiently Sample Invariant Distribution of Stochastic Differential Equation

arXiv: https://arxiv.org/abs/2405.19256 · HTML: https://mlsanity.com/paper/2405.19256/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 72 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 26 of 26)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [An Introduction to Deep Generative Modeling](https://mlsanity.com/paper/2103.05180/) (2021)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [The Randomized Midpoint Method for Log-Concave Sampling](https://mlsanity.com/paper/1909.05503/) (2019)
- [Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learning](https://mlsanity.com/paper/2008.13333/) (2020)
- [On sampling from a log-concave density using kinetic Langevin diffusions](https://mlsanity.com/paper/1807.09382/) (2018)
- [Sum-of-Squares Polynomial Flow](https://mlsanity.com/paper/1905.02325/) (2019)
- [Neural Galerkin Schemes with Active Learning for High-Dimensional Evolution Equations](https://mlsanity.com/paper/2203.01360/) (2022)
- [Adaptive deep density approximation for Fokker-Planck equations](https://mlsanity.com/paper/2103.11181/) (2021)
- [Deep learning probability flows and entropy production rates in active matter](https://mlsanity.com/paper/2309.12991/) (2023)
- [Failure-informed adaptive sampling for PINNs, Part II: combining with re-sampling and subset simulation](https://mlsanity.com/paper/2302.01529/) (2023)
- [VAE-KRnet and its applications to variational Bayes](https://mlsanity.com/paper/2006.16431/) (2020)
- [Stationary Density Estimation of Ito Diffusions Using Deep Learning](https://mlsanity.com/paper/2109.03992/) (2021)
- [Computing the Invariant Distribution of Randomly Perturbed Dynamical Systems Using Deep Learning](https://mlsanity.com/paper/2110.11538/) (2021)
- [Weak Collocation Regression method: fast reveal hidden stochastic dynamics from high-dimensional aggregate data](https://mlsanity.com/paper/2209.02628/) (2022)
- [Residual-Quantile Adjustment for Adaptive Training of Physics-informed Neural Network](https://mlsanity.com/paper/2209.05315/) (2022)
- [Adaptive deep density approximation for fractional Fokker-Planck equations](https://mlsanity.com/paper/2210.14402/) (2022)

## Built on by (most cited first, 4 of 4)

- [Generative Path-Finding Method for Wasserstein Gradient Flow](https://mlsanity.com/paper/2604.11519/) (2026)
- [Adaptive Probability Flow Residual Minimization for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2512.19196/) (2025)
- [A Generative Sampler for distributions with possible discrete parameter based on Reversibility](https://mlsanity.com/paper/2603.09251/) (2026)
- [Deep operator learning for efficient sampling from invariant measures of stochastic differential equations](https://mlsanity.com/paper/2609.11376/) (2026)

## Abstract (arXiv)

Sampling invariant distributions from an Ito diffusion process presents a significant challenge in stochastic simulation. Traditional numerical solvers for stochastic differential equations require both a fine step size and a lengthy simulation period, resulting in biased and correlated samples. The current deep learning-based method solves the stationary Fokker--Planck equation to determine the invariant probability density function in the form of deep neural networks, but they generally do not directly address the problem of sampling from the computed density function. In this work, we introduce a framework that employs a weak generative sampler (WGS) to directly generate independent and identically distributed (iid) samples induced by a transformation map derived from the stationary Fokker--Planck equation. Our proposed loss function is based on the weak form of the Fokker--Planck equation, integrating normalizing flows to characterize the invariant distribution and facilitate sample generation from a base distribution. Our randomized test function circumvents the need for min-max optimization in the traditional weak formulation. Our method necessitates neither the computationally intensive calculation of the Jacobian determinant nor the invertibility of the transformation map. A crucial component of our framework is the adaptively chosen family of test functions in the form of Gaussian kernel functions with centers related to the generated data samples. Experimental results on several benchmark examples demonstrate the effectiveness and scalability of our method, which offers both low computational costs and excellent capability in exploring multiple metastable states.