# Adaptive deep density approximation for stochastic dynamical systems

arXiv: https://arxiv.org/abs/2405.02810 · HTML: https://mlsanity.com/paper/2405.02810/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 65 references that are in the map.
- First posted on arXiv 5 May 2024.

## Builds on (most cited first, 29 of 29)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Long-time integration of parametric evolution equations with physics-informed DeepONets](https://mlsanity.com/paper/2106.05384/) (2021)
- [Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2012.02895/) (2020)
- [DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differential equations](https://mlsanity.com/paper/2112.14038/) (2021)
- [Matching Normalizing Flows and Probability Paths on Manifolds](https://mlsanity.com/paper/2207.04711/) (2022)
- [A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions](https://mlsanity.com/paper/2302.14227/) (2023)
- [Data-Driven Deep Learning of Partial Differential Equations in Modal Space](https://mlsanity.com/paper/1910.06948/) (2019)
- [D3M: A deep domain decomposition method for partial differential equations](https://mlsanity.com/paper/1909.12236/) (2019)
- [PFNN: A Penalty-Free Neural Network Method for Solving a Class of Second-Order Boundary-Value Problems on Complex Geometries](https://mlsanity.com/paper/2004.06490/) (2020)
- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [Adaptive deep density approximation for Fokker-Planck equations](https://mlsanity.com/paper/2103.11181/) (2021)
- [Deep Neural Network Modeling of Unknown Partial Differential Equations in Nodal Space](https://mlsanity.com/paper/2106.03603/) (2021)
- [Solving time dependent Fokker-Planck equations via temporal normalizing flow](https://mlsanity.com/paper/2112.14012/) (2021)
- [Deep Multiscale Model Learning](https://mlsanity.com/paper/1806.04830/) (2018)
- [Approximation capabilities of measure-preserving neural networks](https://mlsanity.com/paper/2106.10911/) (2021)

## Built on by (most cited first, 4 of 4)

- [Amortized Filtering and Smoothing with Conditional Normalizing Flows](https://mlsanity.com/paper/2604.07169/) (2026)
- [Adaptive Probability Flow Residual Minimization for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2512.19196/) (2025)
- [A deep learning framework for jointly solving transient Fokker-Planck equations with arbitrary parameters and initial distributions](https://mlsanity.com/paper/2604.06001/) (2026)
- [A transition-density-based operator learning method for Fokker-Planck equations with various initial conditions](https://mlsanity.com/paper/2606.09434/) (2026)

## Abstract (arXiv)

In this paper we consider adaptive deep neural network approximation for stochastic dynamical systems. Based on the Liouville equation associated with the stochastic dynamical systems, a new temporal KRnet (tKRnet) is proposed to approximate the probability density functions (PDFs) of the state variables. The tKRnet gives an explicit density model for the solution of the Liouville equation, which alleviates the curse of dimensionality issue that limits the application of traditional grid based numerical methods. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function, where samples are generated iteratively using the approximate density function at each iteration. A temporal decomposition technique is also employed to improve the long-time integration. Theoretical analysis of our proposed method is provided, and numerical examples are presented to demonstrate its performance.