# Learning the temporal evolution of multivariate densities via normalizing flows

arXiv: https://arxiv.org/abs/2107.13735 · HTML: https://mlsanity.com/paper/2107.13735/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 65 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 24 of 24)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [Data-driven approximation of the Koopman generator: Model reduction, system identification, and control](https://mlsanity.com/paper/1909.10638/) (2019)
- [Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker-Planck Equation and Physics-informed Neural Networks](https://mlsanity.com/paper/2008.10653/) (2020)
- [Coarse-scale PDEs from fine-scale observations via machine learning](https://mlsanity.com/paper/1909.05707/) (2019)
- [A Data-Driven Approach for Discovering Stochastic Dynamical Systems with Non-Gaussian Levy Noise](https://mlsanity.com/paper/2005.03769/) (2020)
- [Non-parametric Estimation of Stochastic Differential Equations with Sparse Gaussian Processes](https://mlsanity.com/paper/1704.04375/) (2017)
- [Towards Robust and Stable Deep Learning Algorithms for Forward Backward Stochastic Differential Equations](https://mlsanity.com/paper/1910.11623/) (2019)
- [Temporal Normalizing Flows](https://mlsanity.com/paper/1912.09092/) (2019)
- [Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-Informed Deep Generative Models](https://mlsanity.com/paper/2008.01915/) (2020)
- [Vector-Valued Optimal Mass Transport](https://mlsanity.com/paper/1611.09946/) (2016)
- [Bayesian Optical Flow with Uncertainty Quantification](https://mlsanity.com/paper/1611.01230/) (2016)
- [Optimal Transportation of Vector-Valued Measures](https://mlsanity.com/paper/1901.04765/) (2019)

## Built on by (most cited first, 18 of 18)

- [Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2402.07465/) (2024)
- [Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs](https://mlsanity.com/paper/2203.05337/) (2022)
- [Solving time dependent Fokker-Planck equations via temporal normalizing flow](https://mlsanity.com/paper/2112.14012/) (2021)
- [RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators](https://mlsanity.com/paper/2406.05470/) (2024)
- [Extracting stochastic dynamical systems with α-stable L'evy noise from data](https://mlsanity.com/paper/2109.14881/) (2021)
- [Modeling Unknown Stochastic Dynamical System Subject to External Excitation](https://mlsanity.com/paper/2406.15747/) (2024)
- [Extracting Stochastic Governing Laws by Nonlocal Kramers-Moyal Formulas](https://mlsanity.com/paper/2108.12570/) (2021)
- [Generative modeling of time-dependent densities via optimal transport and projection pursuit](https://mlsanity.com/paper/2304.09663/) (2023)
- [Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems](https://mlsanity.com/paper/2305.00669/) (2023)
- [MD-NOMAD: Mixture density nonlinear manifold decoder for emulating stochastic differential equations and uncertainty propagation](https://mlsanity.com/paper/2404.15731/) (2024)
- [Neural Galerkin Normalizing Flow for Transition Probability Density Functions of Diffusion Models](https://mlsanity.com/paper/2603.18907/) (2026)
- [Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equations](https://mlsanity.com/paper/2406.11676/) (2024)
- [Flow to Rare Events: An Application of Normalizing Flow in Temporal Importance Sampling for Automated Vehicle Validation](https://mlsanity.com/paper/2407.07320/) (2024)
- [Least Squares and Marginal Log-Likelihood Model Predictive Control using Normalizing Flows](https://mlsanity.com/paper/2409.17632/) (2024)
- [Integral regularization PINNs for evolution equations](https://mlsanity.com/paper/2503.23729/) (2025)
- [Neural Galerkin Normalizing Flows for Bayesian Inference of Diffusions with Inaccessible Boundaries](https://mlsanity.com/paper/2606.04324/) (2026)
- [Time-dependent density estimation using binary classifiers](https://mlsanity.com/paper/2506.15505/) (2025)
- [A transition-density-based operator learning method for Fokker-Planck equations with various initial conditions](https://mlsanity.com/paper/2606.09434/) (2026)

## Abstract (arXiv)

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g., those produced by integrating local or nonlocal Fokker-Planck equations). We analyze this evolution through machine learning assisted construction of a time-dependent mapping that takes a reference distribution (say, a Gaussian) to each and every instance of our evolving distribution. If the reference distribution is the initial condition of a Fokker-Planck equation, what we learn is the time-T map of the corresponding solution. Specifically, the learned map is a multivariate normalizing flow that deforms the support of the reference density to the support of each and every density snapshot in time. We demonstrate that this approach can approximate probability density function evolutions in time from observed sampled data for systems driven by both Brownian and L'evy noise. We present examples with two- and three-dimensional, uni- and multimodal distributions to validate the method.