# Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models

arXiv: https://arxiv.org/abs/2108.12956 · HTML: https://mlsanity.com/paper/2108.12956/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 16 of 16)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [NICE: Non-linear Independent Components Estimation](https://mlsanity.com/paper/1410.8516/) (2014)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [FloWaveNet : A Generative Flow for Raw Audio](https://mlsanity.com/paper/1811.02155/) (2018)
- [Solving inverse problems using conditional invertible neural networks](https://mlsanity.com/paper/2007.15849/) (2020)
- [Universal audio synthesizer control with normalizing flows](https://mlsanity.com/paper/1907.00971/) (2019)

## Built on by (most cited first, 30 of 35)

- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [Failure-informed adaptive sampling for PINNs](https://mlsanity.com/paper/2210.00279/) (2022)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models](https://mlsanity.com/paper/2506.17139/) (2025)
- [Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2402.07465/) (2024)
- [A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems](https://mlsanity.com/paper/2202.04648/) (2022)
- [Fully probabilistic deep models for forward and inverse problems in parametric PDEs](https://mlsanity.com/paper/2208.04856/) (2022)
- [IB-UQ: Information bottleneck based uncertainty quantification for neural function regression and neural operator learning](https://mlsanity.com/paper/2302.03271/) (2023)
- [Solving time dependent Fokker-Planck equations via temporal normalizing flow](https://mlsanity.com/paper/2112.14012/) (2021)
- [Taming Hyperparameter Tuning in Continuous Normalizing Flows Using the JKO Scheme](https://mlsanity.com/paper/2211.16757/) (2022)
- [Diffusion-Model-Assisted Supervised Learning of Generative Models for Density Estimation](https://mlsanity.com/paper/2310.14458/) (2023)
- [TrafficFlowGAN: Physics-informed Flow based Generative Adversarial Network for Uncertainty Quantification](https://mlsanity.com/paper/2206.09319/) (2022)
- [Physics-Informed Deep Learning For Traffic State Estimation: A Survey and the Outlook](https://mlsanity.com/paper/2303.02063/) (2023)
- [Bounded KRnet and its applications to density estimation and approximation](https://mlsanity.com/paper/2305.09063/) (2023)
- [Modeling Unknown Stochastic Dynamical System Subject to External Excitation](https://mlsanity.com/paper/2406.15747/) (2024)
- [Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems](https://mlsanity.com/paper/2304.12541/) (2023)
- [Adaptive deep density approximation for fractional Fokker-Planck equations](https://mlsanity.com/paper/2210.14402/) (2022)
- [PI-VEGAN: Physics Informed Variational Embedding Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/2307.11289/) (2023)
- [Resolution invariant deep operator network for PDEs with complex geometries](https://mlsanity.com/paper/2402.00825/) (2024)
- [Less Emphasis on Difficult Layer Regions: Curriculum Learning for Singularly Perturbed Convection-Diffusion-Reaction Problems](https://mlsanity.com/paper/2210.12685/) (2022)
- [Conditional Pseudo-Reversible Normalizing Flow for Surrogate Modeling in Quantifying Uncertainty Propagation](https://mlsanity.com/paper/2404.00502/) (2024)
- [A Primer on Variational Inference for Physics-Informed Deep Generative Modelling](https://mlsanity.com/paper/2409.06560/) (2024)
- [Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics](https://mlsanity.com/paper/2409.10972/) (2024)
- [TRADE: Transfer of Distributions between External Conditions with Normalizing Flows](https://mlsanity.com/paper/2410.19492/) (2024)
- [Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems](https://mlsanity.com/paper/2502.16232/) (2025)
- [MC-Nonlocal-PINNs: handling nonlocal operators in PINNs via Monte Carlo sampling](https://mlsanity.com/paper/2212.12984/) (2022)
- [Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equations](https://mlsanity.com/paper/2406.11676/) (2024)
- [PinnDE: Physics-Informed Neural Networks for Solving Differential Equations](https://mlsanity.com/paper/2408.10011/) (2024)
- [Learning Stochastic Hamiltonian Systems via Stochastic Generating Function Neural Network](https://mlsanity.com/paper/2507.14467/) (2025)
- [GenVoid: Uncertainty-Aware Learning of Subsurface Material Defects with an Experimentally Validated Physics-Informed Generative Model](https://mlsanity.com/paper/2609.23761/) (2026)

## Abstract (arXiv)

We introduce in this work the normalizing field flows (NFF) for learning random fields from scattered measurements. More precisely, we construct a bijective transformation (a normalizing flow characterizing by neural networks) between a Gaussian random field with the Karhunen-Lo`eve (KL) expansion structure and the target stochastic field, where the KL expansion coefficients and the invertible networks are trained by maximizing the sum of the log-likelihood on scattered measurements. This NFF model can be used to solve data-driven forward, inverse, and mixed forward/inverse stochastic partial differential equations in a unified framework. We demonstrate the capability of the proposed NFF model for learning Non Gaussian processes and different types of stochastic partial differential equations.