# Learning Functional Priors and Posteriors from Data and Physics

arXiv: https://arxiv.org/abs/2106.05863 · HTML: https://mlsanity.com/paper/2106.05863/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 15 of 15)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [All You Need is a Good Functional Prior for Bayesian Deep Learning](https://mlsanity.com/paper/2011.12829/) (2020)
- [Multi-fidelity Bayesian Neural Networks: Algorithms and Applications](https://mlsanity.com/paper/2012.13294/) (2020)
- [Physics-informed semantic inpainting: Application to geostatistical modeling](https://mlsanity.com/paper/1909.09459/) (2019)
- [tfp.mcmc: Modern Markov Chain Monte Carlo Tools Built for Modern Hardware](https://mlsanity.com/paper/2002.01184/) (2020)
- [GAN-based Priors for Quantifying Uncertainty](https://mlsanity.com/paper/2003.12597/) (2020)

## Built on by (most cited first, 30 of 37)

- [Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons](https://mlsanity.com/paper/2201.07766/) (2022)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [In-Context Operator Learning with Data Prompts for Differential Equation Problems](https://mlsanity.com/paper/2304.07993/) (2023)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems](https://mlsanity.com/paper/2205.08304/) (2022)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [Uncertainty quantification for noisy inputs-outputs in physics-informed neural networks and neural operators](https://mlsanity.com/paper/2311.11262/) (2023)
- [Calibrated Uncertainty Quantification for Operator Learning via Conformal Prediction](https://mlsanity.com/paper/2402.01960/) (2024)
- [Partial Differential Equations Meet Deep Neural Networks: A Survey](https://mlsanity.com/paper/2211.05567/) (2022)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Physics-informed Information Field Theory for Modeling Physical Systems with Uncertainty Quantification](https://mlsanity.com/paper/2301.07609/) (2023)
- [Leveraging viscous Hamilton-Jacobi PDEs for uncertainty quantification in scientific machine learning](https://mlsanity.com/paper/2404.08809/) (2024)
- [A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes](https://mlsanity.com/paper/2305.03184/) (2023)
- [G2Phinet: Relating Genotype and Biomechanical Phenotype of Tissues with Deep Learning](https://mlsanity.com/paper/2208.09889/) (2022)
- [Energy-Based Models for Functional Data using Path Measure Tilting](https://mlsanity.com/paper/2202.01929/) (2022)
- [Deep neural operator for learning transient response of interpenetrating phase composites subject to dynamic loading](https://mlsanity.com/paper/2303.18055/) (2023)
- [PINNs for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2408.01026/) (2024)
- [Error-Aware B-PINNs: Improving Uncertainty Quantification in Bayesian Physics-Informed Neural Networks](https://mlsanity.com/paper/2212.06965/) (2022)
- [Uncertainty Quantification in PINNs for Turbulent Flows: Bayesian Inference and Repulsive Ensembles](https://mlsanity.com/paper/2604.17156/) (2026)
- [Machine Learning in Heterogeneous Porous Materials](https://mlsanity.com/paper/2202.04137/) (2022)
- [Quantification of total uncertainty in the physics-informed reconstruction of CVSim-6 physiology](https://mlsanity.com/paper/2408.07201/) (2024)
- [A Primer on Variational Inference for Physics-Informed Deep Generative Modelling](https://mlsanity.com/paper/2409.06560/) (2024)
- [A Robust Learning Methodology for Uncertainty-aware Scientific Machine Learning models](https://mlsanity.com/paper/2209.01900/) (2022)
- [Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise](https://mlsanity.com/paper/2408.09340/) (2024)
- [Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications](https://mlsanity.com/paper/2409.05234/) (2024)
- [HJ-sampler: A Bayesian sampler for inverse problems of a stochastic process by leveraging Hamilton-Jacobi PDEs and score-based generative models](https://mlsanity.com/paper/2409.09614/) (2024)

## Abstract (arXiv)

We develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physics-informed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field. The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, and can be extended to big data problems using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.