# Bayesian neural networks for weak solution of PDEs with uncertainty quantification

arXiv: https://arxiv.org/abs/2101.04879 · HTML: https://mlsanity.com/paper/2101.04879/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 71 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 30 of 32)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [A Simple Baseline for Bayesian Uncertainty in Deep Learning](https://mlsanity.com/paper/1902.02476/) (2019)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches](https://mlsanity.com/paper/1803.04386/) (2018)
- [Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks](https://mlsanity.com/paper/1906.05747/) (2019)
- [A unified deep artificial neural network approach to partial differential equations in complex geometries](https://mlsanity.com/paper/1711.06464/) (2017)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [By-passing the Kohn-Sham equations with machine learning](https://mlsanity.com/paper/1609.02815/) (2016)
- [D3M: A deep domain decomposition method for partial differential equations](https://mlsanity.com/paper/1909.12236/) (2019)
- [VarNet: Variational Neural Networks for the Solution of Partial Differential Equations](https://mlsanity.com/paper/1912.07443/) (2019)
- [Deep learning of free boundary and Stefan problems](https://mlsanity.com/paper/2006.05311/) (2020)
- [Meta-modeling game for deriving theoretical-consistent, micro-structural-based traction-separation laws via deep reinforcement learning](https://mlsanity.com/paper/1810.10535/) (2018)
- [Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning](https://mlsanity.com/paper/2012.08023/) (2020)
- [Machine learning materials physics: Multi-resolution neural networks learn the free energy and nonlinear elastic response of evolving microstructures](https://mlsanity.com/paper/2001.01575/) (2019)
- [Variational system identification of the partial differential equations governing microstructure evolution in materials: Inference over sparse and spatially unrelated data](https://mlsanity.com/paper/2001.04816/) (2020)

## Built on by (most cited first, 8 of 8)

- [A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: comparison with finite element method](https://mlsanity.com/paper/2206.13103/) (2022)
- [Generating synthetic data for neural operators](https://mlsanity.com/paper/2401.02398/) (2024)
- [Learning Mechanically Driven Emergent Behavior with Message Passing Neural Networks](https://mlsanity.com/paper/2202.01380/) (2022)
- [DGenNO: A Novel Physics-aware Neural Operator for Solving Forward and Inverse PDE Problems based on Deep, Generative Probabilistic Modeling](https://mlsanity.com/paper/2502.06250/) (2025)
- [Machine Learning in Heterogeneous Porous Materials](https://mlsanity.com/paper/2202.04137/) (2022)
- [A heteroencoder architecture for prediction of failure locations in porous metals using variational inference](https://mlsanity.com/paper/2202.00078/) (2022)
- [Investigating Deep Learning Model Calibration for Classification Problems in Mechanics](https://mlsanity.com/paper/2212.00881/) (2022)
- [The Importance of Architecture Choice in Deep Learning for Climate Applications](https://mlsanity.com/paper/2402.13979/) (2024)

## Abstract (arXiv)

Solving partial differential equations (PDEs) is the canonical approach for understanding the behavior of physical systems. However, large scale solutions of PDEs using state of the art discretization techniques remains an expensive proposition. In this work, a new physics-constrained neural network (NN) approach is proposed to solve PDEs without labels, with a view to enabling high-throughput solutions in support of design and decision-making. Distinct from existing physics-informed NN approaches, where the strong form or weak form of PDEs are used to construct the loss function, we write the loss function of NNs based on the discretized residual of PDEs through an efficient, convolutional operator-based, and vectorized implementation. We explore an encoder-decoder NN structure for both deterministic and probabilistic models, with Bayesian NNs (BNNs) for the latter, which allow us to quantify both epistemic uncertainty from model parameters and aleatoric uncertainty from noise in the data. For BNNs, the discretized residual is used to construct the likelihood function. In our approach, both deterministic and probabilistic convolutional layers are used to learn the applied boundary conditions (BCs) and to detect the problem domain. As both Dirichlet and Neumann BCs are specified as inputs to NNs, a single NN can solve for similar physics, but with different BCs and on a number of problem domains. The trained surrogate PDE solvers can also make interpolating and extrapolating (to a certain extent) predictions for BCs that they were not exposed to during training. Such surrogate models are of particular importance for problems, where similar types of PDEs need to be repeatedly solved for many times with slight variations. We demonstrate the capability and performance of the proposed framework by applying it to steady-state diffusion, linear elasticity, and nonlinear elasticity.