# Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise

arXiv: https://arxiv.org/abs/2408.09340 · HTML: https://mlsanity.com/paper/2408.09340/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 18 Aug 2024.

## Builds on (most cited first, 18 of 18)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- ['In-Between' Uncertainty in Bayesian Neural Networks](https://mlsanity.com/paper/1906.11537/) (2019)
- [Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains](https://mlsanity.com/paper/2007.11207/) (2020)
- [Quality of Uncertainty Quantification for Bayesian Neural Network Inference](https://mlsanity.com/paper/1906.09686/) (2019)
- [Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems](https://mlsanity.com/paper/2205.08304/) (2022)
- [Learning Functional Priors and Posteriors from Data and Physics](https://mlsanity.com/paper/2106.05863/) (2021)
- [A practical PINN framework for multi-scale problems with multi-magnitude loss terms](https://mlsanity.com/paper/2308.06672/) (2023)
- [Multi-Output Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Uncertainties](https://mlsanity.com/paper/2202.01710/) (2022)
- [Novel Deep neural networks for solving Bayesian statistical inverse](https://mlsanity.com/paper/2102.03974/) (2021)
- [Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs](https://mlsanity.com/paper/2112.06660/) (2021)
- [Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation](https://mlsanity.com/paper/2407.04617/) (2024)
- [Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion](https://mlsanity.com/paper/2303.07392/) (2023)
- [The Convergence of Markov chain Monte Carlo Methods: From the Metropolis method to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1706.01520/) (2017)

## Built on by (most cited first, 2 of 2)

- [AdS/Deep-Learning made easy II: neural network-based approaches to holography and inverse problems](https://mlsanity.com/paper/2511.22522/) (2025)
- [Real-Time Structural Health Monitoring with Bayesian Neural Networks: Distinguishing Aleatoric and Epistemic Uncertainty for Digital Twin Frameworks](https://mlsanity.com/paper/2512.03115/) (2025)

## Abstract (arXiv)

Bayesian Physics Informed Neural Networks (BPINN) have attracted considerable attention for inferring the system states and physical parameters of differential equations according to noisy observations. However, in practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of the solver for BPINN often encounters these troubles including poor performance and awful convergence for a given step size used to adjust the momentum of those parameters. To address the convergence of HMC for the BPINN method and extend its application scope to multi-scale partial differential equations (PDE), we develop a robust multi-scale BPINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and the BPINN framework. In this newly proposed MBPINN method, we reframe HMC with Stochastic Gradient Descent (SGD) to ensure the most ``likely'' estimation is always provided, and we configure its solver as a Fourier feature mapping-induced MscaleDNN. This novel method offers several key advantages: (1) it is more robust than HMC, (2) it incurs less computational cost than HMC, and (3) it is more flexible for complex problems. We demonstrate the applicability and performance of the proposed method through some general Poisson and multi-scale elliptic problems in one and two-dimensional Euclidean spaces. Our findings indicate that the proposed method can avoid HMC failures and provide valid results. Additionally, our method is capable of handling complex elliptic PDE and producing comparable results for general elliptic PDE under the case of lower signal-to-noise rate. These findings suggest that our proposed approach has great potential for physics-informed machine learning for parameter estimation and solution recovery in the case of ill-posed problems.