# Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach

arXiv: https://arxiv.org/abs/2302.13143 · HTML: https://mlsanity.com/paper/2302.13143/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 30 references that are in the map.
- First posted on arXiv 25 Feb 2023.

## Builds on (most cited first, 5 of 5)

- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)

## Built on by (most cited first, 10 of 10)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies](https://mlsanity.com/paper/2306.17648/) (2023)
- [Physics-informed Neural Network Estimation of Material Properties in Soft Tissue Nonlinear Biomechanical Models](https://mlsanity.com/paper/2312.09787/) (2023)
- [Self-adaptive weighting and sampling for physics-informed neural networks](https://mlsanity.com/paper/2511.05452/) (2025)
- [Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning](https://mlsanity.com/paper/2407.07375/) (2024)
- [Repulsive Ensembles for Bayesian Inference in Physics-informed Neural Networks](https://mlsanity.com/paper/2505.17308/) (2025)
- [Two-scale Neural Networks for Partial Differential Equations with Small Parameters](https://mlsanity.com/paper/2402.17232/) (2024)
- [Variational autoencoders with latent high-dimensional steady geometric flows for dynamics](https://mlsanity.com/paper/2410.10137/) (2024)
- [Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates](https://mlsanity.com/paper/2607.07623/) (2026)
- [Variational Boosting for Physics-Informed Neural Networks](https://mlsanity.com/paper/2607.23940/) (2026)

## Abstract (arXiv)

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given PDE using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs.