# Variational Boosting for Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2607.23940 · HTML: https://mlsanity.com/paper/2607.23940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Builds on (most cited first, 8 of 8)

- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach](https://mlsanity.com/paper/2302.13143/) (2023)
- [Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space](https://mlsanity.com/paper/2501.12116/) (2025)
- [Stiff Transfer Learning for Physics-Informed Neural Networks](https://mlsanity.com/paper/2501.17281/) (2025)
- [Controlling Chaos in Van Der Pol Dynamics Using Signal-Encoded Deep Learning](https://mlsanity.com/paper/2112.14707/) (2021)
- [Scaling Laws and Pathologies of Single-Layer PINNs: Network Width and PDE Nonlinearity](https://mlsanity.com/paper/2603.12556/) (2026)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.