# Improved Physics-informed neural networks loss function regularization with a variance-based term

arXiv: https://arxiv.org/abs/2412.13993 · HTML: https://mlsanity.com/paper/2412.13993/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Transfer learning enhanced physics informed neural network for phase-field modeling of fracture](https://mlsanity.com/paper/1907.02531/) (2019)
- [Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture](https://mlsanity.com/paper/2011.13511/) (2020)
- [Dynamically configured physics-informed neural network in topology optimization applications](https://mlsanity.com/paper/2312.06993/) (2023)

## Built on by (most cited first, 2 of 2)

- [Study Design and Demystification of Physics Informed Neural Networks for Power Flow Simulation](https://mlsanity.com/paper/2509.19233/) (2025)
- [SPIKE: Sparse Koopman Regularization for Physics-Informed Neural Networks](https://mlsanity.com/paper/2601.10282/) (2026)

## Abstract (arXiv)

In machine learning and statistical modeling, the mean square or absolute error is commonly used as an error metric, also called a "loss function." While effective in reducing the average error, this approach may fail to address localized outliers, leading to significant inaccuracies in regions with sharp gradients or discontinuities. This issue is particularly evident in physics-informed neural networks (PINNs), where such localized errors are expected and affect the overall solution. To overcome this limitation, we propose a novel loss function that combines the mean and the standard deviation of the chosen error metric. By minimizing this combined loss function, the method ensures a more uniform error distribution and reduces the impact of localized high-error regions. The proposed loss function is easy to implement and tested on problems of varying complexity: the 1D Poisson equation, the unsteady Burgers' equation, 2D linear elastic solid mechanics, and 2D steady Navier-Stokes equations. Results demonstrate improved solution quality and lower maximum error compared to the standard mean-based loss, with minimal impact on computational time.