# Learning from Integral Losses in Physics Informed Neural Networks

arXiv: https://arxiv.org/abs/2305.17387 · HTML: https://mlsanity.com/paper/2305.17387/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 83 references that are in the map.
- First posted on arXiv 27 May 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Meta Pseudo Labels](https://mlsanity.com/paper/2003.10580/) (2020)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Variational Physics-Informed Neural Networks For Solving Partial Differential Equations](https://mlsanity.com/paper/1912.00873/) (2019)
- [Parallel Physics-Informed Neural Networks via Domain Decomposition](https://mlsanity.com/paper/2104.10013/) (2021)
- [Physics-Informed Neural Network for Modelling the Thermochemical Curing Process of Composite-Tool Systems During Manufacture](https://mlsanity.com/paper/2011.13511/) (2020)
- [Physics informed neural networks for continuum micromechanics](https://mlsanity.com/paper/2110.07374/) (2021)
- [D3M: A deep domain decomposition method for partial differential equations](https://mlsanity.com/paper/1909.12236/) (2019)
- [Machine learning for metal additive manufacturing: Predicting temperature and melt pool fluid dynamics using physics-informed neural networks](https://mlsanity.com/paper/2008.13547/) (2020)
- [Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error](https://mlsanity.com/paper/2201.12417/) (2022)
- [On Instrumental Variable Regression for Deep Offline Policy Evaluation](https://mlsanity.com/paper/2105.10148/) (2021)
- [On the Importance of Firth Bias Reduction in Few-Shot Classification](https://mlsanity.com/paper/2110.02529/) (2021)
- [An Experimental Comparison Between Temporal Difference and Residual Gradient with Neural Network Approximation](https://mlsanity.com/paper/2205.12770/) (2022)

## Built on by (most cited first, 4 of 4)

- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [PINN surrogate of Li-ion battery models for parameter inference. Part II: Regularization and application of the pseudo-2D model](https://mlsanity.com/paper/2312.17336/) (2023)
- [INEUS: Iterative Neural Solver for High-Dimensional PIDEs](https://mlsanity.com/paper/2605.06281/) (2026)
- [Physics and Data Driven Transformer-Mamba Framework for Flow Field](https://mlsanity.com/paper/2609.29087/) (2026)

## Abstract (arXiv)

This work proposes a solution for the problem of training physics-informed networks under partial integro-differential equations. These equations require an infinite or a large number of neural evaluations to construct a single residual for training. As a result, accurate evaluation may be impractical, and we show that naive approximations at replacing these integrals with unbiased estimates lead to biased loss functions and solutions. To overcome this bias, we investigate three types of potential solutions: the deterministic sampling approaches, the double-sampling trick, and the delayed target method. We consider three classes of PDEs for benchmarking; one defining Poisson problems with singular charges and weak solutions of up to 10 dimensions, another involving weak solutions on electro-magnetic fields and a Maxwell equation, and a third one defining a Smoluchowski coagulation problem. Our numerical results confirm the existence of the aforementioned bias in practice and also show that our proposed delayed target approach can lead to accurate solutions with comparable quality to ones estimated with a large sample size integral. Our implementation is open-source and available at https://github.com/ehsansaleh/btspinn.