# Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?

arXiv: https://arxiv.org/abs/2206.02016 · HTML: https://mlsanity.com/paper/2206.02016/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Error estimates for physics informed neural networks approximating the Navier-Stokes equations](https://mlsanity.com/paper/2203.09346/) (2022)
- [Deep splitting method for parabolic PDEs](https://mlsanity.com/paper/1907.03452/) (2019)
- [Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality](https://mlsanity.com/paper/2110.06897/) (2021)
- [Neural networks-based backward scheme for fully nonlinear PDEs](https://mlsanity.com/paper/1908.00412/) (2019)
- [Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs](https://mlsanity.com/paper/1902.03986/) (2019)
- [Distributional Offline Continuous-Time Reinforcement Learning with Neural Physics-Informed PDEs (SciPhy RL for DOCTR-L)](https://mlsanity.com/paper/2104.01040/) (2021)
- [Deep Learning for Constrained Utility Maximisation](https://mlsanity.com/paper/2008.11757/) (2020)

## Built on by (most cited first, 23 of 23)

- [An Expert's Guide to Training Physics-informed Neural Networks](https://mlsanity.com/paper/2308.08468/) (2023)
- [Tackling the Curse of Dimensionality with Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.12306/) (2023)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [A Neural PDE Solver with Temporal Stencil Modeling](https://mlsanity.com/paper/2302.08105/) (2023)
- [Learning Specialized Activation Functions for Physics-informed Neural Networks](https://mlsanity.com/paper/2308.04073/) (2023)
- [Physics-Informed Neural Networks with Hard Nonlinear Equality and Inequality Constraints](https://mlsanity.com/paper/2507.08124/) (2025)
- [FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.10949/) (2025)
- [Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo](https://mlsanity.com/paper/2307.08214/) (2023)
- [Neural Control of Parametric Solutions for High-dimensional Evolution PDEs](https://mlsanity.com/paper/2302.00045/) (2023)
- [Robust Regression with Highly Corrupted Data via Physics Informed Neural Networks](https://mlsanity.com/paper/2210.10646/) (2022)
- [Toward L_∞-recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields](https://mlsanity.com/paper/2305.00322/) (2023)
- [BeamVQ: Beam Search with Vector Quantization to Mitigate Data Scarcity in Physical Spatiotemporal Forecasting](https://mlsanity.com/paper/2502.18925/) (2025)
- [BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics](https://mlsanity.com/paper/2405.17051/) (2024)
- [Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUp](https://mlsanity.com/paper/2310.05169/) (2023)
- [Feature Mapping in Physics-Informed Neural Networks (PINNs)](https://mlsanity.com/paper/2402.06955/) (2024)
- [PACMANN: Point Adaptive Collocation Method for Artificial Neural Networks](https://mlsanity.com/paper/2411.19632/) (2024)
- [Forward and Inverse Simulation of Pseudo-Two-Dimensional Model of Lithium-Ion Batteries Using Neural Networks](https://mlsanity.com/paper/2412.13200/) (2024)
- [Noisy PDE Training Requires Bigger PINNs](https://mlsanity.com/paper/2507.06967/) (2025)
- [naPINN: Noise-Adaptive Physics-Informed Neural Networks for Recovering Physics from Corrupted Measurement](https://mlsanity.com/paper/2602.02547/) (2026)
- [Approximation of Solution Operators for High-dimensional PDEs](https://mlsanity.com/paper/2401.10385/) (2024)
- [Differential equation quantum solvers: engineering measurements to reduce cost](https://mlsanity.com/paper/2503.22656/) (2025)
- [Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations](https://mlsanity.com/paper/2507.03860/) (2025)
- [Mitigating Gradient Pathology in PINNs through Aligned Constraint](https://mlsanity.com/paper/2605.25001/) (2026)

## Abstract (arXiv)

The Physics-Informed Neural Network (PINN) approach is a new and promising way to solve partial differential equations using deep learning. The L2 Physics-Informed Loss is the de-facto standard in training Physics-Informed Neural Networks. In this paper, we challenge this common practice by investigating the relationship between the loss function and the approximation quality of the learned solution. In particular, we leverage the concept of stability in the literature of partial differential equation to study the asymptotic behavior of the learned solution as the loss approaches zero. With this concept, we study an important class of high-dimensional non-linear PDEs in optimal control, the Hamilton-Jacobi-Bellman(HJB) Equation, and prove that for general Lp Physics-Informed Loss, a wide class of HJB equation is stable only if p is sufficiently large. Therefore, the commonly used L2 loss is not suitable for training PINN on those equations, while L∞ loss is a better choice. Based on the theoretical insight, we develop a novel PINN training algorithm to minimize the L∞ loss for HJB equations which is in a similar spirit to adversarial training. The effectiveness of the proposed algorithm is empirically demonstrated through experiments. Our code is released at https://github.com/LithiumDA/Linf-PINN.