# Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks

arXiv: https://arxiv.org/abs/2205.02902 · HTML: https://mlsanity.com/paper/2205.02902/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 76 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks](https://mlsanity.com/paper/2007.07442/) (2020)
- [A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder](https://mlsanity.com/paper/2009.11990/) (2020)
- [Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equations](https://mlsanity.com/paper/1907.03507/) (2019)
- [Optimal control of PDEs using physics-informed neural networks](https://mlsanity.com/paper/2111.09880/) (2021)
- [SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability](https://mlsanity.com/paper/2204.12670/) (2022)
- [Physics-informed neural networks for the shallow-water equations on the sphere](https://mlsanity.com/paper/2104.00615/) (2021)
- [Quadratic Approximation Manifold for Mitigating the Kolmogorov Barrier in Nonlinear Projection-Based Model Order Reduction](https://mlsanity.com/paper/2204.02462/) (2022)
- [Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective](https://mlsanity.com/paper/2005.10815/) (2020)
- [Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels](https://mlsanity.com/paper/2005.10807/) (2020)
- [Certified machine learning: A posteriori error estimation for physics-informed neural networks](https://mlsanity.com/paper/2203.17055/) (2022)
- [Kolmogorov n-widths for linear dynamical systems](https://mlsanity.com/paper/1806.09880/) (2018)
- [Physics-aware registration based auto-encoder for convection dominated PDEs](https://mlsanity.com/paper/2006.15655/) (2020)

## Built on by (most cited first, 26 of 26)

- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.11833/) (2023)
- [A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions](https://mlsanity.com/paper/2302.14227/) (2023)
- [RoPINN: Region Optimized Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.14369/) (2024)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow](https://mlsanity.com/paper/2206.03198/) (2022)
- [Solving High Frequency and Multi-Scale PDEs with Gaussian Processes](https://mlsanity.com/paper/2311.04465/) (2023)
- [Challenges and Advancements in Modeling Shock Fronts with Physics-Informed Neural Networks: A Review and Benchmarking Study](https://mlsanity.com/paper/2503.17379/) (2025)
- [Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires](https://mlsanity.com/paper/2212.00970/) (2022)
- [Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes](https://mlsanity.com/paper/2306.15630/) (2023)
- [Fourier PINNs: From Strong Boundary Conditions to Adaptive Fourier Bases](https://mlsanity.com/paper/2410.03496/) (2024)
- [Characteristics-Informed Neural Networks for Forward and Inverse Hyperbolic Problems](https://mlsanity.com/paper/2212.14012/) (2022)
- [Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels](https://mlsanity.com/paper/2310.05387/) (2023)
- [Residual-Quantile Adjustment for Adaptive Training of Physics-informed Neural Network](https://mlsanity.com/paper/2209.05315/) (2022)
- [iPINNs: Incremental learning for Physics-informed neural networks](https://mlsanity.com/paper/2304.04854/) (2023)
- [A Low Rank Neural Representation of Entropy Solutions](https://mlsanity.com/paper/2406.05694/) (2024)
- [Integration Matters for Learning PDEs with Backward SDEs](https://mlsanity.com/paper/2505.01078/) (2025)
- [SetPINNs: Set-based Physics-informed Neural Networks](https://mlsanity.com/paper/2409.20206/) (2024)
- [Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning (PIML) Methods: Towards Robust Metrics](https://mlsanity.com/paper/2402.11126/) (2024)
- [Gradient Flow Based Phase-Field Modeling Using Separable Neural Networks](https://mlsanity.com/paper/2405.06119/) (2024)
- [Parameterized Temporal Extrapolation of PDEs via Disentangled Latent Dynamics Manifold Fusion](https://mlsanity.com/paper/2603.12676/) (2026)
- [Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks](https://mlsanity.com/paper/2307.06362/) (2023)
- [Macroscopic auxiliary asymptotic preserving neural networks for the linear radiative transfer equations](https://mlsanity.com/paper/2403.01820/) (2024)
- [Hard constraint learning approaches with trainable influence functions for evolutionary equations](https://mlsanity.com/paper/2502.17497/) (2025)
- [Stabilizing PINNs: A regularization scheme for PINN training to avoid unstable fixed points of dynamical systems](https://mlsanity.com/paper/2509.11768/) (2025)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) leverage neural-networks to find the solutions of partial differential equation (PDE)-constrained optimization problems with initial conditions and boundary conditions as soft constraints. These soft constraints are often considered to be the sources of the complexity in the training phase of PINNs. Here, we demonstrate that the challenge of training (i) persists even when the boundary conditions are strictly enforced, and (ii) is closely related to the Kolmogorov n-width associated with problems demonstrating transport, convection, traveling waves, or moving fronts. Given this realization, we describe the mechanism underlying the training schemes such as those used in eXtended PINNs (XPINN), curriculum regularization, and sequence-to-sequence learning. For an important category of PDEs, i.e., governed by non-linear convection-diffusion equation, we propose reformulating PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed solution. A parallel architecture with two branches is proposed. One branch solves for the state variables on the characteristics, and the second branch solves for the low-dimensional characteristics curves. The proposed architecture conforms to the causality innate to the convection, and leverages the direction of travel of the information in the domain. Finally, we demonstrate that the loss landscapes of LPINNs are less sensitive to the so-called "complexity" of the problems, compared to those in the traditional PINNs in the Eulerian framework.