# A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions

arXiv: https://arxiv.org/abs/2302.14227 · HTML: https://mlsanity.com/paper/2302.14227/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 30 of 30)

- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs](https://mlsanity.com/paper/1909.10145/) (2019)
- [Parallel Physics-Informed Neural Networks via Domain Decomposition](https://mlsanity.com/paper/2104.10013/) (2021)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs](https://mlsanity.com/paper/2007.01138/) (2020)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks](https://mlsanity.com/paper/2007.07442/) (2020)
- [Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks](https://mlsanity.com/paper/1905.04817/) (2019)
- [Physics-informed neural networks for inverse problems in supersonic flows](https://mlsanity.com/paper/2202.11821/) (2022)
- [Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks](https://mlsanity.com/paper/2005.03596/) (2020)
- [Error estimates for physics informed neural networks approximating the Navier-Stokes equations](https://mlsanity.com/paper/2203.09346/) (2022)
- [When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?](https://mlsanity.com/paper/2109.09444/) (2021)
- [Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology](https://mlsanity.com/paper/2211.08939/) (2022)
- [Deep learning of free boundary and Stefan problems](https://mlsanity.com/paper/2006.05311/) (2020)
- [Physics-informed neural networks for the shallow-water equations on the sphere](https://mlsanity.com/paper/2104.00615/) (2021)
- [How important are activation functions in regression and classification? A survey, performance comparison, and future directions](https://mlsanity.com/paper/2209.02681/) (2022)
- [Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks](https://mlsanity.com/paper/2205.02902/) (2022)
- [Adaptive Self-supervision Algorithms for Physics-informed Neural Networks](https://mlsanity.com/paper/2207.04084/) (2022)
- [Meta Learning of Interface Conditions for Multi-Domain Physics-Informed Neural Networks](https://mlsanity.com/paper/2210.12669/) (2022)
- [Characteristics-Informed Neural Networks for Forward and Inverse Hyperbolic Problems](https://mlsanity.com/paper/2212.14012/) (2022)
- [Temporal Consistency Loss for Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.13262/) (2023)

## Built on by (most cited first, 30 of 48)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems](https://mlsanity.com/paper/2406.19662/) (2024)
- [Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2402.07465/) (2024)
- [An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions](https://mlsanity.com/paper/2309.07049/) (2023)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [Improving physics-informed neural networks with meta-learned optimization](https://mlsanity.com/paper/2303.07127/) (2023)
- [Simulating Three-dimensional Turbulence with Physics-informed Neural Networks](https://mlsanity.com/paper/2507.08972/) (2025)
- [Solving Poisson Equations using Neural Walk-on-Spheres](https://mlsanity.com/paper/2406.03494/) (2024)
- [Transfer learning for improved generalizability in causal physics-informed neural networks for beam simulations](https://mlsanity.com/paper/2311.00578/) (2023)
- [Physics-Informed Neural Networks and Extensions](https://mlsanity.com/paper/2408.16806/) (2024)
- [Approximating High-Dimensional Minimal Surfaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2309.02589/) (2023)
- [BubbleOKAN: A Physics-Informed Interpretable Neural Operator for High-Frequency Bubble Dynamics](https://mlsanity.com/paper/2508.03965/) (2025)
- [FEKAN: Feature-Enriched Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2602.16530/) (2026)
- [Physics-informed neural networks modeling for systems with moving immersed boundaries: application to an unsteady flow past a plunging foil](https://mlsanity.com/paper/2306.13395/) (2023)
- [Adaptive deep density approximation for stochastic dynamical systems](https://mlsanity.com/paper/2405.02810/) (2024)
- [Physics-Informed Machine Learning in Biomedical Science and Engineering](https://mlsanity.com/paper/2510.05433/) (2025)
- [M-ENIAC: A machine learning recreation of the first successful numerical weather forecasts](https://mlsanity.com/paper/2304.09070/) (2023)
- [A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processing](https://mlsanity.com/paper/2308.06447/) (2023)
- [Optimal time sampling in physics-informed neural networks](https://mlsanity.com/paper/2404.18780/) (2024)
- [Solving Differential Equations with Constrained Learning](https://mlsanity.com/paper/2410.22796/) (2024)
- [Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles](https://mlsanity.com/paper/2505.06459/) (2025)
- [A hybrid FEM-PINN method for time-dependent partial differential equations](https://mlsanity.com/paper/2409.02810/) (2024)
- [Adaptive Collocation Point Strategies For Physics Informed Neural Networks via the QR Discrete Empirical Interpolation Method](https://mlsanity.com/paper/2501.07700/) (2025)
- [Physics-Informed Neural ODEs with Scale-Aware Residuals for Learning Stiff Biophysical Dynamics](https://mlsanity.com/paper/2511.11734/) (2025)
- [A Physics-Informed Fourier-Wavelet Transformer for Multiscale Computational Fluid Dynamics Surrogate Modeling](https://mlsanity.com/paper/2606.24696/) (2026)
- [Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients](https://mlsanity.com/paper/2609.14841/) (2026)
- [Error Analysis of Physics-Informed Neural Networks for Approximating Dynamic PDEs of Second Order in Time](https://mlsanity.com/paper/2303.12245/) (2023)
- [Deep smoothness WENO scheme for two-dimensional hyperbolic conservation laws: A deep learning approach for learning smoothness indicators](https://mlsanity.com/paper/2309.10117/) (2023)
- [Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning (PIML) Methods: Towards Robust Metrics](https://mlsanity.com/paper/2402.11126/) (2024)
- [Integral regularization PINNs for evolution equations](https://mlsanity.com/paper/2503.23729/) (2025)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) as a means of solving partial differential equations (PDE) have garnered much attention in the Computational Science and Engineering (CS&E) world. However, a recent topic of interest is exploring various training (i.e., optimization) challenges - in particular, arriving at poor local minima in the optimization landscape results in a PINN approximation giving an inferior, and sometimes trivial, solution when solving forward time-dependent PDEs with no data. This problem is also found in, and in some sense more difficult, with domain decomposition strategies such as temporal decomposition using XPINNs. We furnish examples and explanations for different training challenges, their cause, and how they relate to information propagation and temporal decomposition. We then propose a new stacked-decomposition method that bridges the gap between time-marching PINNs and XPINNs. We also introduce significant computational speed-ups by using transfer learning concepts to initialize subnetworks in the domain and loss tolerance-based propagation for the subdomains. Finally, we formulate a new time-sweeping collocation point algorithm inspired by the previous PINNs causality literature, which our framework can still describe, and provides a significant computational speed-up via reduced-cost collocation point segmentation. The proposed methods form our unified framework, which overcomes training challenges in PINNs and XPINNs for time-dependent PDEs by respecting the causality in multiple forms and improving scalability by limiting the computation required per optimization iteration. Finally, we provide numerical results for these methods on baseline PDE problems for which unmodified PINNs and XPINNs struggle to train.