# On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2203.13648 · HTML: https://mlsanity.com/paper/2203.13648/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2107.00940/) (2021)
- [Probabilistic ODE Solvers with Runge-Kutta Means](https://mlsanity.com/paper/1406.2582/) (2014)
- [How to Avoid Trivial Solutions in Physics-Informed Neural Networks](https://mlsanity.com/paper/2112.05620/) (2021)
- [PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimization](https://mlsanity.com/paper/2202.01943/) (2022)

## Built on by (most cited first, 23 of 23)

- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems](https://mlsanity.com/paper/2406.19662/) (2024)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [Stacked networks improve physics-informed training: applications to neural networks and deep operator networks](https://mlsanity.com/paper/2311.06483/) (2023)
- [A multifidelity approach to continual learning for physical systems](https://mlsanity.com/paper/2304.03894/) (2023)
- [Predictive Limitations of Physics-Informed Neural Networks in Vortex Shedding](https://mlsanity.com/paper/2306.00230/) (2023)
- [Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective](https://mlsanity.com/paper/2401.08667/) (2024)
- [SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.06286/) (2024)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it](https://mlsanity.com/paper/2302.07557/) (2023)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problems](https://mlsanity.com/paper/2401.07888/) (2024)
- [PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEs](https://mlsanity.com/paper/2405.08558/) (2024)
- [Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUp](https://mlsanity.com/paper/2310.05169/) (2023)
- [Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks](https://mlsanity.com/paper/2506.16443/) (2025)
- [PINNfluence: Interpreting PINNs through Influence Functions](https://mlsanity.com/paper/2409.08958/) (2024)
- [RNNs perform task computations by dynamically warping neural representations](https://mlsanity.com/paper/2512.04310/) (2025)
- [Error whitening: Why Gauss-Newton outperforms Newton](https://mlsanity.com/paper/2605.11316/) (2026)
- [B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling](https://mlsanity.com/paper/2507.01714/) (2025)
- [IP-Basis PINNs: Efficient Multi-Query Inverse Parameter Estimation](https://mlsanity.com/paper/2509.07245/) (2025)
- [Stabilizing PINNs: A regularization scheme for PINN training to avoid unstable fixed points of dynamical systems](https://mlsanity.com/paper/2509.11768/) (2025)
- [Fast PINN Eigensolvers via Biconvex Reformulation](https://mlsanity.com/paper/2511.00792/) (2025)

## Abstract (arXiv)

This paper empirically studies commonly observed training difficulties of Physics-Informed Neural Networks (PINNs) on dynamical systems. Our results indicate that fixed points which are inherent to these systems play a key role in the optimization of the in PINNs embedded physics loss function. We observe that the loss landscape exhibits local optima that are shaped by the presence of fixed points. We find that these local optima contribute to the complexity of the physics loss optimization which can explain common training difficulties and resulting nonphysical predictions. Under certain settings, e.g., initial conditions close to fixed points or long simulations times, we show that those optima can even become better than that of the desired solution.