mlsanityData to 7 Oct 2026

On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks

arXiv 2203.13648 ↗First posted 25 Mar 2022cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

This paper empirically studies commonly observed training difficulties of Physics-Informed Neural Networks (PINNs) on dynamical systems. Our results indicate that fixed points which are inherent to these systems play a key role in the optimization of the in PINNs embedded physics loss function. We observe that the loss landscape exhibits local optima that are shaped by the presence of fixed points. We find that these local optima contribute to the complexity of the physics loss optimization which can explain common training difficulties and resulting nonphysical predictions. Under certain settings, e.g., initial conditions close to fixed points or long simulations times, we show that those optima can even become better than that of the desired solution.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 15 of its 31 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

15 of 31 references.

References · 15

  1. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  2. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  3. 2015Automatic differentiation in machine learning: a surveycited by 846
  4. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  5. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
  6. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  7. 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 233
  8. 2022Respecting causality is all you need for training physics-informed neural networkscited by 154
  9. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 122
  10. 2017Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equationscited by 72
  11. 2021Learning in Sinusoidal Spaces with Physics-Informed Neural Networkscited by 61
  12. 2021Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networkscited by 43
3 more
  1. 2014Probabilistic ODE Solvers with Runge-Kutta Meanscited by 22
  2. 2021How to Avoid Trivial Solutions in Physics-Informed Neural Networkscited by 14
  3. 2022PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimizationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 9; 2025: 7; 2026: 1.

23 papers.

Sort:

  1. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 109
  2. 2022Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Samplingcited by 71
  3. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  4. 2024PINNACLE: PINN Adaptive ColLocation and Experimental points selectioncited by 31
  5. 2023Stacked networks improve physics-informed training: applications to neural networks and deep operator networkscited by 21
  6. 2023A multifidelity approach to continual learning for physical systemscited by 13
  7. 2023Predictive Limitations of Physics-Informed Neural Networks in Vortex Sheddingcited by 12
  8. 2024Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspectivecited by 12
  9. 2024SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networkscited by 12
  10. 2023On the Generalization of PINNs outside the training domain and the Hyperparameters influencing itcited by 10
  11. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 10
  12. 2024Projected Neural Differential Equations for Learning Constrained Dynamicscited by 5
11 more
  1. 2024Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problemscited by 4
  2. 2024PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEscited by 4
  3. 2023Investigating the Ability of PINNs To Solve Burgers' PDE Near Finite-Time BlowUpcited by 2
  4. 2025Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networkscited by 2
  5. 2024PINNfluence: Interpreting PINNs through Influence Functionscited by 1
  6. 2025RNNs perform task computations by dynamically warping neural representationscited by 1
  7. 2026Error whitening: Why Gauss-Newton outperforms Newtoncited by 1
  8. 2025B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labelingcited by 0
  9. 2025IP-Basis PINNs: Efficient Multi-Query Inverse Parameter Estimationcited by 0
  10. 2025Stabilizing PINNs: A regularization scheme for PINN training to avoid unstable fixed points of dynamical systemscited by 0
  11. 2025Fast PINN Eigensolvers via Biconvex Reformulationcited by 0