mlsanityData to 7 Oct 2026

Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?

arXiv 2405.04230 ↗First posted 7 May 2024physics.comp-ph cs.AI cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

This study investigates the potential accuracy boundaries of physics-informed neural networks, contrasting their approach with previous similar works and traditional numerical methods. We find that selecting improved optimization algorithms significantly enhances the accuracy of the results. Simple modifications to the loss function may also improve precision, offering an additional avenue for enhancement. Despite optimization algorithms having a greater impact on convergence than adjustments to the loss function, practical considerations often favor tweaking the latter due to ease of implementation. On a global scale, the integration of an enhanced optimizer and a marginally adjusted loss function enables a reduction in the loss function by several orders of magnitude across diverse physical problems. Consequently, our results obtained using compact networks (typically comprising 2 or 3 layers of 20-30 neurons) achieve accuracies comparable to finite difference schemes employing thousands of grid points. This study encourages the continued advancement of PINNs and associated optimization techniques for broader applications across various fields.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 28 of its 93 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

28 of 93 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  3. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  4. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  5. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  6. 2019An Investigation into Neural Net Optimization via Hessian Eigenvalue Densitycited by 289
  7. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
  8. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  9. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 233
  10. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
  11. 2021Learning to Optimize: A Primer and A Benchmarkcited by 206
  12. 2021Exact imposition of boundary conditions with distance functions in physics-informed deep neural networkscited by 135
16 more
  1. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 122
  2. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 109
  3. 2020A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networkscited by 86
  4. 2023Tackling the Curse of Dimensionality with Physics-Informed Neural Networkscited by 78
  5. 2019Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networkscited by 76
  6. 2022Physics-informed neural networks for inverse problems in supersonic flowscited by 75
  7. 2023Residual-based attention in physics-informed neural networkscited by 72
  8. 2022Error estimates for physics informed neural networks approximating the Navier-Stokes equationscited by 63
  9. 2021Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEscited by 56
  10. 2020A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecturecited by 50
  11. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  12. 2020Physics Informed Neural Networks for Simulating Radiative Transfercited by 35
  13. 2022Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizationscited by 26
  14. 2023Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategiescited by 20
  15. 2023Improving physics-informed neural networks with meta-learned optimizationcited by 19
  16. 2023A practical PINN framework for multi-scale problems with multi-magnitude loss termscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 17; 2026: 13.

31 papers.

Sort:

By area:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 55
  3. 2025Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networkscited by 41
  4. 2026Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networkscited by 11
  5. 2025Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacologycited by 10
  6. 2025ATHENA: Agentic Team for Hierarchical Evolutionary Numerical Algorithmscited by 7
  7. 2025High precision PINNs in unbounded domains: application to singularity formulation in PDEscited by 6
  8. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  9. 2026Spectral bias in physics-informed and operator learning: Analysis and mitigation guidelinescited by 6
  10. 2025Physics-Informed Machine Learning in Biomedical Science and Engineeringcited by 4
  11. 2025Enforcing hidden physics in physics-informed neural networkscited by 3
  12. 2025Hard-Constrained Neural Networks with Physics-Embedded Architecture for Residual Dynamics Learning and Invariant Enforcement in Cyber-Physical Systemscited by 3
19 more
  1. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2
  2. 2025Physics-Informed Neural Networks with Fourier Features and Attention-Driven Decodingcited by 2
  3. 2025Convergence and Sketching-Based Efficient Computation of Neural Tangent Kernel Weights in Physics-Based Losscited by 2
  4. 2025Accelerating Natural Gradient Descent for PINNs with Randomized Numerical Linear Algebracited by 1
  5. 2025Nonlinear discretizations and Newton's method: characterizing stationary points of regression objectivescited by 1
  6. 2025Boundary condition enforcement with PINNs: a comparative study and verification on 3D geometriescited by 1
  7. 2026Lightweight Geometric Adaptation for Training Physics-Informed Neural Networkscited by 1
  8. 2026An Optimisation Framework for the Well-Conditioned Training of Physics-Informed Neural Networkscited by 1
  9. 2025Estimating Parameter Fields in Multi-Physics PDEs from Scarce Measurementscited by 0
  10. 2025Seeing Structural Failure Before it Happens: An Image-Based Physics-Informed Neural Network (PINN) for Spaghetti Bridge Load Predictioncited by 0
  11. 2026Residual Attention Physics-Informed Neural Networks for Robust Multiphysics Simulation of Steady-State Electrothermal Energy Systemscited by 0
  12. 2026Meta-Inverse Physics-Informed Neural Networks for High-Dimensional Ordinary Differential Equationscited by 0
  13. 2026Curvature-aware dynamic precision approach for physics-informed neural networkscited by 0
  14. 2026Adjoint Method versus Physics-Informed Neural Networks in PDE-Constrained Inverse Problemscited by 0
  15. 2026From Non-Convex Self-Concordant Regularization to Scalable Quasi-Newton Training of PINNscited by 0
  16. 2026Accelerated Learning of High Dimensional Functions with a Tensor-Featured Training Networkcited by 0
  17. 2026SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learningcited by 0
  18. 2026Direct Optimization of a 3D Finite-Source Reflector via Neural-Network Parameterizationcited by 0
  19. 2026Linearized PINN with pretrained nonlinear layerscited by 0