mlsanityData to 7 Oct 2026

An Expert's Guide to Training Physics-informed Neural Networks

arXiv 2308.08468 ↗First posted 16 Aug 2023cs.LG cs.NA math.NAcited by 119

Not on the map. Why?

Abstract (arXiv)

Physics-informed neural networks (PINNs) have been popularized as a deep learning framework that can seamlessly synthesize observational data and partial differential equation (PDE) constraints. Their practical effectiveness however can be hampered by training pathologies, but also oftentimes by poor choices made by users who lack deep learning expertise. In this paper we present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs. We also put forth a series of challenging benchmark problems that highlight some of the most prominent difficulties in training PINNs, and present comprehensive and fully reproducible ablation studies that demonstrate how different architecture choices and training strategies affect the test accuracy of the resulting models. We show that the methods and guiding principles put forth in this study lead to state-of-the-art results and provide strong baselines that future studies should use for comparison purposes. To this end, we also release a highly optimized library in JAX that can be used to reproduce all results reported in this paper, enable future research studies, as well as facilitate easy adaptation to new use-case scenarios.

Key facts. Cited by 119 papers in the map (data through 7 Oct 2026). Builds on 46 of its 77 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

46 of 77 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  6. 2018On the Spectral Bias of Neural Networkscited by 1,321
  7. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  8. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 732
  9. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  10. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  11. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  12. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
34 more
  1. 2021Machine learning accelerated computational fluid dynamicscited by 461
  2. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
  3. 2023ClimaX: A foundation model for weather and climatecited by 325
  4. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
  5. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  6. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  7. 2020hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decompositioncited by 236
  8. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 233
  9. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
  10. 2021Gradient-enhanced physics-informed neural networks for forward and inverse PDE problemscited by 220
  11. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 195
  12. 2019Towards Understanding the Spectral Bias of Deep Learningcited by 173
  13. 2020PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domaincited by 173
  14. 2022Respecting causality is all you need for training physics-informed neural networkscited by 154
  15. 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 142
  16. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  17. 2021Exact imposition of boundary conditions with distance functions in physics-informed deep neural networkscited by 135
  18. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 134
  19. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 122
  20. 2019Transfer learning enhanced physics informed neural network for phase-field modeling of fracturecited by 116
  21. 2021Efficient training of physics-informed neural networks via importance samplingcited by 109
  22. 2021CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Methodcited by 93
  23. 2020A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networkscited by 86
  24. 2017DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamicscited by 79
  25. 2020Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domainscited by 78
  26. 2020Transfer learning based multi-fidelity physics informed deep neural networkcited by 68
  27. 2020NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation frameworkcited by 58
  28. 2021One-Shot Transfer Learning of Physics-Informed Neural Networkscited by 49
  29. 2021Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networkscited by 43
  30. 2021Conditional physics informed neural networkscited by 23
  31. 2022Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?cited by 23
  32. 2022PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE Solverscited by 22
  33. 2022Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustrationcited by 21
  34. 2022Random Weight Factorization Improves the Training of Continuous Neural Representationscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 34; 2025: 45; 2026: 35.

119 papers.

Sort:

By area:

  1. 2023A Mathematical Guide to Operator Learningcited by 74
  2. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  3. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 70
  4. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 55
  5. 2024CViT: Continuous Vision Transformer for Operator Learningcited by 47
  6. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 37
  7. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  8. 2024Physics Informed Kolmogorov-Arnold Neural Networks for Dynamical Analysis via Efficent-KAN and WAV-KANcited by 25
  9. 2023Lie Point Symmetry and Physics Informed Networkscited by 24
  10. 2024Derivative-enhanced Deep Operator Networkcited by 19
  11. 2024Dynamical Measure Transport and Neural PDE Solvers for Samplingcited by 19
  12. 2024Knowledge-guided Machine Learning: Current Trends and Future Prospectscited by 13
38 more
  1. 2025Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemblecited by 13
  2. 2025Simulating Three-dimensional Turbulence with Physics-informed Neural Networkscited by 13
  3. 2024Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspectivecited by 12
  4. 2024Deep Learning Alternatives of the Kolmogorov Superposition Theoremcited by 11
  5. 2024PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representationscited by 11
  6. 2025Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learningcited by 11
  7. 2024A finite operator learning technique for mapping the elastic properties of microstructures to their mechanical deformationscited by 10
  8. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 10
  9. 2024Precise-Physics Driven Text-to-3D Generationcited by 9
  10. 2024Separable Physics-informed Neural Networks for Solving the BGK Model of the Boltzmann Equationcited by 8
  11. 2024BiLO: Bilevel Local Operator Learning for PDE Inverse Problemscited by 8
  12. 2025The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operatorscited by 8
  13. 2025ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networkscited by 6
  14. 2023Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effectcited by 5
  15. 2024RBF-PINN: Non-Fourier Positional Embedding in Physics-Informed Neural Networkscited by 5
  16. 2024Micrometer: Micromechanics Transformer for Predicting Mechanical Responses of Heterogeneous Materialscited by 5
  17. 2025Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddingscited by 5
  18. 2025Physics-informed time series analysis with Kolmogorov-Arnold Networks under Ehrenfest constraintscited by 5
  19. 2024Domain-decoupled Physics-informed Neural Networks with Closed-form Gradients for Fast Model Learning of Dynamical Systemscited by 4
  20. 2025Neural Discovery in Mathematics: Do Machines Dream of Colored Planes?cited by 4
  21. 2025Sharp-PINNs: staggered hard-constrained physics-informed neural networks for phase field modelling of corrosioncited by 4
  22. 2025Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinementcited by 4
  23. 2026Scale-PINN: Learning Efficient Physics-Informed Neural Networks Through Sequential Correctioncited by 4
  24. 2023Physics-informed neural networks for transformed geometries and manifoldscited by 3
  25. 2024Computing distances and means on manifolds with a metric-constrained Eikonal approachcited by 3
  26. 2025Enhanced Vascular Flow Simulations in Aortic Aneurysm via Physics-Informed Neural Networks and Deep Operator Networkscited by 3
  27. 2025Integration Matters for Learning PDEs with Backward SDEscited by 3
  28. 2025Data-driven particle dynamics: Structure-preserving coarse-graining for emergent behavior in non-equilibrium systemscited by 3
  29. 2025Neuro-Spectral Architectures for Causal Physics-Informed Networkscited by 3
  30. 2024Feature Mapping in Physics-Informed Neural Networks (PINNs)cited by 2
  31. 2024A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networkscited by 2
  32. 2025Sample-Efficient Behavior Cloning Using General Domain Knowledgecited by 2
  33. 2025Physics-informed neural network estimation of active material properties in time-dependent cardiac biomechanical modelscited by 2
  34. 2025FMEnets: Flow, Material, and Energy networks for non-ideal plug flow reactor designcited by 2
  35. 2025Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networkscited by 2
  36. 2025AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decompositioncited by 2
  37. 2026Hard Constraint Projection in a Physics Informed Neural Networkcited by 2
  38. 2026Free-RBF-KAN: Kolmogorov-Arnold Networks with Adaptive Radial Basis Functions for Efficient Function Learningcited by 2

50 most cited of 119 All 119 (JSON)