# Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

arXiv: https://arxiv.org/abs/2201.05624 · HTML: https://mlsanity.com/paper/2201.05624/

## Key facts

- Cited by 608 papers in the map (data through 14 Oct 2026).
- Builds on 89 of its 220 references that are in the map.
- First posted on arXiv 14 Jan 2022.

## Builds on (most cited first, 30 of 89)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups](https://mlsanity.com/paper/1802.03690/) (2018)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Horovod: fast and easy distributed deep learning in TensorFlow](https://mlsanity.com/paper/1802.05799/) (2018)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition](https://mlsanity.com/paper/2003.05385/) (2020)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [PhyGeoNet: Physics-Informed Geometry-Adaptive Convolutional Neural Networks for Solving Parameterized Steady-State PDEs on Irregular Domain](https://mlsanity.com/paper/2004.13145/) (2020)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)

## Built on by (most cited first, 30 of 608)

- [DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training](https://mlsanity.com/paper/2403.03542/) (2024)
- [Challenges in Training PINNs: A Loss Landscape Perspective](https://mlsanity.com/paper/2402.01868/) (2024)
- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs](https://mlsanity.com/paper/2404.10024/) (2024)
- [A Mathematical Guide to Operator Learning](https://mlsanity.com/paper/2312.14688/) (2023)
- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Can Physics-Informed Neural Networks beat the Finite Element Method?](https://mlsanity.com/paper/2302.04107/) (2023)
- [Neural Operators with Localized Integral and Differential Kernels](https://mlsanity.com/paper/2402.16845/) (2024)
- [Graph Neural Networks for temporal graphs: State of the art, open challenges, and opportunities](https://mlsanity.com/paper/2302.01018/) (2023)
- [KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics](https://mlsanity.com/paper/2407.04192/) (2024)
- [A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions](https://mlsanity.com/paper/2302.14227/) (2023)
- [Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systems](https://mlsanity.com/paper/2504.02918/) (2025)
- [ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2408.11104/) (2024)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning](https://mlsanity.com/paper/2402.10926/) (2024)
- [Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems](https://mlsanity.com/paper/2212.03130/) (2022)
- [NeuralUQ: A comprehensive library for uncertainty quantification in neural differential equations and operators](https://mlsanity.com/paper/2208.11866/) (2022)
- [TimeDiT: General-purpose Diffusion Transformers for Time Series Foundation Model](https://mlsanity.com/paper/2409.02322/) (2024)
- [PINNACLE: PINN Adaptive ColLocation and Experimental points selection](https://mlsanity.com/paper/2404.07662/) (2024)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [An operator preconditioning perspective on training in physics-informed machine learning](https://mlsanity.com/paper/2310.05801/) (2023)
- [Partial Differential Equations Meet Deep Neural Networks: A Survey](https://mlsanity.com/paper/2211.05567/) (2022)
- [Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks in Scientific Computing](https://mlsanity.com/paper/2211.07377/) (2022)
- [Physics-informed neural networks for solving forward and inverse problems in complex beam systems](https://mlsanity.com/paper/2303.01055/) (2023)
- [A Survey on Physics Informed Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2309.01909/) (2023)
- [Efficient physics-informed neural networks using hash encoding](https://mlsanity.com/paper/2302.13397/) (2023)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, integral-differential equations, and stochastic PDEs. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related advantages and disadvantages. The review also attempts to incorporate publications on a broader range of collocation-based physics informed neural networks, which stars form the vanilla PINN, as well as many other variants, such as physics-constrained neural networks (PCNN), variational hp-VPINN, and conservative PINN (CPINN). The study indicates that most research has focused on customizing the PINN through different activation functions, gradient optimization techniques, neural network structures, and loss function structures. Despite the wide range of applications for which PINNs have been used, by demonstrating their ability to be more feasible in some contexts than classical numerical techniques like Finite Element Method (FEM), advancements are still possible, most notably theoretical issues that remain unresolved.