# Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems

arXiv: https://arxiv.org/abs/2212.03130 · HTML: https://mlsanity.com/paper/2212.03130/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 87 of its 177 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 30 of 87)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)

## Built on by (most cited first, 30 of 34)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Can Physics-Informed Neural Networks beat the Finite Element Method?](https://mlsanity.com/paper/2302.04107/) (2023)
- [Deep Learning in Deterministic Computational Mechanics](https://mlsanity.com/paper/2309.15421/) (2023)
- [Gaussian Process Priors for Systems of Linear Partial Differential Equations with Constant Coefficients](https://mlsanity.com/paper/2212.14319/) (2022)
- [Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation](https://mlsanity.com/paper/2302.05104/) (2023)
- [Exact Enforcement of Temporal Continuity in Sequential Physics-Informed Neural Networks](https://mlsanity.com/paper/2403.03223/) (2024)
- [BiLO: Bilevel Local Operator Learning for PDE Inverse Problems](https://mlsanity.com/paper/2404.17789/) (2024)
- [An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning](https://mlsanity.com/paper/2408.13222/) (2024)
- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [A Direct Sampling-Based Deep Learning Approach for Inverse Medium Scattering Problems](https://mlsanity.com/paper/2305.00250/) (2023)
- [Inverse Problems with Learned Forward Operators](https://mlsanity.com/paper/2311.12528/) (2023)
- [Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning](https://mlsanity.com/paper/2407.07375/) (2024)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [Solving Elliptic Optimal Control Problems via Neural Networks and Optimality System](https://mlsanity.com/paper/2308.11925/) (2023)
- [When do World Models Successfully Learn Dynamical Systems?](https://mlsanity.com/paper/2507.04898/) (2025)
- [Deep Neural ODE Operator Networks for PDEs](https://mlsanity.com/paper/2510.15651/) (2025)
- [Conductivity Imaging from Internal Measurements with Mixed Least-Squares Deep Neural Networks](https://mlsanity.com/paper/2303.16454/) (2023)
- [Electrical Impedance Tomography: A Fair Comparative Study on Deep Learning and Analytic-based Approaches](https://mlsanity.com/paper/2310.18636/) (2023)
- [GN-SINDy: Greedy Sampling Neural Network in Sparse Identification of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2405.08613/) (2024)
- [Preconditioned FEM-based Neural Networks for Solving Incompressible Fluid Flows and Related Inverse Problems](https://mlsanity.com/paper/2409.04067/) (2024)
- [Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation](https://mlsanity.com/paper/2508.21571/) (2025)
- [Autoregression-Free Neural Operators for Time-Dependent PDEs](https://mlsanity.com/paper/2605.25413/) (2026)
- [An Iterative Deep Ritz Method for Monotone Elliptic Problems](https://mlsanity.com/paper/2501.15186/) (2025)
- [Solved in Unit Domain: JacobiNet for Differentiable Coordinate-Transformed PINNs](https://mlsanity.com/paper/2508.02537/) (2025)
- [Noise-adapted Neural Operator for Robust Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2508.09655/) (2025)
- [Physically consistent model learning for reaction-diffusion systems](https://mlsanity.com/paper/2512.14240/) (2025)
- [Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations](https://mlsanity.com/paper/2603.18331/) (2026)
- [Dynamics-Encoded Deep Learning for Robust System Identification and Parameter Estimation](https://mlsanity.com/paper/2410.04299/) (2024)
- [The learned range test method for the inverse inclusion problem](https://mlsanity.com/paper/2411.00463/) (2024)
- [Estimating properties of a homogeneous bounded soil using machine learning models](https://mlsanity.com/paper/2506.04256/) (2025)

## Abstract (arXiv)

Recent years have witnessed a growth in mathematics for deep learning--which seeks a deeper understanding of the concepts of deep learning with mathematics and explores how to make it more robust--and deep learning for mathematics, where deep learning algorithms are used to solve problems in mathematics. The latter has popularised the field of scientific machine learning where deep learning is applied to problems in scientific computing. Specifically, more and more neural network architectures have been developed to solve specific classes of partial differential equations (PDEs). Such methods exploit properties that are inherent to PDEs and thus solve the PDEs better than standard feed-forward neural networks, recurrent neural networks, or convolutional neural networks. This has had a great impact in the area of mathematical modeling where parametric PDEs are widely used to model most natural and physical processes arising in science and engineering. In this work, we review such methods as well as their extensions for parametric studies and for solving the related inverse problems. We equally proceed to show their relevance in some industrial applications.