# Partial Differential Equations Meet Deep Neural Networks: A Survey

arXiv: https://arxiv.org/abs/2211.05567 · HTML: https://mlsanity.com/paper/2211.05567/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 82 of its 257 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 30 of 82)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [Deep Potential Molecular Dynamics: a scalable model with the accuracy of quantum mechanics](https://mlsanity.com/paper/1707.09571/) (2017)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Towards Physics-informed Deep Learning for Turbulent Flow Prediction](https://mlsanity.com/paper/1911.08655/) (2019)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems](https://mlsanity.com/paper/1809.08327/) (2018)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs](https://mlsanity.com/paper/2006.16144/) (2020)
- [Efficient training of physics-informed neural networks via importance sampling](https://mlsanity.com/paper/2104.12325/) (2021)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs](https://mlsanity.com/paper/2007.01138/) (2020)

## Built on by (most cited first, 25 of 25)

- [Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey](https://mlsanity.com/paper/2408.12171/) (2024)
- [From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyond](https://mlsanity.com/paper/2310.10121/) (2023)
- [Time Marching Neural Operator FE Coupling: AI Accelerated Physics Modeling](https://mlsanity.com/paper/2504.11383/) (2025)
- [Efficient Neural PDE-Solvers using Quantization Aware Training](https://mlsanity.com/paper/2308.07350/) (2023)
- [An Overview on Machine Learning Methods for Partial Differential Equations: from Physics Informed Neural Networks to Deep Operator Learning](https://mlsanity.com/paper/2408.13222/) (2024)
- [Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operators](https://mlsanity.com/paper/2508.00643/) (2025)
- [Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programming](https://mlsanity.com/paper/2409.10777/) (2024)
- [GN-SINDy: Greedy Sampling Neural Network in Sparse Identification of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2405.08613/) (2024)
- [Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solvers](https://mlsanity.com/paper/2505.12556/) (2025)
- [Autoregression-Free Neural Operators for Time-Dependent PDEs](https://mlsanity.com/paper/2605.25413/) (2026)
- [GNRK: Graph Neural Runge-Kutta method for solving partial differential equations](https://mlsanity.com/paper/2310.00618/) (2023)
- [A Numerical Study of Chaotic Dynamics of K-S Equation with FNOs](https://mlsanity.com/paper/2410.12280/) (2024)
- [Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equations](https://mlsanity.com/paper/2603.18331/) (2026)
- [Application of Tensor Neural Networks to Pricing Bermudan Swaptions](https://mlsanity.com/paper/2304.09750/) (2023)
- [Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks](https://mlsanity.com/paper/2307.06362/) (2023)
- [Learning thermodynamic master equations for open quantum systems](https://mlsanity.com/paper/2506.01882/) (2025)
- [Operator Learning for Consolidation: An Architectural Comparison for DeepONet Variants](https://mlsanity.com/paper/2507.10368/) (2025)
- [AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver](https://mlsanity.com/paper/2507.23141/) (2025)
- [Physics-informed post-processing of stabilized finite element solutions for transient convection-dominated problems](https://mlsanity.com/paper/2603.03259/) (2026)
- [Efficient Approximation to Analytic and Lp functions by Height-Augmented ReLU Networks](https://mlsanity.com/paper/2603.11128/) (2026)
- [Coordinate Encoding on Linear Grids for Physics-Informed Neural Networks](https://mlsanity.com/paper/2603.22700/) (2026)
- [Variational Matrix-Learning Fourier Networks for Parametric Multiphysics Surrogates](https://mlsanity.com/paper/2605.02280/) (2026)
- [Meta-Inverse Physics-Informed Neural Networks for High-Dimensional Ordinary Differential Equations](https://mlsanity.com/paper/2605.03511/) (2026)
- [One-Step Evolution for Long-Time Extrapolation: An Error-Bound-Informed and Prior-Guided Neural Residual Framework for Autonomous PDEs](https://mlsanity.com/paper/2608.22026/) (2026)
- [Where to Compute and How to Interact: Operator-Readable Adaptation with Gauge-Aware Transport](https://mlsanity.com/paper/2609.15620/) (2026)

## Abstract (arXiv)

Many problems in science and engineering can be represented by a set of partial differential equations (PDEs) through mathematical modeling. Mechanism-based computation following PDEs has long been an essential paradigm for studying topics such as computational fluid dynamics, multiphysics simulation, molecular dynamics, or even dynamical systems. It is a vibrant multi-disciplinary field of increasing importance and with extraordinary potential. At the same time, solving PDEs efficiently has been a long-standing challenge. Generally, except for a few differential equations for which analytical solutions are directly available, many more equations must rely on numerical approaches such as the finite difference method, finite element method, finite volume method, and boundary element method to be solved approximately. These numerical methods usually divide a continuous problem domain into discrete points and then concentrate on solving the system at each of those points. Though the effectiveness of these traditional numerical methods, the vast number of iterative operations accompanying each step forward significantly reduces the efficiency. Recently, another equally important paradigm, data-based computation represented by deep learning, has emerged as an effective means of solving PDEs. Surprisingly, a comprehensive review for this interesting subfield is still lacking. This survey aims to categorize and review the current progress on Deep Neural Networks (DNNs) for PDEs. We discuss the literature published in this subfield over the past decades and present them in a common taxonomy, followed by an overview and classification of applications of these related methods in scientific research and engineering scenarios. The origin, developing history, character, sort, as well as the future trends in each potential direction of this subfield are also introduced.