# Parametric Complexity Bounds for Approximating PDEs with Neural Networks

arXiv: https://arxiv.org/abs/2103.02138 · HTML: https://mlsanity.com/paper/2103.02138/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [A Theoretical Analysis of Deep Neural Networks and Parametric PDEs](https://mlsanity.com/paper/1904.00377/) (2019)
- [A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations](https://mlsanity.com/paper/1809.02362/) (2018)
- [Neural networks and rational functions](https://mlsanity.com/paper/1706.03301/) (2017)
- [A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients](https://mlsanity.com/paper/1809.07321/) (2018)
- [Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditions](https://mlsanity.com/paper/2007.05384/) (2020)

## Built on by (most cited first, 19 of 19)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [Generic bounds on the approximation error for physics-informed (and) operator learning](https://mlsanity.com/paper/2205.11393/) (2022)
- [Operator learning with PCA-Net: upper and lower complexity bounds](https://mlsanity.com/paper/2303.16317/) (2023)
- [Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models](https://mlsanity.com/paper/2309.11420/) (2023)
- [Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning](https://mlsanity.com/paper/2402.10926/) (2024)
- [On the Representation of Solutions to Elliptic PDEs in Barron Spaces](https://mlsanity.com/paper/2106.07539/) (2021)
- [Deep Equilibrium Based Neural Operators for Steady-State PDEs](https://mlsanity.com/paper/2312.00234/) (2023)
- [U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models](https://mlsanity.com/paper/2404.18444/) (2024)
- [Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective](https://mlsanity.com/paper/2210.12101/) (2022)
- [Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent](https://mlsanity.com/paper/2205.07331/) (2022)
- [UGrid: An Efficient-And-Rigorous Neural Multigrid Solver for Linear PDEs](https://mlsanity.com/paper/2408.04846/) (2024)
- [From Monte Carlo to neural networks approximations of boundary value problems](https://mlsanity.com/paper/2209.01432/) (2022)
- [Statistical Spatially Inhomogeneous Diffusion Inference](https://mlsanity.com/paper/2312.05793/) (2023)
- [Barron Space Representations for Elliptic PDEs with Homogeneous Boundary Conditions](https://mlsanity.com/paper/2508.07559/) (2025)
- [A Regularity Theory for Static Schr"odinger Equations on ℝd in Spectral Barron Spaces](https://mlsanity.com/paper/2201.10072/) (2022)
- [Learning to Relax: Setting Solver Parameters Across a Sequence of Linear System Instances](https://mlsanity.com/paper/2310.02246/) (2023)
- [Benign overfitting in Fixed Dimension via Physics-Informed Learning with Smooth Inductive Bias](https://mlsanity.com/paper/2406.09194/) (2024)
- [Feature Learning for the High Dimensional Stationary Sch"odinger Equation with Deep Ritz Method](https://mlsanity.com/paper/2607.06369/) (2026)
- [Localized Operator Learning with Adaptive Partition-of-Unity Mixture-of-Expert Networks](https://mlsanity.com/paper/2610.04708/) (2026)

## Abstract (arXiv)

Recent experiments have shown that deep networks can approximate solutions to high-dimensional PDEs, seemingly escaping the curse of dimensionality. However, questions regarding the theoretical basis for such approximations, including the required network size, remain open. In this paper, we investigate the representational power of neural networks for approximating solutions to linear elliptic PDEs with Dirichlet boundary conditions. We prove that when a PDE's coefficients are representable by small neural networks, the parameters required to approximate its solution scale polynomially with the input dimension d and proportionally to the parameter counts of the coefficient networks. To this we end, we develop a proof technique that simulates gradient descent (in an appropriate Hilbert space) by growing a neural network architecture whose iterates each participate as sub-networks in their (slightly larger) successors, and converge to the solution of the PDE. We bound the size of the solution, showing a polynomial dependence on d and no dependence on the volume of the domain.