# Tackling the Curse of Dimensionality with Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2307.12306 · HTML: https://mlsanity.com/paper/2307.12306/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 75 references that are in the map.
- First posted on arXiv 23 Jul 2023.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM](https://mlsanity.com/paper/2104.04473/) (2021)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs](https://mlsanity.com/paper/2006.16144/) (2020)
- [A physics-informed variational DeepONet for predicting the crack path in brittle materials](https://mlsanity.com/paper/2108.06905/) (2021)
- [On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems](https://mlsanity.com/paper/2006.11144/) (2020)
- [Convergence rates for the stochastic gradient descent method for non-convex objective functions](https://mlsanity.com/paper/1904.01517/) (2019)
- [Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations](https://mlsanity.com/paper/1709.05963/) (2017)
- [Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks](https://mlsanity.com/paper/1905.01205/) (2019)
- [When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?](https://mlsanity.com/paper/2109.09444/) (2021)
- [Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations](https://mlsanity.com/paper/1804.07010/) (2018)
- [Deep splitting method for parabolic PDEs](https://mlsanity.com/paper/1907.03452/) (2019)
- [Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions](https://mlsanity.com/paper/1902.00908/) (2019)
- [Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodology](https://mlsanity.com/paper/2211.08939/) (2022)
- [Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory](https://mlsanity.com/paper/2006.15733/) (2020)

## Built on by (most cited first, 30 of 79)

- [From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learning](https://mlsanity.com/paper/2410.13228/) (2024)
- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [State-space models are accurate and efficient neural operators for dynamical systems](https://mlsanity.com/paper/2409.03231/) (2024)
- [Large Language Model-Based Evolutionary Optimizer: Reasoning with elitism](https://mlsanity.com/paper/2403.02054/) (2024)
- [Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks](https://mlsanity.com/paper/2312.14499/) (2023)
- [Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equations](https://mlsanity.com/paper/2402.07465/) (2024)
- [Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators](https://mlsanity.com/paper/2412.00088/) (2024)
- [Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs](https://mlsanity.com/paper/2311.15283/) (2023)
- [A Practitioner's Guide to Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2510.25781/) (2025)
- [Physics-constrained polynomial chaos expansion for scientific machine learning and uncertainty quantification](https://mlsanity.com/paper/2402.15115/) (2024)
- [Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble](https://mlsanity.com/paper/2503.06320/) (2025)
- [Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants](https://mlsanity.com/paper/2405.06443/) (2024)
- [PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representations](https://mlsanity.com/paper/2412.05994/) (2024)
- [Anant-Net: Breaking the Curse of Dimensionality with Scalable and Interpretable Neural Surrogate for High-Dimensional PDEs](https://mlsanity.com/paper/2505.03595/) (2025)
- [QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2503.16678/) (2025)
- [Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNs](https://mlsanity.com/paper/2506.07958/) (2025)
- [Binary structured physics-informed neural networks for solving equations with rapidly changing solutions](https://mlsanity.com/paper/2401.12806/) (2024)
- [Separable Physics-informed Neural Networks for Solving the BGK Model of the Boltzmann Equation](https://mlsanity.com/paper/2403.06342/) (2024)
- [Complex Physics-Informed Neural Network](https://mlsanity.com/paper/2502.04917/) (2025)
- [ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks](https://mlsanity.com/paper/2502.00803/) (2025)
- [Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.21404/) (2025)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [Tackling the Curse of Dimensionality in Fractional and Tempered Fractional PDEs with Physics-Informed Neural Networks](https://mlsanity.com/paper/2406.11708/) (2024)
- [Separable Operator Networks](https://mlsanity.com/paper/2407.11253/) (2024)
- [Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees](https://mlsanity.com/paper/2410.18153/) (2024)
- [BubbleOKAN: A Physics-Informed Interpretable Neural Operator for High-Frequency Bubble Dynamics](https://mlsanity.com/paper/2508.03965/) (2025)
- [An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networks](https://mlsanity.com/paper/2501.11222/) (2025)
- [Sharp-PINNs: staggered hard-constrained physics-informed neural networks for phase field modelling of corrosion](https://mlsanity.com/paper/2502.11942/) (2025)
- [Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Features](https://mlsanity.com/paper/2510.19399/) (2025)
- [Optimal time sampling in physics-informed neural networks](https://mlsanity.com/paper/2404.18780/) (2024)

## Abstract (arXiv)

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional PDEs, as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerically partial differential equations (PDEs) in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman (HJB) and the Schr"odinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in 100,000 effective dimensions in 12 hours on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.