mlsanityData to 7 Oct 2026

Tackling the Curse of Dimensionality with Physics-Informed Neural Networks

arXiv 2307.12306 ↗First posted 23 Jul 2023cs.LG cs.AI cs.NAcited by 78

Not on the map. Why?

Abstract (arXiv)

The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional PDEs, as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerically partial differential equations (PDEs) in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved. We develop a new method of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional PDEs. The new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes a gradient of PDEs into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs. We prove theoretically the convergence and other desired properties of the proposed method. We demonstrate in various diverse tests that the proposed method can solve many notoriously hard high-dimensional PDEs, including the Hamilton-Jacobi-Bellman (HJB) and the Schr"odinger equations in tens of thousands of dimensions very fast on a single GPU using the PINNs mesh-free approach. Notably, we solve nonlinear PDEs with nontrivial, anisotropic, and inseparable solutions in 100,000 effective dimensions in 12 hours on a single GPU using SDGD with PINNs. Since SDGD is a general training methodology of PINNs, it can be applied to any current and future variants of PINNs to scale them up for arbitrary high-dimensional PDEs.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 23 Jul 2023.

Builds on

41 of 75 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  5. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  8. 2019Adversarial Training for Free!cited by 638
  9. 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
  10. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  11. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  12. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 516
29 more
  1. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 470
  2. 2021Physics-informed neural networks with hard constraints for inverse designcited by 277
  3. 2016Deep Learning without Poor Local Minimacited by 233
  4. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  5. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
  6. 2018Adversarial Uncertainty Quantification in Physics-Informed Neural Networkscited by 120
  7. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 115
  8. 2021A physics-informed variational DeepONet for predicting the crack path in brittle materialscited by 98
  9. 2020On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problemscited by 79
  10. 2019Convergence rates for the stochastic gradient descent method for non-convex objective functionscited by 66
  11. 2019Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networkscited by 63
  12. 2017Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equationscited by 62
  13. 2021When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?cited by 57
  14. 2018Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equationscited by 54
  15. 2019Deep splitting method for parabolic PDEscited by 53
  16. 2019Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptionscited by 48
  17. 2022Augmented Physics-Informed Neural Networks (APINNs): A gating network-based soft domain decomposition methodologycited by 43
  18. 2020Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theorycited by 41
  19. 2023Learning stiff chemical kinetics using extended deep neural operatorscited by 30
  20. 2018Machine Learning for semi linear PDEscited by 28
  21. 2022Learning Physics-Informed Neural Networks without Stacked Back-propagationcited by 23
  22. 2022Is L2 Physics-Informed Loss Always Suitable for Training Physics-Informed Neural Network?cited by 23
  23. 2016Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewherecited by 21
  24. 2019Solving high-dimensional optimal stopping problems using deep learningcited by 21
  25. 2023Deep neural operators can serve as accurate surrogates for shape optimization: A case study for airfoilscited by 19
  26. 2021A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equationscited by 18
  27. 2019Three algorithms for solving high-dimensional fully-coupled FBSDEs through deep learningcited by 15
  28. 2020Neural Decomposition: Functional ANOVA with Variational Autoencoderscited by 10
  29. 2022Separable PINN: Mitigating the Curse of Dimensionality in Physics-Informed Neural Networkscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 27; 2025: 31; 2026: 16.

78 papers.

Sort:

By area:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2024Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?cited by 31
  3. 2024Large Language Model-Based Evolutionary Optimizer: Reasoning with elitismcited by 27
  4. 2024State-space models are accurate and efficient neural operators for dynamical systemscited by 27
  5. 2023Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networkscited by 23
  6. 2024Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equationscited by 22
  7. 2024Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operatorscited by 20
  8. 2023Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEscited by 19
  9. 2024Physics-constrained polynomial chaos expansion for scientific machine learning and uncertainty quantificationcited by 17
  10. 2025A Practitioner's Guide to Kolmogorov-Arnold Networkscited by 16
  11. 2025Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemblecited by 13
  12. 2024Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plantscited by 11
38 more
  1. 2024PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representationscited by 11
  2. 2025Anant-Net: Breaking the Curse of Dimensionality with Scalable and Interpretable Neural Surrogate for High-Dimensional PDEscited by 11
  3. 2025QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEscited by 9
  4. 2025Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNscited by 9
  5. 2024Binary structured physics-informed neural networks for solving equations with rapidly changing solutionscited by 8
  6. 2024Separable Physics-informed Neural Networks for Solving the BGK Model of the Boltzmann Equationcited by 8
  7. 2025ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networkscited by 6
  8. 2025Complex Physics-Informed Neural Networkcited by 6
  9. 2025Dual Natural Gradient Descent for Scalable Training of Physics-Informed Neural Networkscited by 6
  10. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  11. 2024Tackling the Curse of Dimensionality in Fractional and Tempered Fractional PDEs with Physics-Informed Neural Networkscited by 5
  12. 2024Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guaranteescited by 5
  13. 2025BubbleOKAN: A Physics-Informed Interpretable Neural Operator for High-Frequency Bubble Dynamicscited by 5
  14. 2024Separable Operator Networkscited by 4
  15. 2025An Imbalanced Learning-based Sampling Method for Physics-informed Neural Networkscited by 4
  16. 2025Sharp-PINNs: staggered hard-constrained physics-informed neural networks for phase field modelling of corrosioncited by 4
  17. 2024Optimal time sampling in physics-informed neural networkscited by 3
  18. 2025ELM-DeepONets: Backpropagation-Free Training of Deep Operator Networks via Extreme Learning Machinescited by 3
  19. 2025FlowKac: An Efficient Neural Fokker-Planck solver using Temporal Normalizing Flows and the Feynman-Kac Formulacited by 3
  20. 2025Improving Memory Efficiency for Training KANs via Meta Learningcited by 3
  21. 2024Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numericscited by 2
  22. 2024On Learning what to Learn: heterogeneous observations of dynamics and establishing (possibly causal) relations among themcited by 2
  23. 2024Base Models for Parabolic Partial Differential Equationscited by 2
  24. 2024A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equationscited by 2
  25. 2025Riemann Tensor Neural Networks: Learning Conservative Systems with Physics-Constrained Networkscited by 2
  26. 2025Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solverscited by 2
  27. 2025Examining the robustness of Physics-Informed Neural Networks to noise for Inverse Problemscited by 2
  28. 2025Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Featurescited by 2
  29. 2023PICProp: Physics-Informed Confidence Propagation for Uncertainty Quantificationcited by 1
  30. 2023PINNsFailureRegion Localization and Refinement through White-box AdversarialAttackcited by 1
  31. 2024Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximationcited by 1
  32. 2024Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equationscited by 1
  33. 2024An efficient wavelet-based physics-informed neural network for multiscale problemscited by 1
  34. 2025Solving Time-Fractional Partial Integro-Differential Equations Using Tensor Neural Networkcited by 1
  35. 2025Finding separatrices of dynamical flows with Deep Koopman Eigenfunctionscited by 1
  36. 2025HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEscited by 1
  37. 2025A Physics Informed Machine Learning Framework for Optimal Sensor Placement and Parameter Estimationcited by 1
  38. 2024The Principle of Minimum Pressure Gradient: An Alternative Basis for Physics-Informed Learning of Incompressible Fluid Mechanicscited by 0

50 most cited of 78 All 78 (JSON)