mlsanityData to 7 Oct 2026

An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions

arXiv 2309.07049 ↗First posted 13 Sep 2023math.NA cs.LG cs.NAcited by 21

Not on the map. Why?

Abstract (arXiv)

We present two effective methods for solving high-dimensional partial differential equations (PDE) based on randomized neural networks. Motivated by the universal approximation property of this type of networks, both methods extend the extreme learning machine (ELM) approach from low to high dimensions. With the first method the unknown solution field in d dimensions is represented by a randomized feed-forward neural network, in which the hidden-layer parameters are randomly assigned and fixed while the output-layer parameters are trained. The PDE and the boundary/initial conditions, as well as the continuity conditions (for the local variant of the method), are enforced on a set of random interior/boundary collocation points. The resultant linear or nonlinear algebraic system, through its least squares solution, provides the trained values for the network parameters. With the second method the high-dimensional PDE problem is reformulated through a constrained expression based on an Approximate variant of the Theory of Functional Connections (A-TFC), which avoids the exponential growth in the number of terms of TFC as the dimension increases. The free field function in the A-TFC constrained expression is represented by a randomized neural network and is trained by a procedure analogous to the first method. We present ample numerical simulations for a number of high-dimensional linear/nonlinear stationary/dynamic PDEs to demonstrate their performance. These methods can produce accurate solutions to high-dimensional PDEs, in particular with their errors reaching levels not far from the machine accuracy for relatively lower dimensions. Compared with the physics-informed neural network (PINN) method, the current method is both cost-effective and more accurate for high-dimensional PDEs.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 36 of its 109 references that are in the map. First posted on arXiv 13 Sep 2023.

Builds on

36 of 109 references.

References · 36

  1. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  2. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  3. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  4. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  5. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  6. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  7. 2019Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Datacited by 261
  8. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
  9. 2019Variational Physics-Informed Neural Networks For Solving Partial Differential Equationscited by 144
  10. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
  11. 2020A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networkscited by 86
  12. 2020Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equationscited by 78
24 more
  1. 2020Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learningcited by 77
  2. 2019Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equationscited by 66
  3. 2017Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equationscited by 62
  4. 2018Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equationscited by 54
  5. 2023A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositionscited by 48
  6. 2018Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equationscited by 44
  7. 2019D3M: A deep domain decomposition method for partial differential equationscited by 44
  8. 2019Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpointcited by 42
  9. 2019Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equationscited by 40
  10. 2021Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionalitycited by 34
  11. 2018A proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficientscited by 31
  12. 2020Meshless physics-informed deep learning method for three-dimensional solid mechanicscited by 27
  13. 2021On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elementscited by 25
  14. 2018A Deep Neural Network Surrogate for High-Dimensional Random Partial Differential Equationscited by 23
  15. 2016Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewherecited by 21
  16. 2020Random Vector Functional Link Networks for Function Approximation on Manifoldscited by 19
  17. 2022Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEscited by 19
  18. 2022A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networkscited by 14
  19. 2020A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networkscited by 13
  20. 2021A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machinescited by 13
  21. 2017The Theory of Connections. Connecting Pointscited by 10
  22. 2022Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machinecited by 9
  23. 2022Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networkscited by 8
  24. 2023Temporal Difference Learning for High-Dimensional PIDEs with Jumpscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 2; 2026: 9.

21 papers.

Sort:

  1. 2023Universal approximation property of Banach space-valued random feature models including random neural networkscited by 12
  2. 2024Full error analysis of the random deep splitting method for nonlinear parabolic PDEs and PIDEscited by 12
  3. 2024Solving stochastic partial differential equations using neural networks in the Wiener chaos expansioncited by 6
  4. 2024Solving Partial Differential Equations with Random Feature Modelscited by 6
  5. 2024A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problemscited by 4
  6. 2024Quantification of total uncertainty in the physics-informed reconstruction of CVSim-6 physiologycited by 3
  7. 2024RandNet-Parareal: a time-parallel PDE solver using Random Neural Networkscited by 3
  8. 2024A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equationscited by 2
  9. 2025HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEscited by 1
  10. 2026A Structure-Adaptive Random Feature Method for High-Dimensional Elliptic PDEscited by 1
  11. 2024A Physics-driven GraphSAGE Method for Physical Process Simulations Described by Partial Differential Equationscited by 0
  12. 2024Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networkscited by 0
9 more
  1. 2025A Neumann-Neumann Acceleration with Coarse Space for Domain Decomposition of Extreme Learning Machinescited by 0
  2. 2026Solving Inverse Parametrized Problems via Finite Elements and Extreme Learning Networkscited by 0
  3. 2026Rigorous Error Certification for Neural PDE Solvers: From Empirical Residuals to Solution Guaranteescited by 0
  4. 2026Variational Matrix-Learning Fourier Networks for Parametric Multiphysics Surrogatescited by 0
  5. 2026Random Neural Network Expressivity for Non-Linear Partial Differential Equationscited by 0
  6. 2026Curvature-aware dynamic precision approach for physics-informed neural networkscited by 0
  7. 2026Structure-Oriented Randomized Neural Networks for Poisson-Nernst-Planck and Poisson-Nernst-Planck-Navier-Stokes Systemscited by 0
  8. 2026Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samplescited by 0
  9. 2026Physics Informed Random Feature Neural Networks for Solving PDEscited by 0