mlsanityData to 7 Oct 2026

Error Analysis of Three-Layer Neural Network Trained with PGD for Deep Ritz Method

arXiv 2405.11451 ↗First posted 19 May 2024math.NA cs.AI cs.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

Machine learning is a rapidly advancing field with diverse applications across various domains. One prominent area of research is the utilization of deep learning techniques for solving partial differential equations(PDEs). In this work, we specifically focus on employing a three-layer tanh neural network within the framework of the deep Ritz method(DRM) to solve second-order elliptic equations with three different types of boundary conditions. We perform projected gradient descent(PDG) to train the three-layer network and we establish its global convergence. To the best of our knowledge, we are the first to provide a comprehensive error analysis of using overparameterized networks to solve PDE problems, as our analysis simultaneously includes estimates for approximation error, generalization error, and optimization error. We present error bound in terms of the sample size n and our work provides guidance on how to set the network depth, width, step size, and number of iterations for the projected gradient descent algorithm. Importantly, our assumptions in this work are classical and we do not require any additional assumptions on the solution of the equation. This ensures the broad applicability and generality of our results.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 92 references that are in the map. First posted on arXiv 19 May 2024.

Builds on

56 of 92 references.

References · 56

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  3. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  4. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  5. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  6. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  7. 2016Error bounds for approximations with deep ReLU networkscited by 567
  8. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  9. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  10. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  11. 2018Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layerscited by 340
  12. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
38 more
  1. 2017PDE-Net: Learning PDEs from Datacited by 304
  2. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  3. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  4. 2021Deep learning: a statistical viewpointcited by 239
  5. 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 233
  6. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 231
  7. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 221
  8. 2019Towards moderate overparameterization: global convergence guarantees for training shallow neural networkscited by 176
  9. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  10. 2020Deep Network Approximation for Smooth Functionscited by 163
  11. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
  12. 2020On the linearity of large non-linear models: when and why the tangent kernel is constantcited by 129
  13. 2017Theoretical insights into the optimization landscape of over-parameterized shallow neural networkscited by 128
  14. 2019An Improved Analysis of Training Over-parameterized Deep Neural Networkscited by 120
  15. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  16. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 115
  17. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  18. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  19. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  20. 2019How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?cited by 84
  21. 2018On the Convergence Rate of Training Recurrent Neural Networkscited by 82
  22. 2021On the approximation of functions by tanh neural networkscited by 81
  23. 2020Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learningcited by 77
  24. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  25. 2020Global Convergence of Deep Networks with One Wide Layer Followed by Pyramidal Topologycited by 66
  26. 2022Failure-informed adaptive sampling for PINNscited by 64
  27. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  28. 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
  29. 2021Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEscited by 56
  30. 2020Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networkscited by 44
  31. 2021On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widthscited by 44
  32. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  33. 2021Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimalitycited by 40
  34. 2021Elementary superexpressive activationscited by 35
  35. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  36. 2023Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivativescited by 30
  37. 2021A Priori Generalization Error Analysis of Two-Layer Neural Networks for Solving High Dimensional Schr"odinger Eigenvalue Problemscited by 19
  38. 2020Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learningcited by 18

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024DRM Revisited: A Complete Error Analysiscited by 4