mlsanityData to 7 Oct 2026

Practical Aspects on Solving Differential Equations Using Deep Learning: A Primer

arXiv 2408.11266 ↗First posted 21 Aug 2024cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

Deep learning is now common across many scientific fields, including the study of partial differential equations. This article provides a brief, accessible introduction to core deep learning concepts, including neural networks, backpropagation, and the universal approximation theorem. It mainly covers how to use deep learning in solving differential equations. The article aims to help undergraduate and graduate students in mathematics, physics, and related areas learn how to use Deep Learning to solve partial differential equations. Instructors in mathematics or physics can also use this article to introduce students to Deep Galerkin method and scientific deep learning. We focus on key questions: What is deep learning, and how can it help solve mathematical or physical problems? How can you implement a neural network and choose the right numerical method to solve differential equations? How do you select the best hyperparameters? How can you improve accuracy and speed up convergence? We should mention that all the problems in this article can be solved on a machine without a GPU, so any student can follow the presented methodology.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 74 references that are in the map. First posted on arXiv 21 Aug 2024.

Builds on

29 of 74 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  7. 2020Learning to summarize from human feedbackcited by 2,562
  8. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  9. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  12. 2016An overview of gradient descent optimization algorithmscited by 1,021
17 more
  1. 2015Automatic differentiation in machine learning: a surveycited by 846
  2. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  3. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  4. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  5. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  6. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  7. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  8. 2015Highway Networkscited by 257
  9. 2017Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equationscited by 188
  10. 2017On weight initialization in deep neural networkscited by 67
  11. 2020An overview on deep learning-based approximation methods for partial differential equationscited by 65
  12. 2019Deep splitting method for parabolic PDEscited by 53
  13. 2018Solving the Kolmogorov PDE by means of deep learningcited by 41
  14. 2018SwitchNet: a neural network model for forward and inverse scattering problemscited by 35
  15. 2022Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustrationcited by 21
  16. 2023Predictive Limitations of Physics-Informed Neural Networks in Vortex Sheddingcited by 12
  17. 2018Unbiased deep solvers for linear parametric PDEscited by 5

Built on by

Not cited yet.