mlsanityData to 14 Oct 2026

Accuracy and Architecture Studies of Residual Neural Network solving Ordinary Differential Equations

arXiv 2101.03583 ↗First posted 10 Jan 2021math.NA cs.LG cs.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we consider utilizing a residual neural network (ResNet) to solve ordinary differential equations. Stochastic gradient descent method is applied to obtain the optimal parameter set of weights and biases of the network. We apply forward Euler, Runge-Kutta2 and Runge-Kutta4 finite difference methods to generate three sets of targets training the ResNet and carry out the target study. The well trained ResNet behaves just as its counterpart of the corresponding one-step finite difference method. In particular, we carry out (1) the architecture study in terms of number of hidden layers and neurons per layer to find the optimal ResNet structure; (2) the target study to verify the ResNet solver behaves as accurate as its finite difference method counterpart; (3) solution trajectory simulation. Even the ResNet solver looks like and is implemented in a way similar to forward Euler scheme, its accuracy can be as high as any one step method. A sequence of numerical examples are presented to demonstrate the performance of the ResNet solver.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 10 Jan 2021.

Builds on

23 of 52 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2016Wide Residual Networkscited by 3,070
  4. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  5. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  6. 2017Stable Architectures for Deep Neural Networkscited by 341
  7. 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 210
  8. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  9. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  10. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  11. 2017VAMPnets: Deep learning of molecular kineticscited by 136
  12. 2017Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systemscited by 127
11 more
  1. 2018Data Driven Governing Equations Approximation Using Deep Neural Networkscited by 96
  2. 2018Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systemscited by 89
  3. 2017Reversible Architectures for Arbitrarily Deep Residual Neural Networkscited by 70
  4. 2017Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equationscited by 65
  5. 2019NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Datacited by 42
  6. 2017Deep Relaxation: partial differential equations for optimizing deep neural networkscited by 27
  7. 2018Numerical Aspects for Approximating Governing Equations Using Datacited by 21
  8. 2019Discovery of Dynamics Using Linear Multistep Methodscited by 17
  9. 2019Variational training of neural network approximations of solution maps for physical modelscited by 8
  10. 2019Robust learning with implicit residual networkscited by 6
  11. 2017Learning across scales - A multiscale method for Convolution Neural Networkscited by 5

Built on by

Not cited yet.