mlsanityData to 7 Oct 2026

The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation

arXiv 2103.11488 ↗First posted 21 Mar 2021math.NA cs.LG cs.NAcited by 14

Not on the map. Why?

Abstract (arXiv)

Identifying hidden dynamics from observed data is a significant and challenging task in a wide range of applications. Recently, the combination of linear multistep methods (LMMs) and deep learning has been successfully employed to discover dynamics, whereas a complete convergence analysis of this approach is still under development. In this work, we consider the deep network-based LMMs for the discovery of dynamics. We put forward error estimates for these methods using the approximation property of deep networks. It indicates, for certain families of LMMs, that the ℓ2 grid error is bounded by the sum of O(hp) and the network approximation error, where h is the time step size and p is the local truncation error order. Numerical results of several physically relevant examples are provided to demonstrate our theory.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 40 of its 73 references that are in the map. First posted on arXiv 21 Mar 2021.

Builds on

40 of 73 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  3. 2018On the Spectral Bias of Neural Networkscited by 1,321
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  5. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  6. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 463
  7. 2018Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equationscited by 233
  8. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
  9. 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 208
  10. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  11. 2018Training behavior of deep neural network in frequency domaincited by 174
  12. 2019Towards Understanding the Spectral Bias of Deep Learningcited by 173
28 more
  1. 2020Deep Network Approximation for Smooth Functionscited by 163
  2. 2017Generalization in Deep Learningcited by 146
  3. 2017Machine Learning of Linear Differential Equations using Gaussian Processescited by 139
  4. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  5. 2018Data Driven Governing Equations Approximation Using Deep Neural Networkscited by 94
  6. 2018Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systemscited by 88
  7. 2019How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?cited by 84
  8. 2018On the Convergence Rate of Training Recurrent Neural Networkscited by 82
  9. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  10. 2019Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theoremcited by 49
  11. 2019Data-Driven Deep Learning of Partial Differential Equations in Modal Spacecited by 46
  12. 2019Theory of the Frequency Principle for General Deep Neural Networkscited by 45
  13. 2018Nonparametric inference of interaction laws in systems of agents from trajectory datacited by 43
  14. 2019NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Datacited by 42
  15. 2020Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theorycited by 41
  16. 2017Geometry of Optimization and Implicit Regularization in Deep Learningcited by 39
  17. 2016Inferring solutions of differential equations using noisy multi-fidelity datacited by 38
  18. 2019Machine Learning for Prediction with Missing Dynamicscited by 27
  19. 2020On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysiscited by 26
  20. 2021Approximation capability of two hidden layer feedforward neural networks with fixed weightscited by 24
  21. 2018Numerical Aspects for Approximating Governing Equations Using Datacited by 21
  22. 2019Can SGD Learn Recurrent Neural Networks with Provable Generalization?cited by 17
  23. 2018Stochastic Gradient Descent Learns State Equations with Nonlinear Activationscited by 16
  24. 2019Discovery of Dynamics Using Linear Multistep Methodscited by 16
  25. 2019Data-driven Discovery of Emergent Behaviors in Collective Dynamicscited by 10
  26. 2019A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relationscited by 9
  27. 2018Implicit Regularization of Stochastic Gradient Descent in Natural Language Processing: Observations and Implicationscited by 4
  28. 2019FD-Net with Auxiliary Time Steps: Fast Prediction of PDEs using Hessian-Free Trust-Region Methodscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 2; 2024: 1; 2025: 3; 2026: 2.

14 papers.

Sort:

  1. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  2. 2021A Framework for Machine Learning of Model Error in Dynamical Systemscited by 37
  3. 2022On Numerical Integration in Neural Ordinary Differential Equationscited by 21
  4. 2021Solving PDEs on Unknown Manifolds with Machine Learningcited by 12
  5. 2023Finite Expression Methods for Discovering Physical Laws from Datacited by 9
  6. 2025DUE: A Deep Learning Framework and Library for Modeling Unknown Equationscited by 8
  7. 2025Learning dynamical systems with hit-and-run random feature mapscited by 7
  8. 2023Implementation and (Inverse Modified) Error Analysis for implicitly-templated ODE-netscited by 6
  9. 2022Weak Collocation Regression method: fast reveal hidden stochastic dynamics from high-dimensional aggregate datacited by 4
  10. 2022VPNets: Volume-preserving neural networks for learning source-free dynamicscited by 2
  11. 2024Weak Collocation Regression for Inferring Stochastic Dynamics with L'evy Noisecited by 1
  12. 2025SPIKE: Stable Physics-Informed Kernel Evolution Method for Solving Hyperbolic Conservation Lawscited by 1
2 more
  1. 2026Hypothesis-driven construction of mesoscopic dynamicscited by 1
  2. 2026Flow map learning in nonlinear vector autoregressive models: influence of the feature-library structure on the training errorcited by 0