mlsanityData to 14 Oct 2026

Deep Learning via Dynamical Systems: An Approximation Perspective

arXiv 1912.10382 ↗First posted 22 Dec 2019cs.LG math.OC stat.MLcited by 78

Not on the map. Why?

Abstract (arXiv)

We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective. In particular, we establish general sufficient conditions for universal approximation using continuous-time deep residual networks, which can also be understood as approximation theories in Lp using flow maps of dynamical systems. In specific cases, rates of approximation in terms of the time horizon are also established. Overall, these results reveal that composition function approximation through flow maps present a new paradigm in approximation theory and contributes to building a useful mathematical framework to investigate deep learning.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). First posted on arXiv 22 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 16; 2023: 13; 2024: 12; 2025: 15; 2026: 8.

78 papers.

Sort:

By area:

  1. 2023A mathematical perspective on Transformerscited by 164
  2. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 101
  3. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 96
  4. 2021Heavy Ball Neural Ordinary Differential Equationscited by 42
  5. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  6. 2021Momentum Residual Neural Networkscited by 37
  7. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  8. 2022Universal approximation property of invertible neural networkscited by 30
  9. 2024Measure-to-measure interpolation using Transformerscited by 29
  10. 2022Do Residual Neural Networks discretize Neural Ordinary Differential Equations?cited by 27
  11. 2022Achieve the Minimum Width of Neural Networks for Universal Approximationcited by 26
  12. 2023From Continuous Dynamics to Graph Neural Networks: Neural Diffusion and Beyondcited by 26
38 more
  1. 2023Distribution learning via neural differential equations: a nonparametric statistical perspectivecited by 22
  2. 2022Residual-based error correction for neural operator accelerated infinite-dimensional Bayesian inverse problemscited by 21
  3. 2023Minimum width for universal approximation using ReLU networks on compact domaincited by 18
  4. 2022Neural Network Architecture Beyond Width and Depthcited by 17
  5. 2021Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networkscited by 16
  6. 2024Clustering in pure-attention hardmax transformers and its role in sentiment analysiscited by 15
  7. 2021Neural network stochastic differential equation models with applications to financial data forecastingcited by 14
  8. 2021Deep Network Approximation in Terms of Intrinsic Parameterscited by 12
  9. 2021An Optimal Control Framework for Joint-channel Parallel MRI Reconstruction without Coil Sensitivitiescited by 10
  10. 2022Dynamical systems' based neural networkscited by 10
  11. 2023Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemescited by 10
  12. 2022Deep Neural Network Approximation of Invariant Functions through Dynamical Systemscited by 9
  13. 2024Interplay between depth and width for interpolation in neural ODEscited by 9
  14. 2024Feedback Favors the Generalization of Neural ODEscited by 8
  15. 2022On the Universal Approximation Property of Deep Fully Convolutional Neural Networkscited by 7
  16. 2022Learning Robust State Observers using Neural ODEs (longer version)cited by 7
  17. 2024Differential Equations for Continuous-Time Deep Learningcited by 7
  18. 2021Calibrating multi-dimensional complex ODE from noisy data via deep neural networkscited by 6
  19. 2023On Enhancing Expressive Power via Compositions of Single Fixed-Size ReLU Networkcited by 6
  20. 2023Vocabulary for Universal Approximation: A Linguistic Perspective of Mapping Compositionscited by 6
  21. 2025Universal Approximation Theorem of Deep Q-Networkscited by 6
  22. 2021Sparsity in long-time control of neural ODEscited by 5
  23. 2021Approximation capabilities of measure-preserving neural networkscited by 4
  24. 2022Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systemscited by 4
  25. 2023Cluster-based classification with neural ODEs via controlcited by 4
  26. 2024Diffeomorphic Measure Matching with Kernels for Generative Modelingcited by 4
  27. 2024Constructive approximate transport maps with normalizing flowscited by 4
  28. 2025Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamicscited by 4
  29. 2022Learning via nonlinear conjugate gradients and depth-varying neural ODEscited by 3
  30. 2022Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problemscited by 3
  31. 2022Analysis of function approximation and stability of general DNNs in directed acyclic graphs using un-rectifying analysiscited by 3
  32. 2022Nonlinear controllability and function representation by neural stochastic differential equationscited by 3
  33. 2023A Minimal Control Family of Dynamical Systems for Universal Approximationcited by 3
  34. 2025Numerical and statistical analysis of NeuralODE with Runge-Kutta time integrationcited by 3
  35. 2025Uncertainty propagation in feed-forward neural network modelscited by 3
  36. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  37. 2021TaylorSwiftNet: Taylor Driven Temporal Modeling for Swift Future Frame Predictioncited by 2
  38. 2022The Mori-Zwanzig formulation of deep learningcited by 2

50 most cited of 78 All 78 (JSON)