mlsanityData to 14 Oct 2026

Universal approximation property of ODENet and ResNet with a single activation function

arXiv 2410.16709 ↗First posted 22 Oct 2024cs.LG cs.AI cs.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

We study a universal approximation property of ODENet and ResNet. The ODENet is a map from an initial value to the final value of an ODE system in a finite interval. It is considered a mathematical model of a ResNet-type deep learning system. We consider dynamical systems with vector fields given by a single composition of the activation function and an affine mapping, which is the most common choice of the ODENet or ResNet vector field in actual machine learning systems. We show that such an ODENet and ResNet with a restricted vector field can uniformly approximate ODENet with a general vector field.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 30 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

17 of 30 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Going Deeper with Convolutionscited by 4,241
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  5. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  7. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  8. 2014Convolutional Neural Networks at Constrained Time Costcited by 81
  9. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 79
  10. 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
  11. 2015Neural Network with Unbounded Activation Functions is Universal Approximatorcited by 63
  12. 2021Neural ODE control for classification, approximation and transportcited by 57
5 more
  1. 2020Universal Approximation Property of Neural Ordinary Differential Equationscited by 32
  2. 2020Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theorycited by 18
  3. 2023Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEscited by 5
  4. 2023Universal Approximation Property of Hamiltonian Deep Neural Networkscited by 2
  5. 2020Universal Approximation Properties for an ODENet and a ResNet: Mathematical Analysis and Numerical Experimentscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximationscited by 0