# Universal Approximation Properties for an ODENet and a ResNet: Mathematical Analysis and Numerical Experiments

arXiv: https://arxiv.org/abs/2101.10229 · HTML: https://mlsanity.com/paper/2101.10229/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2020.

## Built on by (most cited first, 1 of 1)

- [Universal approximation property of ODENet and ResNet with a single activation function](https://mlsanity.com/paper/2410.16709/) (2024)

## Abstract (arXiv)

We prove a universal approximation property (UAP) for a class of ODENet and a class of ResNet, which are simplified mathematical models for deep learning systems with skip connections. The UAP can be stated as follows. Let n and m be the dimension of input and output data, and assume m≤ n. Then we show that ODENet of width n+m with any non-polynomial continuous activation function can approximate any continuous function on a compact subset on ℝn. We also show that ResNet has the same property as the depth tends to infinity. Furthermore, we derive the gradient of a loss function explicitly with respect to a certain tuning variable. We use this to construct a learning algorithm for ODENet. To demonstrate the usefulness of this algorithm, we apply it to a regression problem, a binary classification, and a multinomial classification in MNIST.