# Universal Approximation Property of Neural Ordinary Differential Equations

arXiv: https://arxiv.org/abs/2012.02414 · HTML: https://mlsanity.com/paper/2012.02414/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2020.

## Built on by (most cited first, 30 of 32)

- [Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers](https://mlsanity.com/paper/2107.13163/) (2021)
- [Neural Flows: Efficient Alternative to Neural ODEs](https://mlsanity.com/paper/2110.13040/) (2021)
- [Momentum Residual Neural Networks](https://mlsanity.com/paper/2102.07870/) (2021)
- [Universal approximation property of invertible neural networks](https://mlsanity.com/paper/2204.07415/) (2022)
- [Do Residual Neural Networks discretize Neural Ordinary Differential Equations?](https://mlsanity.com/paper/2205.14612/) (2022)
- [Distribution learning via neural differential equations: a nonparametric statistical perspective](https://mlsanity.com/paper/2309.01043/) (2023)
- [Implicit regularization of deep residual networks towards neural ODEs](https://mlsanity.com/paper/2309.01213/) (2023)
- [On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows](https://mlsanity.com/paper/2402.06578/) (2024)
- [Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints](https://mlsanity.com/paper/2306.09739/) (2023)
- [Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation](https://mlsanity.com/paper/2305.18460/) (2023)
- [FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities](https://mlsanity.com/paper/2210.00272/) (2022)
- [Symplectic Adjoint Method for Exact Gradient of Neural ODE with Minimal Memory](https://mlsanity.com/paper/2102.09750/) (2021)
- [Dynamical systems' based neural networks](https://mlsanity.com/paper/2210.02373/) (2022)
- [Feedback Favors the Generalization of Neural ODEs](https://mlsanity.com/paper/2410.10253/) (2024)
- [Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODE](https://mlsanity.com/paper/2303.05323/) (2023)
- [Whitening Convergence Rate of Coupling-based Normalizing Flows](https://mlsanity.com/paper/2210.14032/) (2022)
- [On the Convergence Rate of Gaussianization with Random Rotations](https://mlsanity.com/paper/2306.13520/) (2023)
- [Finding emergence in data by maximizing effective information](https://mlsanity.com/paper/2308.09952/) (2023)
- [Neural Information Squeezer for Causal Emergence](https://mlsanity.com/paper/2201.10154/) (2022)
- [Learning via nonlinear conjugate gradients and depth-varying neural ODEs](https://mlsanity.com/paper/2202.05766/) (2022)
- [Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model](https://mlsanity.com/paper/2211.14573/) (2022)
- [Variational formulations of ODE-Net as a mean-field optimal control problem and existence results](https://mlsanity.com/paper/2303.05924/) (2023)
- [Modeling chaotic Lorenz ODE System using Scientific Machine Learning](https://mlsanity.com/paper/2410.06452/) (2024)
- [Universal approximation property of ODENet and ResNet with a single activation function](https://mlsanity.com/paper/2410.16709/) (2024)
- [NeuralFLoC: Neural Flow-Based Joint Registration and Clustering of Functional Data](https://mlsanity.com/paper/2602.03169/) (2026)
- [Exact Fixed-Point Constraints in Neural-ODEs with Provable Universality](https://mlsanity.com/paper/2605.10613/) (2026)
- [Deep Koopman-layered Model with Universal Property Based on Toeplitz Matrices](https://mlsanity.com/paper/2410.02199/) (2024)
- [A comparative study of NeuralODE and Universal ODE approaches to solving Chandrasekhar White Dwarf equation](https://mlsanity.com/paper/2410.14998/) (2024)
- [Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networks](https://mlsanity.com/paper/2504.02607/) (2025)
- [ODE-ViT: Plug & Play Attention Layer from the Generalization of the ViT as an Ordinary Differential Equation](https://mlsanity.com/paper/2511.16501/) (2025)

## Abstract (arXiv)

Neural ordinary differential equations (NODEs) is an invertible neural network architecture promising for its free-form Jacobian and the availability of a tractable Jacobian determinant estimator. Recently, the representation power of NODEs has been partly uncovered: they form an Lp-universal approximator for continuous maps under certain conditions. However, the Lp-universality may fail to guarantee an approximation for the entire input domain as it may still hold even if the approximator largely differs from the target function on a small region of the input space. To further uncover the potential of NODEs, we show their stronger approximation property, namely the sup-universality for approximating a large class of diffeomorphisms. It is shown by leveraging a structure theorem of the diffeomorphism group, and the result complements the existing literature by establishing a fairly large set of mappings that NODEs can approximate with a stronger guarantee.