mlsanityData to 14 Oct 2026

Approximation Capabilities of Neural ODEs and Invertible Residual Networks

arXiv 1907.12998 ↗First posted 30 Jul 2019cs.LG stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

Neural ODEs and i-ResNet are recently proposed methods for enforcing invertibility of residual neural models. Having a generic technique for constructing invertible models can open new avenues for advances in learning systems, but so far the question of whether Neural ODEs and i-ResNets can model any continuous invertible function remained unresolved. Here, we show that both of these models are limited in their approximation capabilities. We then prove that any homeomorphism on a p-dimensional Euclidean space can be approximated by a Neural ODE operating on a 2p-dimensional Euclidean space, and a similar result for i-ResNets. We conclude by showing that capping a Neural ODE or an i-ResNet with a single linear layer is sufficient to turn the model into a universal approximator for non-invertible continuous functions.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 9; 2023: 12; 2024: 9; 2025: 4; 2026: 6.

56 papers.

Sort:

By area:

  1. 2023A mathematical perspective on Transformerscited by 164
  2. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 102
  3. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 101
  4. 2022Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equationscited by 59
  5. 2024Learning with 3D rotations, a hitchhiker's guide to SO(3)cited by 42
  6. 2022Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flowcited by 38
  7. 2021Momentum Residual Neural Networkscited by 37
  8. 2021A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equationscited by 28
  9. 2021Implicit Normalizing Flowscited by 28
  10. 2022On Numerical Integration in Neural Ordinary Differential Equationscited by 21
  11. 2023Implicit regularization of deep residual networks towards neural ODEscited by 20
  12. 2021Policy Analysis using Synthetic Controls in Continuous-Timecited by 18
38 more
  1. 2024On the Universality of Volume-Preserving and Coupling-Based Normalizing Flowscited by 18
  2. 2022Efficient Image Generation with Variadic Attention Headscited by 17
  3. 2021Ab-initio study of interacting fermions at finite temperature with neural canonical transformationcited by 16
  4. 2021On the expressivity of bi-Lipschitz normalizing flowscited by 16
  5. 2023Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraintscited by 15
  6. 2024When are dynamical systems learned from time series data statistically accurate?cited by 15
  7. 2023Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximationcited by 13
  8. 2021Density estimation on low-dimensional manifolds: an inflation-deflation approachcited by 11
  9. 2021Equivariant Finite Normalizing Flowscited by 11
  10. 2022Dynamical systems' based neural networkscited by 10
  11. 2021Jacobian Determinant of Normalizing Flowscited by 8
  12. 2024Feedback Favors the Generalization of Neural ODEscited by 8
  13. 2025Efficient Regression-Based Training of Normalizing Flows for Boltzmann Generatorscited by 8
  14. 2021Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flowscited by 7
  15. 2023Controllable Video Generation by Learning the Underlying Dynamical System with Neural ODEcited by 7
  16. 2022Whitening Convergence Rate of Coupling-based Normalizing Flowscited by 6
  17. 2024Partially Stochastic Infinitely Deep Bayesian Neural Networkscited by 6
  18. 2021Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamicscited by 5
  19. 2023Stability-Informed Initialization of Neural Ordinary Differential Equationscited by 5
  20. 2021Approximation capabilities of measure-preserving neural networkscited by 4
  21. 2022Neural Piecewise-Constant Delay Differential Equationscited by 4
  22. 2024Switched Flow Matching: Eliminating Singularities via Switching ODEscited by 4
  23. 2023Do We Need an Encoder-Decoder to Model Dynamical Systems on Networks?cited by 3
  24. 2023On the Generalization and Approximation Capacities of Neural Controlled Differential Equationscited by 3
  25. 2023A Minimal Control Family of Dynamical Systems for Universal Approximationcited by 3
  26. 2025A Dynamical Systems Perspective on the Analysis of Neural Networkscited by 3
  27. 2023Universal Approximation Property of Hamiltonian Deep Neural Networkscited by 2
  28. 2023Mixed Variational Flows for Discrete Variablescited by 2
  29. 2024Equivariant Manifold Neural ODEs and Differential Invariantscited by 2
  30. 2024Demystifying the Token Dynamics of Deep Selective State Space Modelscited by 2
  31. 2025Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encodingcited by 2
  32. 2021Characteristic Neural Ordinary Differential Equationscited by 1
  33. 2022Convergence Analysis of Deep Residual Networkscited by 1
  34. 2024VeriFlow: Modeling Distributions for Neural Network Verificationcited by 1
  35. 2025Autonomous-Flow-Based Generationcited by 1
  36. 2026Enhancing Low-resolution Image Representation Through Normalizing Flowscited by 1
  37. 2021NeurInt : Learning to Interpolate through Neural ODEscited by 0
  38. 2022Selected aspects of complex, hypercomplex and fuzzy neural networkscited by 0

50 most cited of 56 All 56 (JSON)