Universal Approximation Property of Hamiltonian Deep Neural Networks
Not on the map. Why?
Abstract (arXiv)
This paper investigates the universal approximation capabilities of Hamiltonian Deep Neural Networks (HDNNs) that arise from the discretization of Hamiltonian Neural Ordinary Differential Equations. Recently, it has been shown that HDNNs enjoy, by design, non-vanishing gradients, which provide numerical stability during training. However, although HDNNs have demonstrated state-of-the-art performance in several applications, a comprehensive study to quantify their expressivity is missing. In this regard, we provide a universal approximation theorem for HDNNs and prove that a portion of the flow of HDNNs can approximate arbitrary well any continuous function over a compact domain. This result provides a solid theoretical foundation for the practical use of HDNNs.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 21 Mar 2023.
Builds on
24 of 36 references.
References · 24
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2014Going Deeper with Convolutionscited by 4,231
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2019Deep Equilibrium Modelscited by 603
- 2019Augmented Neural ODEscited by 396
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2016On the Expressive Power of Deep Neural Networkscited by 302
- 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 125
- 2019AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networkscited by 112
- 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 78
- 2019Deep Learning via Dynamical Systems: An Approximation Perspectivecited by 78
12 more
- 2017Reversible Architectures for Arbitrarily Deep Residual Neural Networkscited by 69
- 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
- 2019Approximation Capabilities of Neural ODEs and Invertible Residual Networkscited by 56
- 2020Expressivity of Deep Neural Networkscited by 31
- 2016Learning Functions: When Is Deep Better Than Shallowcited by 19
- 2021Hamiltonian Deep Neural Networks Guaranteeing Non-vanishing Gradients by Designcited by 18
- 2020Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theorycited by 17
- 2022Robust Classification using Contractive Hamiltonian Neural ODEscited by 13
- 2022Dynamical systems' based neural networkscited by 10
- 2022Approximate Predictive Control Barrier Functions using Neural Networks: A Computationally Cheap and Permissive Safety Filtercited by 4
- 2022Physically Consistent Neural ODEs for Learning Multi-Physics Systemscited by 3
- 2022Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networkscited by 2
Built on by
Citing papers per year: 2024: 2.
2 papers.