# KAM Theory Meets Statistical Learning Theory: Hamiltonian Neural Networks with Non-Zero Training Loss

arXiv: https://arxiv.org/abs/2102.11923 · HTML: https://mlsanity.com/paper/2102.11923/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 22 Feb 2021.

## Builds on (most cited first, 18 of 18)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [Hamiltonian Graph Networks with ODE Integrators](https://mlsanity.com/paper/1909.12790/) (2019)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks](https://mlsanity.com/paper/2012.03133/) (2020)
- [Euclideanizing Flows: Diffeomorphic Reduction for Learning Stable Dynamical Systems](https://mlsanity.com/paper/2005.13143/) (2020)
- [Nonseparable Symplectic Neural Networks](https://mlsanity.com/paper/2010.12636/) (2020)
- [Sparse Symplectically Integrated Neural Networks](https://mlsanity.com/paper/2006.12972/) (2020)
- [Neural Canonical Transformation with Symplectic Flows](https://mlsanity.com/paper/1910.00024/) (2019)
- [Learning Symmetries of Classical Integrable Systems](https://mlsanity.com/paper/1906.04645/) (2019)
- [Learning Principle of Least Action with Reinforcement Learning](https://mlsanity.com/paper/2011.11891/) (2020)
- [Time-Continuous Energy-Conservation Neural Network for Structural Dynamics Analysis](https://mlsanity.com/paper/2012.14334/) (2020)

## Built on by (most cited first, 2 of 2)

- [Unifying physical systems' inductive biases in neural ODE using dynamics constraints](https://mlsanity.com/paper/2208.02632/) (2022)
- [Poisson-Dirac Neural Networks for Modeling Coupled Dynamical Systems across Domains](https://mlsanity.com/paper/2410.11480/) (2024)

## Abstract (arXiv)

Many physical phenomena are described by Hamiltonian mechanics using an energy function (the Hamiltonian). Recently, the Hamiltonian neural network, which approximates the Hamiltonian as a neural network, and its extensions have attracted much attention. This is a very powerful method, but its use in theoretical studies remains limited. In this study, by combining the statistical learning theory and Kolmogorov-Arnold-Moser (KAM) theory, we provide a theoretical analysis of the behavior of Hamiltonian neural networks when the learning error is not completely zero. A Hamiltonian neural network with non-zero errors can be considered as a perturbation from the true dynamics, and the perturbation theory of the Hamilton equation is widely known as the KAM theory. To apply the KAM theory, we provide a generalization error bound for Hamiltonian neural networks by deriving an estimate of the covering number of the gradient of the multi-layer perceptron, which is the key ingredient of the model. This error bound gives an L^∞ bound on the Hamiltonian that is required in the application of the KAM theory.