# Physics-enhanced Neural Networks in the Small Data Regime

arXiv: https://arxiv.org/abs/2111.10329 · HTML: https://mlsanity.com/paper/2111.10329/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 19 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [Understanding and mitigating gradient pathologies in physics-informed neural networks](https://mlsanity.com/paper/2001.04536/) (2020)
- [Hamiltonian Graph Networks with ODE Integrators](https://mlsanity.com/paper/1909.12790/) (2019)
- [Trajectory Prediction using Equivariant Continuous Convolution](https://mlsanity.com/paper/2010.11344/) (2020)
- [Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems](https://mlsanity.com/paper/1912.12976/) (2019)
- [Physics-aware, probabilistic model order reduction with guaranteed stability](https://mlsanity.com/paper/2101.05834/) (2021)

## Built on by (most cited first, 3 of 3)

- [Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs](https://mlsanity.com/paper/2205.12550/) (2022)
- [Frequency-Separable Hamiltonian Neural Network for Multi-Timescale Dynamics](https://mlsanity.com/paper/2603.06354/) (2026)
- [Can physical information aid the generalization ability of Neural Networks for hydraulic modeling?](https://mlsanity.com/paper/2403.08589/) (2024)

## Abstract (arXiv)

Identifying the dynamics of physical systems requires a machine learning model that can assimilate observational data, but also incorporate the laws of physics. Neural Networks based on physical principles such as the Hamiltonian or Lagrangian NNs have recently shown promising results in generating extrapolative predictions and accurately representing the system's dynamics. We show that by additionally considering the actual energy level as a regularization term during training and thus using physical information as inductive bias, the results can be further improved. Especially in the case where only small amounts of data are available, these improvements can significantly enhance the predictive capability. We apply the proposed regularization term to a Hamiltonian Neural Network (HNN) and Constrained Hamiltonian Neural Network (CHHN) for a single and double pendulum, generate predictions under unseen initial conditions and report significant gains in predictive accuracy.