# Learning POD of Complex Dynamics Using Heavy-ball Neural ODEs

arXiv: https://arxiv.org/abs/2202.12373 · HTML: https://mlsanity.com/paper/2202.12373/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 65 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Learning Long-Term Dependencies in Irregularly-Sampled Time Series](https://mlsanity.com/paper/2006.04418/) (2020)
- [On Second Order Behaviour in Augmented Neural ODEs](https://mlsanity.com/paper/2006.07220/) (2020)
- [Heavy Ball Neural Ordinary Differential Equations](https://mlsanity.com/paper/2110.04840/) (2021)
- [Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent](https://mlsanity.com/paper/2002.10583/) (2020)
- [Model Reduction with Memory and the Machine Learning of Dynamical Systems](https://mlsanity.com/paper/1808.04258/) (2018)
- [DR-RNN: A deep residual recurrent neural network for model reduction](https://mlsanity.com/paper/1709.00939/) (2017)
- [MomentumRNN: Integrating Momentum into Recurrent Neural Networks](https://mlsanity.com/paper/2006.06919/) (2020)
- [How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies](https://mlsanity.com/paper/2110.07034/) (2021)
- [Reduced order modeling of subsurface multiphase flow models using deep residual recurrent neural networks](https://mlsanity.com/paper/1810.10422/) (2018)
- [Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum](https://mlsanity.com/paper/2012.02188/) (2020)
- [Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization](https://mlsanity.com/paper/2110.09057/) (2021)
- [Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics](https://mlsanity.com/paper/2104.13962/) (2021)
- [Data-driven reduced order modeling of environmental hydrodynamics using deep autoencoders and neural ODEs](https://mlsanity.com/paper/2107.02784/) (2021)

## Built on by (most cited first, 2 of 2)

- [Variational Rank Reduction Autoencoders for Generative Thermal Design](https://mlsanity.com/paper/2509.08515/) (2025)
- [Application of Reduced-Order Models for Temporal Multiscale Representations in the Prediction of Dynamical Systems](https://mlsanity.com/paper/2510.18925/) (2025)

## Abstract (arXiv)

Proper orthogonal decomposition (POD) allows reduced-order modeling of complex dynamical systems at a substantial level, while maintaining a high degree of accuracy in modeling the underlying dynamical systems. Advances in machine learning algorithms enable learning POD-based dynamics from data and making accurate and fast predictions of dynamical systems. In this paper, we leverage the recently proposed heavy-ball neural ODEs (HBNODEs) [Xia et al. NeurIPS, 2021] for learning data-driven reduced-order models (ROMs) in the POD context, in particular, for learning dynamics of time-varying coefficients generated by the POD analysis on training snapshots generated from solving full order models. HBNODE enjoys several practical advantages for learning POD-based ROMs with theoretical guarantees, including 1) HBNODE can learn long-term dependencies effectively from sequential observations and 2) HBNODE is computationally efficient in both training and testing. We compare HBNODE with other popular ROMs on several complex dynamical systems, including the von K'arm'an Street flow, the Kurganov-Petrova-Popov equation, and the one-dimensional Euler equations for fluids modeling.