# Learning Dynamics from Noisy Measurements using Deep Learning with a Runge-Kutta Constraint

arXiv: https://arxiv.org/abs/2109.11446 · HTML: https://mlsanity.com/paper/2109.11446/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 59 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Learning Deep Neural Network Representations for Koopman Operators of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1708.06850/) (2017)
- [Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1801.01236/) (2018)
- [DeepMoD: Deep learning for Model Discovery in noisy data](https://mlsanity.com/paper/1904.09406/) (2019)
- [Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model](https://mlsanity.com/paper/1803.04779/) (2018)
- [Discovery of Nonlinear Dynamical Systems using a Runge-Kutta Inspired Dictionary-based Sparse Regression Approach](https://mlsanity.com/paper/2105.04869/) (2021)
- [Long-time predictive modeling of nonlinear dynamical systems using neural networks](https://mlsanity.com/paper/1805.12547/) (2018)

## Built on by (most cited first, 7 of 7)

- [Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2402.18377/) (2024)
- [Generalized Quadratic Embeddings for Nonlinear Dynamics using Deep Learning](https://mlsanity.com/paper/2211.00357/) (2022)
- [A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization](https://mlsanity.com/paper/2208.09104/) (2022)
- [A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms](https://mlsanity.com/paper/2211.12386/) (2022)
- [Learning Low-Dimensional Quadratic-Embeddings of High-Fidelity Nonlinear Dynamics using Deep Learning](https://mlsanity.com/paper/2111.12995/) (2021)
- [NeuralPrefix: A Zero-shot Sensory Data Imputation Plugin](https://mlsanity.com/paper/2502.05883/) (2025)
- [BLADE: Bayesian Langevin Active Discovery with Replica Exchange for Identification of Complex Systems](https://mlsanity.com/paper/2503.02983/) (2025)

## Abstract (arXiv)

Measurement noise is an integral part while collecting data of a physical process. Thus, noise removal is a necessary step to draw conclusions from these data, and it often becomes quite essential to construct dynamical models using these data. We discuss a methodology to learn differential equation(s) using noisy and sparsely sampled measurements. In our methodology, the main innovation can be seen in of integration of deep neural networks with a classical numerical integration method. Precisely, we aim at learning a neural network that implicitly represents the data and an additional neural network that models the vector fields of the dependent variables. We combine these two networks by enforcing the constraint that the data at the next time-steps can be given by following a numerical integration scheme such as the fourth-order Runge-Kutta scheme. The proposed framework to learn a model predicting the vector field is highly effective under noisy measurements. The approach can handle scenarios where dependent variables are not available at the same temporal grid. We demonstrate the effectiveness of the proposed method to learning models using data obtained from various differential equations. The proposed approach provides a promising methodology to learn dynamic models, where the first-principle understanding remains opaque.