# A Neural Network Ensemble Approach to System Identification

arXiv: https://arxiv.org/abs/2110.08382 · HTML: https://mlsanity.com/paper/2110.08382/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Data Driven Governing Equations Approximation Using Deep Neural Networks](https://mlsanity.com/paper/1811.05537/) (2018)
- [Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1801.01236/) (2018)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [Data-driven discovery of PDEs in complex datasets](https://mlsanity.com/paper/1808.10788/) (2018)
- [Lipschitz regularized Deep Neural Networks generalize and are adversarially robust](https://mlsanity.com/paper/1808.09540/) (2018)
- [Nonlinear Systems Identification Using Deep Dynamic Neural Networks](https://mlsanity.com/paper/1610.01439/) (2016)
- [On Robust Computation of Koopman Operator and Prediction in Random Dynamical Systems](https://mlsanity.com/paper/1803.08562/) (2018)
- [Learning Partial Differential Equations from Data Using Neural Networks](https://mlsanity.com/paper/1910.10262/) (2019)
- [System Identification Through Lipschitz Regularized Deep Neural Networks](https://mlsanity.com/paper/2009.03288/) (2020)

## Built on by (most cited first, 1 of 1)

- [Parameter Inference of Time Series by Delay Embeddings and Learning Differentiable Operators](https://mlsanity.com/paper/2203.06269/) (2022)

## Abstract (arXiv)

We present a new algorithm for learning unknown governing equations from trajectory data, using and ensemble of neural networks. Given samples of solutions x(t) to an unknown dynamical system dotx(t)=f(t,x(t)), we approximate the function f using an ensemble of neural networks. We express the equation in integral form and use Euler method to predict the solution at every successive time step using at each iteration a different neural network as a prior for f. This procedure yields M-1 time-independent networks, where M is the number of time steps at which x(t) is observed. Finally, we obtain a single function f(t,x(t)) by neural network interpolation. Unlike our earlier work, where we numerically computed the derivatives of data, and used them as target in a Lipschitz regularized neural network to approximate f, our new method avoids numerical differentiations, which are unstable in presence of noise. We test the new algorithm on multiple examples both with and without noise in the data. We empirically show that generalization and recovery of the governing equation improve by adding a Lipschitz regularization term in our loss function and that this method improves our previous one especially in presence of noise, when numerical differentiation provides low quality target data. Finally, we compare our results with the method proposed by Raissi, et al. arXiv:1801.01236 (2018) and with SINDy.