# Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification

arXiv: https://arxiv.org/abs/2212.13902 · HTML: https://mlsanity.com/paper/2212.13902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 90 references that are in the map.
- First posted on arXiv 20 Dec 2022.

## Builds on (most cited first, 29 of 29)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Essentially No Barriers in Neural Network Energy Landscape](https://mlsanity.com/paper/1803.00885/) (2018)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [A Bayesian Perspective on Generalization and Stochastic Gradient Descent](https://mlsanity.com/paper/1710.06451/) (2017)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [Data-driven approximation of the Koopman generator: Model reduction, system identification, and control](https://mlsanity.com/paper/1909.10638/) (2019)
- [Non-asymptotic Identification of LTI Systems from a Single Trajectory](https://mlsanity.com/paper/1806.05722/) (2018)
- [Finite Sample Analysis of Stochastic System Identification](https://mlsanity.com/paper/1903.09122/) (2019)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/1810.13075/) (2018)
- [Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems](https://mlsanity.com/paper/2003.11227/) (2020)
- [Non-asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectories](https://mlsanity.com/paper/2009.00739/) (2020)
- [Continuous-time system identification with neural networks: Model structures and fitting criteria](https://mlsanity.com/paper/2006.02915/) (2020)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation](https://mlsanity.com/paper/2007.07383/) (2020)
- [On the smoothness of nonlinear system identification](https://mlsanity.com/paper/1905.00820/) (2019)
- [Bayesian System ID: Optimal management of parameter, model, and measurement uncertainty](https://mlsanity.com/paper/2003.02359/) (2020)
- [Deep Multi-Output Forecasting: Learning to Accurately Predict Blood Glucose Trajectories](https://mlsanity.com/paper/1806.05357/) (2018)
- [Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint](https://mlsanity.com/paper/1507.00564/) (2015)
- [Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows](https://mlsanity.com/paper/2002.08335/) (2020)
- [A Family of Iterative Gauss-Newton Shooting Methods for Nonlinear Optimal Control](https://mlsanity.com/paper/1711.11006/) (2017)
- [Adaptive Model Predictive Control for A Class of Constrained Linear Systems with Parametric Uncertainties](https://mlsanity.com/paper/1901.03930/) (2019)
- [An Efficient Move Blocking Strategy for Multiple Shooting based Nonlinear Model Predictive Control](https://mlsanity.com/paper/1909.09710/) (2019)
- [Online Data-Enabled Predictive Control](https://mlsanity.com/paper/2003.03866/) (2020)

## Built on by (most cited first, 1 of 1)

- [Bayesian identification of nonseparable Hamiltonians with multiplicative noise using deep learning and reduced-order modeling](https://mlsanity.com/paper/2401.12476/) (2024)

## Abstract (arXiv)

This paper considers the problem of system identification (ID) of linear and nonlinear non-autonomous systems from noisy and sparse data. We propose and analyze an objective function derived from a Bayesian formulation for learning a hidden Markov model with stochastic dynamics. We then analyze this objective function in the context of several state-of-the-art approaches for both linear and nonlinear system ID. In the former, we analyze least squares approaches for Markov parameter estimation, and in the latter, we analyze the multiple shooting approach. We demonstrate the limitations of the optimization problems posed by these existing methods by showing that they can be seen as special cases of the proposed optimization objective under certain simplifying assumptions: conditional independence of data and zero model error. Furthermore, we observe that our proposed approach has improved smoothness and inherent regularization that make it well-suited for system ID and provide mathematical explanations for these characteristics' origins. Finally, numerical simulations demonstrate a mean squared error over 8.7 times lower compared to multiple shooting when data are noisy and/or sparse. Moreover, the proposed approach can identify accurate and generalizable models even when there are more parameters than data or when the underlying system exhibits chaotic behavior.