Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference

This study presents a Bayesian maximum textita~posteriori (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint maximum textita~posteriori (JMAP) and variational Bayesian approximation (VBA), are compared to the LASSO, ridge regression and SINDy algorithms for sparse regression, by application to several dynamical systems with added Gaussian or Laplace noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or ``Gaussian norm'' ||vecy-hatvecy||2_M-1 = (vecy-hatvecy)^top M-1 (vecy-hatvecy), where vecy is a vector variable, hatvecy is its estimator and M is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection for the different systems and noise models examined.