# Dealing with Collinearity in Large-Scale Linear System Identification Using Gaussian Regression

arXiv: https://arxiv.org/abs/2302.10959 · HTML: https://mlsanity.com/paper/2302.10959/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 52 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Sparse Bayesian Deep Learning for Dynamic System Identification](https://mlsanity.com/paper/2107.12910/) (2021)
- [Prediction error identification of linear dynamic networks with rank-reduced noise](https://mlsanity.com/paper/1711.06369/) (2017)
- [A new kernel-based approach to system identification with quantized output data](https://mlsanity.com/paper/1610.00470/) (2016)
- [A local direct method for module identification in dynamic networks with correlated noise](https://mlsanity.com/paper/1908.00976/) (2019)
- [Identification of Low Rank Vector Processes](https://mlsanity.com/paper/2111.10899/) (2021)
- [Dynamic Network Reconstruction from Heterogeneous Datasets](https://mlsanity.com/paper/1612.01963/) (2016)
- [Identifiability of dynamical networks: which nodes need be measured?](https://mlsanity.com/paper/1709.04398/) (2017)
- [Identifiability of dynamical networks with partial node measurements](https://mlsanity.com/paper/1803.05885/) (2018)
- [Modeling of Low Rank Time Series](https://mlsanity.com/paper/2109.11814/) (2021)

## Abstract (arXiv)

Many problems arising in control require the determination of a mathematical model of the application. This has often to be performed starting from input-output data, leading to a task known as system identification in the engineering literature. One emerging topic in this field is estimation of networks consisting of several interconnected dynamic systems. We consider the linear setting assuming that system outputs are the result of many correlated inputs, hence making system identification severely ill-conditioned. This is a scenario often encountered when modeling complex cybernetics systems composed by many sub-units with feedback and algebraic loops. We develop a strategy cast in a Bayesian regularization framework where any impulse response is seen as realization of a zero-mean Gaussian process. Any covariance is defined by the so called stable spline kernel which includes information on smooth exponential decay. We design a novel Markov chain Monte Carlo scheme able to reconstruct the impulse responses posterior by efficiently dealing with collinearity. Our scheme relies on a variation of the Gibbs sampling technique: beyond considering blocks forming a partition of the parameter space, some other (overlapping) blocks are also updated on the basis of the level of collinearity of the system inputs. Theoretical properties of the algorithm are studied obtaining its convergence rate. Numerical experiments are included using systems containing hundreds of impulse responses and highly correlated inputs.