# Improved Initialization of State-Space Artificial Neural Networks

arXiv: https://arxiv.org/abs/2103.14516 · HTML: https://mlsanity.com/paper/2103.14516/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 26 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Model structures and fitting criteria for system identification with neural networks](https://mlsanity.com/paper/1911.13034/) (2019)
- [Identification of block-oriented nonlinear systems starting from linear approximations: A survey](https://mlsanity.com/paper/1607.01217/) (2016)
- [On the Initialization of Nonlinear LFR Model Identification with the Best Linear Approximation](https://mlsanity.com/paper/2004.05040/) (2020)
- [Linear System Identification in a Nonlinear Setting - Nonparametric analysis of the nonlinear distortions and their impact on the best linear approximation](https://mlsanity.com/paper/1804.09587/) (2018)
- [A nonlinear state-space approach to hysteresis identification](https://mlsanity.com/paper/1610.09138/) (2016)
- [Probabilistic learning of nonlinear dynamical systems using sequential Monte Carlo](https://mlsanity.com/paper/1703.02419/) (2017)

## Built on by (most cited first, 8 of 8)

- [Deep Subspace Encoders for Nonlinear System Identification](https://mlsanity.com/paper/2210.14816/) (2022)
- [Stable Linear Subspace Identification: A Machine Learning Approach](https://mlsanity.com/paper/2311.03197/) (2023)
- [AI-Aided Kalman Filters](https://mlsanity.com/paper/2410.12289/) (2024)
- [Manifold meta-learning for reduced-complexity neural system identification](https://mlsanity.com/paper/2504.11811/) (2025)
- [Decoupling multivariate functions using a nonparametric filtered tensor decomposition](https://mlsanity.com/paper/2205.11153/) (2022)
- [Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach](https://mlsanity.com/paper/2304.02119/) (2023)
- [Identification For Control Based on Neural Networks: Approximately Linearizable Models](https://mlsanity.com/paper/2409.15858/) (2024)
- [State Derivative Normalization for Continuous-Time Deep Neural Networks](https://mlsanity.com/paper/2401.02902/) (2024)

## Abstract (arXiv)

The identification of black-box nonlinear state-space models requires a flexible representation of the state and output equation. Artificial neural networks have proven to provide such a representation. However, as in many identification problems, a nonlinear optimization problem needs to be solved to obtain the model parameters (layer weights and biases). A well-thought initialization of these model parameters can often avoid that the nonlinear optimization algorithm converges to a poorly performing local minimum of the considered cost function. This paper introduces an improved initialization approach for nonlinear state-space models represented as a recurrent artificial neural network and emphasizes the importance of including an explicit linear term in the model structure. Some of the neural network weights are initialized starting from a linear approximation of the nonlinear system, while others are initialized using random values or zeros. The effectiveness of the proposed initialization approach over previously proposed methods is illustrated on two benchmark examples.