# Learning neural state-space models: do we need a state estimator?

arXiv: https://arxiv.org/abs/2206.12928 · HTML: https://mlsanity.com/paper/2206.12928/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 26 Jun 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [dynoNet: a neural network architecture for learning dynamical systems](https://mlsanity.com/paper/2006.02250/) (2020)
- [On the smoothness of nonlinear system identification](https://mlsanity.com/paper/1905.00820/) (2019)
- [Model structures and fitting criteria for system identification with neural networks](https://mlsanity.com/paper/1911.13034/) (2019)
- [Deep Identification of Nonlinear Systems in Koopman Form](https://mlsanity.com/paper/2110.02583/) (2021)
- [Recursive nonlinear-system identification using latent variables](https://mlsanity.com/paper/1606.04366/) (2016)

## Built on by (most cited first, 6 of 6)

- [Deep Subspace Encoders for Nonlinear System Identification](https://mlsanity.com/paper/2210.14816/) (2022)
- [Meta-Learning of Neural State-Space Models Using Data From Similar Systems](https://mlsanity.com/paper/2211.07768/) (2022)
- [Model order reduction of deep structured state-space models: A system-theoretic approach](https://mlsanity.com/paper/2403.14833/) (2024)
- [Meta-Learning for Physically-Constrained Neural System Identification](https://mlsanity.com/paper/2501.06167/) (2025)
- [Manifold meta-learning for reduced-complexity neural system identification](https://mlsanity.com/paper/2504.11811/) (2025)
- [Tuning the burn-in phase in training recurrent neural networks improves their performance](https://mlsanity.com/paper/2602.10911/) (2026)

## Abstract (arXiv)

In recent years, several algorithms for system identification with neural state-space models have been introduced. Most of the proposed approaches are aimed at reducing the computational complexity of the learning problem, by splitting the optimization over short sub-sequences extracted from a longer training dataset. Different sequences are then processed simultaneously within a minibatch, taking advantage of modern parallel hardware for deep learning. An issue arising in these methods is the need to assign an initial state for each of the sub-sequences, which is required to run simulations and thus to evaluate the fitting loss. In this paper, we provide insights for calibration of neural state-space training algorithms based on extensive experimentation and analyses performed on two recognized system identification benchmarks. Particular focus is given to the choice and the role of the initial state estimation. We demonstrate that advanced initial state estimation techniques are really required to achieve high performance on certain classes of dynamical systems, while for asymptotically stable ones basic procedures such as zero or random initialization already yield competitive performance.