# Deep Subspace Encoders for Nonlinear System Identification

arXiv: https://arxiv.org/abs/2210.14816 · HTML: https://mlsanity.com/paper/2210.14816/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 51 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Unbiasing Truncated Backpropagation Through Time](https://mlsanity.com/paper/1705.08209/) (2017)
- [Deep State Space Models for Nonlinear System Identification](https://mlsanity.com/paper/2003.14162/) (2020)
- [Continuous-time system identification with neural networks: Model structures and fitting criteria](https://mlsanity.com/paper/2006.02915/) (2020)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [On the smoothness of nonlinear system identification](https://mlsanity.com/paper/1905.00820/) (2019)
- [Identification of block-oriented nonlinear systems starting from linear approximations: A survey](https://mlsanity.com/paper/1607.01217/) (2016)
- [On the Initialization of Nonlinear LFR Model Identification with the Best Linear Approximation](https://mlsanity.com/paper/2004.05040/) (2020)
- [Improved Initialization of State-Space Artificial Neural Networks](https://mlsanity.com/paper/2103.14516/) (2021)
- [Learning neural state-space models: do we need a state estimator?](https://mlsanity.com/paper/2206.12928/) (2022)
- [Non-linear State-space Model Identification from Video Data using Deep Encoders](https://mlsanity.com/paper/2012.07721/) (2020)
- [Regularized Nonparametric Volterra Kernel Estimation](https://mlsanity.com/paper/1804.10435/) (2018)
- [Nonlinear system modeling based on constrained Volterra series estimates](https://mlsanity.com/paper/1804.07258/) (2018)

## Built on by (most cited first, 13 of 13)

- [From system models to class models: An in-context learning paradigm](https://mlsanity.com/paper/2308.13380/) (2023)
- [On the adaptation of in-context learners for system identification](https://mlsanity.com/paper/2312.04083/) (2023)
- [Enhanced Transformer architecture for in-context learning of dynamical systems](https://mlsanity.com/paper/2410.03291/) (2024)
- [Stable Linear Subspace Identification: A Machine Learning Approach](https://mlsanity.com/paper/2311.03197/) (2023)
- [Online Control-Informed Learning](https://mlsanity.com/paper/2410.03924/) (2024)
- [Manifold meta-learning for reduced-complexity neural system identification](https://mlsanity.com/paper/2504.11811/) (2025)
- [Efficient identification of linear, parameter-varying, and nonlinear systems with noise models](https://mlsanity.com/paper/2504.11982/) (2025)
- [Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach](https://mlsanity.com/paper/2304.02119/) (2023)
- [Orthogonal projection-based regularization for efficient model augmentation](https://mlsanity.com/paper/2501.05842/) (2025)
- [Tuning the burn-in phase in training recurrent neural networks improves their performance](https://mlsanity.com/paper/2602.10911/) (2026)
- [The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling](https://mlsanity.com/paper/2605.31547/) (2026)
- [Online learning of neural state-space models](https://mlsanity.com/paper/2607.17614/) (2026)
- [Scalable Identification of Physics-Based and Neural Dynamics from Partial Measurements](https://mlsanity.com/paper/2610.16559/) (2026)

## Abstract (arXiv)

Using Artificial Neural Networks (ANN) for nonlinear system identification has proven to be a promising approach, but despite of all recent research efforts, many practical and theoretical problems still remain open. Specifically, noise handling and models, issues of consistency and reliable estimation under minimisation of the prediction error are the most severe problems. The latter comes with numerous practical challenges such as explosion of the computational cost in terms of the number of data samples and the occurrence of instabilities during optimization. In this paper, we aim to overcome these issues by proposing a method which uses a truncated prediction loss and a subspace encoder for state estimation. The truncated prediction loss is computed by selecting multiple truncated subsections from the time series and computing the average prediction loss. To obtain a computationally efficient estimation method that minimizes the truncated prediction loss, a subspace encoder represented by an artificial neural network is introduced. This encoder aims to approximate the state reconstructability map of the estimated model to provide an initial state for each truncated subsection given past inputs and outputs. By theoretical analysis, we show that, under mild conditions, the proposed method is locally consistent, increases optimization stability, and achieves increased data efficiency by allowing for overlap between the subsections. Lastly, we provide practical insights and user guidelines employing a numerical example and state-of-the-art benchmark results.