# Continuous-time identification of dynamic state-space models by deep subspace encoding

arXiv: https://arxiv.org/abs/2204.09405 · HTML: https://mlsanity.com/paper/2204.09405/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 20 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [On Interpretability of Artificial Neural Networks: A Survey](https://mlsanity.com/paper/2001.02522/) (2020)
- [Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: a case study with the Lorenz 96 model](https://mlsanity.com/paper/2001.01520/) (2020)
- [Learning Dynamical Systems from Partial Observations](https://mlsanity.com/paper/1902.11136/) (2019)
- [Continuous-time system identification with neural networks: Model structures and fitting criteria](https://mlsanity.com/paper/2006.02915/) (2020)
- [Nonlinear state-space identification using deep encoder networks](https://mlsanity.com/paper/2012.07697/) (2020)
- [dynoNet: a neural network architecture for learning dynamical systems](https://mlsanity.com/paper/2006.02250/) (2020)
- [A flexible state space model for learning nonlinear dynamical systems](https://mlsanity.com/paper/1603.05486/) (2016)
- [On the smoothness of nonlinear system identification](https://mlsanity.com/paper/1905.00820/) (2019)
- [Sparse Bayesian Deep Learning for Dynamic System Identification](https://mlsanity.com/paper/2107.12910/) (2021)
- [Nonlinear system identification with regularized Tensor Network B-splines](https://mlsanity.com/paper/2003.07594/) (2020)
- [Efficient Multidimensional Regularization for Volterra Series Estimation](https://mlsanity.com/paper/1804.10026/) (2018)

## Built on by (most cited first, 6 of 6)

- [Baseline Results for Selected Nonlinear System Identification Benchmarks](https://mlsanity.com/paper/2405.10779/) (2024)
- [SOLIS: Physics-Informed Learning of Interpretable Neural Surrogates for Nonlinear Systems](https://mlsanity.com/paper/2604.14879/) (2026)
- [Discovering Latent Response Laws in Forced Physical Systems](https://mlsanity.com/paper/2607.09801/) (2026)
- [State Derivative Normalization for Continuous-Time Deep Neural Networks](https://mlsanity.com/paper/2401.02902/) (2024)
- [Tensor Network Kernel Machines: A JAX Framework for Machine Learning and Nonlinear System Identification](https://mlsanity.com/paper/2608.07043/) (2026)
- [Dirac-Interconnected Neural Elements: Discovering Modularity in Physical Systems Without Reduction](https://mlsanity.com/paper/2610.02960/) (2026)

## Abstract (arXiv)

Continuous-time (CT) modeling has proven to provide improved sample efficiency and interpretability in learning the dynamical behavior of physical systems compared to discrete-time (DT) models. However, even with numerous recent developments, the CT nonlinear state-space (NL-SS) model identification problem remains to be solved in full, considering common experimental aspects such as the presence of external inputs, measurement noise, latent states, and general robustness. This paper presents a novel estimation method that addresses all these aspects and that can obtain state-of-the-art results on multiple benchmarks with compact fully connected neural networks capturing the CT dynamics. The proposed estimation method called the subspace encoder approach (SUBNET) ascertains these results by efficiently approximating the complete simulation loss by evaluating short simulations on subsections of the data, by using an encoder function to estimate the initial state for each subsection and a novel state-derivative normalization to ensure stability and good numerical conditioning of the training process. We prove that the use of subsections increases cost function smoothness together with the necessary requirements for the existence of the encoder function and we show that the proposed state-derivative normalization is essential for reliable estimation of CT NL-SS models.