# On the smoothness of nonlinear system identification

arXiv: https://arxiv.org/abs/1905.00820 · HTML: https://mlsanity.com/paper/1905.00820/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2019.

## Built on by (most cited first, 17 of 17)

- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [Latent Neural ODEs with Sparse Bayesian Multiple Shooting](https://mlsanity.com/paper/2210.03466/) (2022)
- [Deep Subspace Encoders for Nonlinear System Identification](https://mlsanity.com/paper/2210.14816/) (2022)
- [Training Recurrent Neural Networks by Sequential Least Squares and the Alternating Direction Method of Multipliers](https://mlsanity.com/paper/2112.15348/) (2021)
- [Learning Dynamical Systems from Noisy Sensor Measurements using Multiple Shooting](https://mlsanity.com/paper/2106.11712/) (2021)
- [Continuous-time identification of dynamic state-space models by deep subspace encoding](https://mlsanity.com/paper/2204.09405/) (2022)
- [Learning neural state-space models: do we need a state estimator?](https://mlsanity.com/paper/2206.12928/) (2022)
- [Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processes](https://mlsanity.com/paper/2303.01925/) (2023)
- [Learning Space-Time Continuous Neural PDEs from Partially Observed States](https://mlsanity.com/paper/2307.04110/) (2023)
- [Effective Latent Differential Equation Models via Attention and Multiple Shooting](https://mlsanity.com/paper/2307.05735/) (2023)
- [Efficient identification of linear, parameter-varying, and nonlinear systems with noise models](https://mlsanity.com/paper/2504.11982/) (2025)
- [Likelihood-based generalization of Markov parameter estimation and multiple shooting objectives in system identification](https://mlsanity.com/paper/2212.13902/) (2022)
- [Initialization Approach for Nonlinear State-Space Identification via the Subspace Encoder Approach](https://mlsanity.com/paper/2304.02119/) (2023)
- [Orthogonal projection-based regularization for efficient model augmentation](https://mlsanity.com/paper/2501.05842/) (2025)
- [Tuning the burn-in phase in training recurrent neural networks improves their performance](https://mlsanity.com/paper/2602.10911/) (2026)
- [Distributed Identification of Contracting and/or Monotone Network Dynamics](https://mlsanity.com/paper/2107.14309/) (2021)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)

## Abstract (arXiv)

We shed new light on the textitsmoothness of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems. We show that for regions of the parameter space where the model is not contractive, the Lipschitz constant and β-smoothness of the objective function might blow up exponentially with the simulation length, making it hard to numerically find minima within those regions or, even, to escape from them. In addition to providing theoretical understanding of this problem, this paper also proposes the use of multiple shooting as a viable solution. The proposed method minimizes the error between a prediction model and the observed values. Rather than running the prediction model over the entire dataset, multiple shooting splits the data into smaller subsets and runs the prediction model over each subset, making the simulation length a design parameter and making it possible to solve problems that would be infeasible using a standard approach. The equivalence to the original problem is obtained by including constraints in the optimization. The new method is illustrated by estimating the parameters of nonlinear systems with chaotic or unstable behavior, as well as neural networks. We also present a comparative analysis of the proposed method with multi-step-ahead prediction error minimization.