# A Tutorial on the Non-Asymptotic Theory of System Identification

arXiv: https://arxiv.org/abs/2309.03873 · HTML: https://mlsanity.com/paper/2309.03873/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 7 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification](https://mlsanity.com/paper/1802.08334/) (2018)
- [Near optimal finite time identification of arbitrary linear dynamical systems](https://mlsanity.com/paper/1812.01251/) (2018)
- [Non-asymptotic Identification of LTI Systems from a Single Trajectory](https://mlsanity.com/paper/1806.05722/) (2018)
- [Finite Sample Analysis of Stochastic System Identification](https://mlsanity.com/paper/1903.09122/) (2019)
- [Finite Time Identification in Unstable Linear Systems](https://mlsanity.com/paper/1710.01852/) (2017)
- [Learning Linear Dynamical Systems with Semi-Parametric Least Squares](https://mlsanity.com/paper/1902.00768/) (2019)
- [Statistical Learning Theory for Control: A Finite Sample Perspective](https://mlsanity.com/paper/2209.05423/) (2022)
- [Learning without Concentration](https://mlsanity.com/paper/1401.0304/) (2014)
- [Learning nonlinear dynamical systems from a single trajectory](https://mlsanity.com/paper/2004.14681/) (2020)
- [Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2002.08538/) (2020)
- [Learning with Square Loss: Localization through Offset Rademacher Complexity](https://mlsanity.com/paper/1502.06134/) (2015)
- [Active Learning for Identification of Linear Dynamical Systems](https://mlsanity.com/paper/2002.00495/) (2020)
- [Learning with little mixing](https://mlsanity.com/paper/2206.08269/) (2022)
- [Learning from many trajectories](https://mlsanity.com/paper/2203.17193/) (2022)
- [Linear Systems can be Hard to Learn](https://mlsanity.com/paper/2104.01120/) (2021)
- [Near-optimal Offline and Streaming Algorithms for Learning Non-Linear Dynamical Systems](https://mlsanity.com/paper/2105.11558/) (2021)
- [Improved rates for prediction and identification of partially observed linear dynamical systems](https://mlsanity.com/paper/2011.10006/) (2020)
- [Non-asymptotic Closed-Loop System Identification using Autoregressive Processes and Hankel Model Reduction](https://mlsanity.com/paper/1909.02192/) (2019)
- [The noise level in linear regression with dependent data](https://mlsanity.com/paper/2305.11165/) (2023)
- [Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems](https://mlsanity.com/paper/2108.11959/) (2021)
- [A note on the smallest eigenvalue of the empirical covariance of causal Gaussian processes](https://mlsanity.com/paper/2212.09508/) (2022)

## Built on by (most cited first, 26 of 26)

- [Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Loss](https://mlsanity.com/paper/2402.05928/) (2024)
- [Active Learning for Control-Oriented Identification of Nonlinear Systems](https://mlsanity.com/paper/2404.09030/) (2024)
- [Nonasymptotic Regret Analysis of Adaptive Linear Quadratic Control with Model Misspecification](https://mlsanity.com/paper/2401.00073/) (2023)
- [Learning Linear Dynamics from Bilinear Observations](https://mlsanity.com/paper/2409.16499/) (2024)
- [Finite Sample Identification of Partially Observed Bilinear Dynamical Systems](https://mlsanity.com/paper/2501.07652/) (2025)
- [Regret Analysis of Multi-task Representation Learning for Linear-Quadratic Adaptive Control](https://mlsanity.com/paper/2407.05781/) (2024)
- [Outlier-Robust Linear System Identification Under Heavy-tailed Noise](https://mlsanity.com/paper/2501.00421/) (2024)
- [Rate-Optimal Non-Asymptotics for the Quadratic Prediction Error Method](https://mlsanity.com/paper/2404.07937/) (2024)
- [Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples](https://mlsanity.com/paper/2410.11227/) (2024)
- [Nonconvex Linear System Identification with Minimal State Representation](https://mlsanity.com/paper/2504.18791/) (2025)
- [LUCID: Learning-Enabled Uncertainty-Aware Certification of Stochastic Dynamical Systems](https://mlsanity.com/paper/2512.11750/) (2025)
- [Finite Sample Frequency Domain Identification](https://mlsanity.com/paper/2404.01100/) (2024)
- [A least-square method for non-asymptotic identification in linear switching control](https://mlsanity.com/paper/2404.08120/) (2024)
- [Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums](https://mlsanity.com/paper/2406.05440/) (2024)
- [Sample Complexity of the Sign-Perturbed Sums Method](https://mlsanity.com/paper/2409.01243/) (2024)
- [End-to-end guarantees for indirect data-driven control of bilinear systems with finite stochastic data](https://mlsanity.com/paper/2409.18010/) (2024)
- [Logarithmic Regret and Polynomial Scaling in Online Multi-step-ahead Prediction](https://mlsanity.com/paper/2511.12467/) (2025)
- [Optimistic Online LQR via Intrinsic Rewards](https://mlsanity.com/paper/2603.28938/) (2026)
- [Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights](https://mlsanity.com/paper/2604.17578/) (2026)
- [Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems](https://mlsanity.com/paper/2312.05382/) (2023)
- [Frequency Response Identification of Low-Order Systems: Finite-Sample Analysis](https://mlsanity.com/paper/2508.17142/) (2025)
- [Near-optimal Rank Adaptive Inference of High Dimensional Matrices](https://mlsanity.com/paper/2510.08117/) (2025)
- [Online Learning for Supervisory Switching Control](https://mlsanity.com/paper/2603.14762/) (2026)
- [CLT-Optimal Parameter Error Bounds for Linear System Identification](https://mlsanity.com/paper/2604.21270/) (2026)
- [Two-Layer Linear Auto-Regressive Models Estimate Latent States](https://mlsanity.com/paper/2606.12691/) (2026)
- [Learning Linear Systems under Heavy-Tailed Noise: A Non-Asymptotic Analysis from A Single Trajectory](https://mlsanity.com/paper/2610.00637/) (2026)

## Abstract (arXiv)

This tutorial serves as an introduction to recently developed non-asymptotic methods in the theory of -- mainly linear -- system identification. We emphasize tools we deem particularly useful for a range of problems in this domain, such as the covering technique, the Hanson-Wright Inequality and the method of self-normalized martingales. We then employ these tools to give streamlined proofs of the performance of various least-squares based estimators for identifying the parameters in autoregressive models. We conclude by sketching out how the ideas presented herein can be extended to certain nonlinear identification problems.