# Learning Low-dimensional Latent Dynamics from High-dimensional Observations: Non-asymptotics and Lower Bounds

arXiv: https://arxiv.org/abs/2405.06089 · HTML: https://mlsanity.com/paper/2405.06089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 58 references that are in the map.
- First posted on arXiv 9 May 2024.

## Builds on (most cited first, 18 of 18)

- [Provable Meta-Learning of Linear Representations](https://mlsanity.com/paper/2002.11684/) (2020)
- [Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification](https://mlsanity.com/paper/1802.08334/) (2018)
- [Near optimal finite time identification of arbitrary linear dynamical systems](https://mlsanity.com/paper/1812.01251/) (2018)
- [Non-asymptotic Identification of LTI Systems from a Single Trajectory](https://mlsanity.com/paper/1806.05722/) (2018)
- [Finite Time Identification in Unstable Linear Systems](https://mlsanity.com/paper/1710.01852/) (2017)
- [Machine Learning Advances for Time Series Forecasting](https://mlsanity.com/paper/2012.12802/) (2020)
- [Non-asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectories](https://mlsanity.com/paper/2009.00739/) (2020)
- [Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery](https://mlsanity.com/paper/1711.09492/) (2017)
- [A New Approach to Learning Linear Dynamical Systems](https://mlsanity.com/paper/2301.09519/) (2023)
- [Streaming PCA and Subspace Tracking: The Missing Data Case](https://mlsanity.com/paper/1806.04609/) (2018)
- [Improved rates for prediction and identification of partially observed linear dynamical systems](https://mlsanity.com/paper/2011.10006/) (2020)
- [Learning Partially Observed Linear Dynamical Systems from Logarithmic Number of Samples](https://mlsanity.com/paper/2010.04015/) (2020)
- [Sample Complexity of Dictionary Learning and other Matrix Factorizations](https://mlsanity.com/paper/1312.3790/) (2013)
- [Efficient learning of hidden state LTI state space models of unknown order](https://mlsanity.com/paper/2202.01625/) (2022)
- [Non asymptotic estimation lower bounds for LTI state space models with Cram'er-Rao and van Trees](https://mlsanity.com/paper/2109.08582/) (2021)
- [Data-driven Optimal Filtering for Linear Systems with Unknown Noise Covariances](https://mlsanity.com/paper/2305.17836/) (2023)
- [Finite-Time Model Inference From A Single Noisy Trajectory](https://mlsanity.com/paper/2010.06616/) (2020)
- [Consistent Estimation of Low-Dimensional Latent Structure in High-Dimensional Data](https://mlsanity.com/paper/1510.03497/) (2015)

## Built on by (most cited first, 2 of 2)

- [Near-optimal Rank Adaptive Inference of High Dimensional Matrices](https://mlsanity.com/paper/2510.08117/) (2025)
- [Calibrated Principal Component Regression](https://mlsanity.com/paper/2510.19020/) (2025)

## Abstract (arXiv)

In this paper, we focus on learning a linear time-invariant (LTI) model with low-dimensional latent variables but high-dimensional observations. We provide an algorithm that recovers the high-dimensional features, i.e. column space of the observer, embeds the data into low dimensions and learns the low-dimensional model parameters. Our algorithm enjoys a sample complexity guarantee of order tildeO(n/ε2), where n is the observation dimension. We further establish a fundamental lower bound indicating this complexity bound is optimal up to logarithmic factors and dimension-independent constants. We show that this inevitable linear factor of n is due to the learning error of the observer's column space in the presence of high-dimensional noises. Extending our results, we consider a meta-learning problem inspired by various real-world applications, where the observer column space can be collectively learned from datasets of multiple LTI systems. An end-to-end algorithm is then proposed, facilitating learning LTI systems from a meta-dataset which breaks the sample complexity lower bound in certain scenarios.