# Data-driven discovery of intrinsic dynamics

arXiv: https://arxiv.org/abs/2108.05928 · HTML: https://mlsanity.com/paper/2108.05928/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 79 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [Multiscale Simulations of Complex Systems by Learning their Effective Dynamics](https://mlsanity.com/paper/2006.13431/) (2020)
- [Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems](https://mlsanity.com/paper/1808.01346/) (2018)
- [Deep learning to discover and predict dynamics on an inertial manifold](https://mlsanity.com/paper/2001.04263/) (2019)
- [Implicit Rank-Minimizing Autoencoder](https://mlsanity.com/paper/2010.00679/) (2020)
- [Chart Auto-Encoders for Manifold Structured Data](https://mlsanity.com/paper/1912.10094/) (2019)

## Built on by (most cited first, 30 of 45)

- [Generative learning for nonlinear dynamics](https://mlsanity.com/paper/2311.04128/) (2023)
- [Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning](https://mlsanity.com/paper/2205.00579/) (2022)
- [Manifold Learning by Mixture Models of VAEs for Inverse Problems](https://mlsanity.com/paper/2303.15244/) (2023)
- [Autoencoders for discovering manifold dimension and coordinates in data from complex dynamical systems](https://mlsanity.com/paper/2305.01090/) (2023)
- [Interpretable statistical representations of neural population dynamics and geometry](https://mlsanity.com/paper/2304.03376/) (2023)
- [Interpretable learning of effective dynamics for multiscale systems](https://mlsanity.com/paper/2309.05812/) (2023)
- [LD-EnSF: Synergizing Latent Dynamics with Ensemble Score Filters for Fast Data Assimilation with Sparse Observations](https://mlsanity.com/paper/2411.19305/) (2024)
- [Data-driven low-dimensional dynamic model of Kolmogorov flow](https://mlsanity.com/paper/2210.16708/) (2022)
- [VENI, VINDy, VICI: a generative reduced-order modeling framework with uncertainty quantification](https://mlsanity.com/paper/2405.20905/) (2024)
- [Semi-Supervised Manifold Learning with Complexity Decoupled Chart Autoencoders](https://mlsanity.com/paper/2208.10570/) (2022)
- [Deep learning delay coordinate dynamics for chaotic attractors from partial observable data](https://mlsanity.com/paper/2211.11061/) (2022)
- [Emerging trends in machine learning for computational fluid dynamics](https://mlsanity.com/paper/2211.15145/) (2022)
- [Dynamics of a data-driven low-dimensional model of turbulent minimal Couette flow](https://mlsanity.com/paper/2301.04638/) (2023)
- [Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets](https://mlsanity.com/paper/2310.16741/) (2023)
- [Discovering Symbolic Differential Equations with Symmetry Invariants](https://mlsanity.com/paper/2505.12083/) (2025)
- [Turbulence control in plane Couette flow using low-dimensional neural ODE-based models and deep reinforcement learning](https://mlsanity.com/paper/2301.12098/) (2023)
- [Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems](https://mlsanity.com/paper/2304.01732/) (2023)
- [Latent Dynamics Networks (LDNets): learning the intrinsic dynamics of spatio-temporal processes](https://mlsanity.com/paper/2305.00094/) (2023)
- [Finding emergence in data by maximizing effective information](https://mlsanity.com/paper/2308.09952/) (2023)
- [LES-SINDy: Laplace-Enhanced Sparse Identification of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2411.01719/) (2024)
- [Conservation-informed Graph Learning for Spatiotemporal Dynamics Prediction](https://mlsanity.com/paper/2412.20962/) (2024)
- [Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping Points](https://mlsanity.com/paper/2309.14334/) (2023)
- [Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional data](https://mlsanity.com/paper/2401.01100/) (2024)
- [Learning solution operator of dynamical systems with diffusion maps kernel ridge regression](https://mlsanity.com/paper/2512.17203/) (2025)
- [Learning Continuous Network Emerging Dynamics from Scarce Observations via Data-Adaptive Stochastic Processes](https://mlsanity.com/paper/2310.16466/) (2023)
- [Building symmetries into data-driven manifold dynamics models for complex flows: application to two-dimensional Kolmogorov flow](https://mlsanity.com/paper/2312.10235/) (2023)
- [RefreshNet: Learning Multiscale Dynamics through Hierarchical Refreshing](https://mlsanity.com/paper/2401.13282/) (2024)
- [On instabilities in neural network-based physics simulators](https://mlsanity.com/paper/2406.13101/) (2024)
- [Ultralow-dimensionality reduction for identifying critical transitions by spatial-temporal PCA](https://mlsanity.com/paper/2501.12582/) (2025)
- [Efficient Manifold-Constrained Neural ODE for High-Dimensional Datasets](https://mlsanity.com/paper/2510.04138/) (2025)

## Abstract (arXiv)

Dynamical models underpin our ability to understand and predict the behavior of natural systems. Whether dynamical models are developed from first-principles derivations or from observational data, they are predicated on our choice of state variables. The choice of state variables is driven by convenience and intuition, and in the data-driven case the observed variables are often chosen to be the state variables. The dimensionality of these variables (and consequently the dynamical models) can be arbitrarily large, obscuring the underlying behavior of the system. In truth, these variables are often highly redundant and the system is driven by a much smaller set of latent intrinsic variables. In this study, we combine the mathematical theory of manifolds with the representational capacity of neural networks to develop a method that learns a system's intrinsic state variables directly from time series data, and also learns predictive models for their dynamics. What distinguishes our method is its ability to reduce data to the intrinsic dimensionality of the nonlinear manifold they live on. This ability is enabled by the concepts of charts and atlases from the theory of manifolds, whereby a manifold is represented by a collection of patches that are sewn together -- a necessary representation to attain intrinsic dimensionality. We demonstrate this approach on several high-dimensional systems with low-dimensional behavior. The resulting framework provides the ability to develop dynamical models of the lowest possible dimension, capturing the essence of a system.