# LyTimeT: Towards Robust and Interpretable State-Variable Discovery

arXiv: https://arxiv.org/abs/2510.19716 · HTML: https://mlsanity.com/paper/2510.19716/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks](https://mlsanity.com/paper/2110.12976/) (2021)
- [LyaNet: A Lyapunov Framework for Training Neural ODEs](https://mlsanity.com/paper/2202.02526/) (2022)
- [Discovering State Variables Hidden in Experimental Data](https://mlsanity.com/paper/2112.10755/) (2021)
- [Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Experts](https://mlsanity.com/paper/2402.05970/) (2024)
- [Aligning AI-driven discovery with human intuition](https://mlsanity.com/paper/2410.07397/) (2024)

## Built on by (most cited first, 2 of 2)

- [Damage Adaptation in Seconds for Architected Materials](https://mlsanity.com/paper/2606.17394/) (2026)
- [CHLU: The Causal Hamiltonian Learning Unit as a Symplectic Primitive for Deep Learning](https://mlsanity.com/paper/2603.01768/) (2026)

## Abstract (arXiv)

Extracting the true dynamical variables of a system from high-dimensional video is challenging due to distracting visual factors such as background motion, occlusions, and texture changes. We propose LyTimeT, a two-phase framework for interpretable variable extraction that learns robust and stable latent representations of dynamical systems. In Phase 1, LyTimeT employs a spatio-temporal TimeSformer-based autoencoder that uses global attention to focus on dynamically relevant regions while suppressing nuisance variation, enabling distraction-robust latent state learning and accurate long-horizon video prediction. In Phase 2, we probe the learned latent space, select the most physically meaningful dimensions using linear correlation analysis, and refine the transition dynamics with a Lyapunov-based stability regularizer to enforce contraction and reduce error accumulation during roll-outs. Experiments on five synthetic benchmarks and four real-world dynamical systems, including chaotic phenomena, show that LyTimeT achieves mutual information and intrinsic dimension estimates closest to ground truth, remains invariant under background perturbations, and delivers the lowest analytical mean squared error among CNN-based (TIDE) and transformer-only baselines. Our results demonstrate that combining spatio-temporal attention with stability constraints yields predictive models that are not only accurate but also physically interpretable.