# Tensorized Transformer for Dynamical Systems Modeling

arXiv: https://arxiv.org/abs/2006.03445 · HTML: https://mlsanity.com/paper/2006.03445/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2020.

## Built on by (most cited first, 4 of 4)

- [Data-driven Prediction of General Hamiltonian Dynamics via Learning Exactly-Symplectic Maps](https://mlsanity.com/paper/2103.05632/) (2021)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformers](https://mlsanity.com/paper/2206.06131/) (2022)
- [Contrastive and Multi-Task Learning on Noisy Brain Signals with Nonlinear Dynamical Signatures](https://mlsanity.com/paper/2601.08549/) (2026)

## Abstract (arXiv)

The identification of nonlinear dynamics from observations is essential for the alignment of the theoretical ideas and experimental data. The last, in turn, is often corrupted by the side effects and noise of different natures, so probabilistic approaches could give a more general picture of the process. At the same time, high-dimensional probabilities modeling is a challenging and data-intensive task. In this paper, we establish a parallel between the dynamical systems modeling and language modeling tasks. We propose a transformer-based model that incorporates geometrical properties of the data and provide an iterative training algorithm allowing the fine-grid approximation of the conditional probabilities of high-dimensional dynamical systems.