# True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics

arXiv: https://arxiv.org/abs/2505.13192 · HTML: https://mlsanity.com/paper/2505.13192/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 65 of its 105 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 30 of 65)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [A decoder-only foundation model for time-series forecasting](https://mlsanity.com/paper/2310.10688/) (2023)
- [Large Language Models Are Zero-Shot Time Series Forecasters](https://mlsanity.com/paper/2310.07820/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Series](https://mlsanity.com/paper/2308.08241/) (2023)
- [Are Language Models Actually Useful for Time Series Forecasting?](https://mlsanity.com/paper/2406.16964/) (2024)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Sparse identification of nonlinear dynamics for model predictive control in the low-data limit](https://mlsanity.com/paper/1711.05501/) (2017)
- [TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2306.09364/) (2023)
- [Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series](https://mlsanity.com/paper/2401.03955/) (2024)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts](https://mlsanity.com/paper/2410.10469/) (2024)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [Context is Key: A Benchmark for Forecasting with Essential Textual Information](https://mlsanity.com/paper/2410.18959/) (2024)
- [On the difficulty of learning chaotic dynamics with RNNs](https://mlsanity.com/paper/2110.07238/) (2021)
- [Meta-in-context learning in large language models](https://mlsanity.com/paper/2305.12907/) (2023)
- [Generalized Teacher Forcing for Learning Chaotic Dynamics](https://mlsanity.com/paper/2306.04406/) (2023)

## Built on by (most cited first, 20 of 20)

- [Weight-Space Linear Recurrent Neural Networks](https://mlsanity.com/paper/2506.01153/) (2025)
- [Continuous-Time Piecewise-Linear Recurrent Neural Networks](https://mlsanity.com/paper/2602.15649/) (2026)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [TempusBench: An Evaluation Framework for Time-Series Forecasting](https://mlsanity.com/paper/2604.11529/) (2026)
- [Foundation Inference Models for Ordinary Differential Equations](https://mlsanity.com/paper/2602.08733/) (2026)
- [Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics](https://mlsanity.com/paper/2507.06050/) (2025)
- [A Mechanistic Analysis of Transformers for Dynamical Systems](https://mlsanity.com/paper/2512.21113/) (2025)
- [ChaosNexus: A Foundation Model for ODE-based Chaotic System Forecasting with Hierarchical Multi-scale Awareness](https://mlsanity.com/paper/2509.21802/) (2025)
- [The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling](https://mlsanity.com/paper/2605.31547/) (2026)
- [PhyxMamba: Chaotic System Reconstruction from Short Context Observations with Generative State-Space Models](https://mlsanity.com/paper/2505.23863/) (2025)
- [Scaling Law for Large-Scale Pre-Training Using Chaotic Time Series and Predictability in Financial Time Series](https://mlsanity.com/paper/2509.04921/) (2025)
- [Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Data](https://mlsanity.com/paper/2602.12267/) (2026)
- [CINDI: Conditional Imputation and Noisy Data Integrity with Flows in Power Grid Data](https://mlsanity.com/paper/2603.11745/) (2026)
- [Storage and selection of multiple chaotic attractors in minimal reservoir computers](https://mlsanity.com/paper/2603.15155/) (2026)
- [Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics](https://mlsanity.com/paper/2604.17670/) (2026)
- [Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction](https://mlsanity.com/paper/2605.12683/) (2026)
- [Topological Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2606.22969/) (2026)
- [A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems](https://mlsanity.com/paper/2607.14937/) (2026)
- [Predicting the Unpredictable: LLM-powered Long-term Chaotic Time Series Forecasting under Short-term Observations](https://mlsanity.com/paper/2608.29579/) (2026)
- [Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Events](https://mlsanity.com/paper/2609.30746/) (2026)

## Abstract (arXiv)

Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce DynaMix, a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail -- at a fraction of the number of parameters (0.1%) and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, but not at all part of DynaMix' training corpus. We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field.