mlsanityData to 14 Oct 2026

True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics

arXiv 2505.13192 ↗First posted 19 May 2025cs.LG cs.AI math.DScited by 20

Not on the map. Why?

Abstract (arXiv)

Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce DynaMix, a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail -- at a fraction of the number of parameters (0.1%) and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, but not at all part of DynaMix' training corpus. We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 65 of its 105 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

65 of 105 references.

References · 65

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2018Neural Ordinary Differential Equationscited by 3,779
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  6. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  7. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  8. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  9. 2024Chronos: Learning the Language of Time Seriescited by 759
  10. 2022Transformers learn in-context by gradient descentcited by 732
  11. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  12. 2023A decoder-only foundation model for time-series forecastingcited by 692
38 more
  1. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  2. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  3. 2021Modern Koopman Theory for Dynamical Systemscited by 276
  4. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  5. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 168
  6. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  7. 2017Linearly-Recurrent Autoencoder Networks for Learning Dynamicscited by 145
  8. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 140
  9. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 139
  10. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  11. 2020Forecasting Sequential Data using Consistent Koopman Autoencoderscited by 115
  12. 2020Transformers for Modeling Physical Systemscited by 106
  13. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  14. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 88
  15. 2024Context is Key: A Benchmark for Forecasting with Essential Textual Informationcited by 87
  16. 2021On the difficulty of learning chaotic dynamics with RNNscited by 67
  17. 2023Meta-in-context learning in large language modelscited by 59
  18. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 58
  19. 2022Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practicescited by 55
  20. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 50
  21. 2024Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilitiescited by 48
  22. 2023Training neural operators to preserve invariant measures of chaotic attractorscited by 46
  23. 2024Dual Operating Modes of In-Context Learningcited by 43
  24. 2019Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRIcited by 41
  25. 2021Long Expressive Memory for Sequence Modelingcited by 41
  26. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 40
  27. 2022Koopman Neural Forecaster for Time Series with Temporal Distribution Shiftscited by 37
  28. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 36
  29. 2022Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systemscited by 33
  30. 2024Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 33
  31. 2022A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamicscited by 30
  32. 2024DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systemscited by 30
  33. 2024Zero-shot forecasting of chaotic systemscited by 28
  34. 2023Model scale versus domain knowledge in statistical forecasting of chaotic systemscited by 27
  35. 2016A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurementscited by 25
  36. 2023Generative learning for nonlinear dynamicscited by 22
  37. 2020Sparse Identification of Nonlinear Dynamical Systems via Reweighted ℓ1-regularized Least Squarescited by 21
  38. 2024Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstructioncited by 19

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 14.

20 papers.

Sort:

  1. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
  2. 2026Continuous-Time Piecewise-Linear Recurrent Neural Networkscited by 4
  3. 2026Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modelingcited by 4
  4. 2026TempusBench: An Evaluation Framework for Time-Series Forecastingcited by 4
  5. 2026Foundation Inference Models for Ordinary Differential Equationscited by 3
  6. 2025Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamicscited by 2
  7. 2025A Mechanistic Analysis of Transformers for Dynamical Systemscited by 2
  8. 2025ChaosNexus: A Foundation Model for ODE-based Chaotic System Forecasting with Hierarchical Multi-scale Awarenesscited by 1
  9. 2026The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modelingcited by 1
  10. 2025PhyxMamba: Chaotic System Reconstruction from Short Context Observations with Generative State-Space Modelscited by 0
  11. 2025Scaling Law for Large-Scale Pre-Training Using Chaotic Time Series and Predictability in Financial Time Seriescited by 0
  12. 2026Self-Supervised Learning via Flow-Guided Neural Operator on Time-Series Datacited by 0
8 more
  1. 2026CINDI: Conditional Imputation and Noisy Data Integrity with Flows in Power Grid Datacited by 0
  2. 2026Storage and selection of multiple chaotic attractors in minimal reservoir computerscited by 0
  3. 2026Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokineticscited by 0
  4. 2026Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstructioncited by 0
  5. 2026Topological Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 0
  6. 2026A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systemscited by 0
  7. 2026Predicting the Unpredictable: LLM-powered Long-term Chaotic Time Series Forecasting under Short-term Observationscited by 0
  8. 2026Mechanism-Aware Ensemble Conditioning for Data-Limited Emulation of Extreme Eventscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Series
  2. 2024eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling
  3. 2026More Experts, Worse Dynamics: Inverse Scaling and Spectral Bias in Mixture-of-Experts State-Space Models
  4. 2026Towards Infinitely Long Neural Simulations: Self-Refining Neural Surrogate Models for Dynamical Systems
  5. 2026DAWIS: Data Assimilation with Windowed Inverse Sampling via Multitask Interpolants
  6. 2025DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Series
  7. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning
  8. 2025Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme