mlsanityData to 7 Oct 2026

A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems

arXiv 2607.14937 ↗First posted 16 Jul 2026cs.LG cs.AI math.DScited by 0

Not on the map. Why?

Abstract (arXiv)

Recent foundation models (FMs) for zero-shot reconstruction of dynamical systems (DS) achieve strong out-of-domain generalization but provide little insight into the mechanisms that underlie their forecasts. Such an understanding could help to strip down overladen FM architectures to their bare essence and expose the minimal requirements for in-context learning in the DS domain. Toward this goal, here we iteratively reduce a recent powerful SOTA model for DS reconstruction, DynaMix (Hemmer & Durstewitz, 2025), to a minimal interpretable two-parameter form, which we call DynaBase. DynaBase produces forecasts through a linear blend of the current latent state and the nearest in-context neighbor and its temporal successor. Surprisingly, despite its extreme simplicity, DynaBase produces highly competitive zero-shot DS reconstructions across chaotic and cyclic systems, with a negligible parameter load, many orders of magnitude below that of other FMs. Even more, this extreme simplicity permits direct model optimization on DS reconstruction measures, as well as closed-form one-step analytical solutions on prediction MSE. Theoretical and empirical analysis of DynaBase further leads to a 1-parameter family of maps, with the context-parroting algorithm of (Zhang & Gilpin, 2026) recovered at one end, and chaotic (divergent but bounded) behavior at the other. We further show how different training strategies lead to models either optimal for short-term prediction or for DS reconstruction. Thus, DynaBase not only exposes the minimal mechanisms required for producing zero-shot DS reconstruction, but also reconciles within an accessible mathematical frame divergent observations in the literature.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 52 of its 91 references that are in the map. First posted on arXiv 16 Jul 2026.

Builds on

52 of 91 references.

References · 52

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2022In-context Learning and Induction Headscited by 930
  5. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  6. 2022Transformers learn in-context by gradient descentcited by 714
  7. 2024Chronos: Learning the Language of Time Seriescited by 712
  8. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  9. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  10. 2023A decoder-only foundation model for time-series forecastingcited by 650
  11. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  12. 2024xLSTM: Extended Long Short-Term Memorycited by 320
38 more
  1. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  2. 2021Modern Koopman Theory for Dynamical Systemscited by 269
  3. 2025Hierarchical Reasoning Modelcited by 167
  4. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  5. 2017Linearly-Recurrent Autoencoder Networks for Learning Dynamicscited by 142
  6. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 115
  7. 2020Forecasting Sequential Data using Consistent Koopman Autoencoderscited by 114
  8. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 87
  9. 2021On the difficulty of learning chaotic dynamics with RNNscited by 64
  10. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
  11. 2022Forecast Evaluation for Data Scientists: Common Pitfalls and Best Practicescited by 54
  12. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
  13. 2024Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformerscited by 46
  14. 2023Training neural operators to preserve invariant measures of chaotic attractorscited by 43
  15. 2024Dual Operating Modes of In-Context Learningcited by 42
  16. 2021Long Expressive Memory for Sequence Modelingcited by 41
  17. 2019Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRIcited by 39
  18. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 39
  19. 2022Koopman Neural Forecaster for Time Series with Temporal Distribution Shiftscited by 35
  20. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 34
  21. 2022Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systemscited by 33
  22. 2024Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 33
  23. 2022A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamicscited by 30
  24. 2023Model scale versus domain knowledge in statistical forecasting of chaotic systemscited by 27
  25. 2024Zero-shot forecasting of chaotic systemscited by 25
  26. 2016A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurementscited by 24
  27. 2024Can In-context Learning Really Generalize to Out-of-distribution Tasks?cited by 23
  28. 2025True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statisticscited by 20
  29. 2024Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstructioncited by 18
  30. 2020DyNODE: Neural Ordinary Differential Equations for Dynamics Modeling in Continuous Controlcited by 17
  31. 2024Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Modelscited by 17
  32. 2015Synthesis of recurrent neural networks for dynamical system simulationcited by 16
  33. 2024Inferring stochastic low-rank recurrent neural networks from neural datacited by 15
  34. 2022Homotopy-based training of NeuralODEs for accurate dynamics discoverycited by 14
  35. 2024Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Datacited by 14
  36. 2025Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learningcited by 14
  37. 2024Optimal Recurrent Network Topologies for Dynamical Systems Reconstructioncited by 8
  38. 2024Meta-Dynamical State Space Models for Integrative Neural Data Analysiscited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.