mlsanityData to 7 Oct 2026

Out-of-Domain Generalization in Dynamical Systems Reconstruction

arXiv 2402.18377 ↗First posted 28 Feb 2024cs.LG cs.AI math.DScited by 33

Not on the map. Why?

Abstract (arXiv)

In science we are interested in finding the governing equations, the dynamical rules, underlying empirical phenomena. While traditionally scientific models are derived through cycles of human insight and experimentation, recently deep learning (DL) techniques have been advanced to reconstruct dynamical systems (DS) directly from time series data. State-of-the-art dynamical systems reconstruction (DSR) methods show promise in capturing invariant and long-term properties of observed DS, but their ability to generalize to unobserved domains remains an open challenge. Yet, this is a crucial property we would expect from any viable scientific theory. In this work, we provide a formal framework that addresses generalization in DSR. We explain why and how out-of-domain (OOD) generalization (OODG) in DSR profoundly differs from OODG considered elsewhere in machine learning. We introduce mathematical notions based on topological concepts and ergodic theory to formalize the idea of learnability of a DSR model. We formally prove that black-box DL techniques, without adequate structural priors, generally will not be able to learn a generalizing DSR model. We also show this empirically, considering major classes of DSR algorithms proposed so far, and illustrate where and why they fail to generalize across the whole phase space. Our study provides the first comprehensive mathematical treatment of OODG in DSR, and gives a deeper conceptual understanding of where the fundamental problems in OODG lie and how they could possibly be addressed in practice.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 69 of its 151 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

69 of 151 references.

References · 69

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  4. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  5. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  6. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  7. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 491
  8. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  9. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  10. 2020Universal Differential Equations for Scientific Machine Learningcited by 345
  11. 2018Deep learning generalizes because the parameter-function map is biased towards simple functionscited by 180
  12. 2021Next Generation Reservoir Computingcited by 160
38 more
  1. 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
  2. 2017Linearly-Recurrent Autoencoder Networks for Learning Dynamicscited by 142
  3. 2017Sparse identification of nonlinear dynamics for model predictive control in the low-data limitcited by 139
  4. 2019Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamicscited by 132
  5. 2021Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolationcited by 130
  6. 2020Forecasting Sequential Data using Consistent Koopman Autoencoderscited by 114
  7. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  8. 2020Transformers for Modeling Physical Systemscited by 105
  9. 2018Data Driven Governing Equations Approximation Using Deep Neural Networkscited by 94
  10. 2018Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systemscited by 88
  11. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 87
  12. 2020Multiscale Simulations of Complex Systems by Learning their Effective Dynamicscited by 86
  13. 2022Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functionscited by 68
  14. 2015Sliced Wasserstein Kernels for Probability Distributionscited by 64
  15. 2021On the difficulty of learning chaotic dynamics with RNNscited by 64
  16. 2021Attractor and integrator networks in the braincited by 60
  17. 2023ODEFormer: Symbolic Regression of Dynamical Systems with Transformerscited by 58
  18. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
  19. 2019Understanding Generalization through Visualizationscited by 54
  20. 2022Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systemscited by 50
  21. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 49
  22. 2023Training neural operators to preserve invariant measures of chaotic attractorscited by 43
  23. 2021Long Expressive Memory for Sequence Modelingcited by 41
  24. 2019Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRIcited by 39
  25. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 39
  26. 2020Generalization bounds for deep learningcited by 38
  27. 2019Learning Dynamical Systems from Partial Observationscited by 36
  28. 2020Automatic Differentiation to Simultaneously Identify Nonlinear Dynamics and Extract Noise Probability Distributions from Datacited by 36
  29. 2016Recurrent switching linear dynamical systemscited by 34
  30. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 34
  31. 2022Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systemscited by 33
  32. 2020On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Toolscited by 31
  33. 2022A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamicscited by 30
  34. 2016LFADS - Latent Factor Analysis via Dynamical Systemscited by 29
  35. 2019Learning interpretable continuous-time models of latent stochastic dynamical systemscited by 29
  36. 2020Sparse Identification of Nonlinear Dynamical Systems via Reweighted ℓ1-regularized Least Squarescited by 21
  37. 2021DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimizationcited by 19
  38. 2020Neural Dynamical Systems: Balancing Structure and Flexibility in Physical Predictioncited by 18

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 13; 2026: 13.

33 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  2. 2024Zero-shot forecasting of chaotic systemscited by 25
  3. 2025True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statisticscited by 20
  4. 2025Out-of-Distribution Generalization in Time Series: A Surveycited by 15
  5. 2024Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Datacited by 14
  6. 2024When are dynamical systems learned from time series data statistically accurate?cited by 14
  7. 2025Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learningcited by 14
  8. 2025Panda: A pretrained forecast model for chaotic dynamicscited by 12
  9. 2024Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter Diffusion Guidancecited by 5
  10. 2024How more data can hurt: Instability and regularization in next-generation reservoir computingcited by 5
  11. 2025Towards Foundational Models for Dynamical System Reconstruction: Hierarchical Meta-Learning via Mixture of Expertscited by 5
  12. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
21 more
  1. 2026Continuous-Time Piecewise-Linear Recurrent Neural Networkscited by 3
  2. 2026Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modelingcited by 3
  3. 2024Reevaluating Meta-Learning Optimization Algorithms Through Contextual Self-Modulationcited by 2
  4. 2025Tailored Forecasting from Short Time Series via Meta-learningcited by 2
  5. 2025Learning Beyond Experience: Generalizing to Unseen State Space with Reservoir Computingcited by 2
  6. 2025Environment Inference for Learning Generalizable Dynamical Systemcited by 2
  7. 2026Is Flow Matching Just Trajectory Replay for Sequential Data?cited by 2
  8. 2026Teacher Forcing as Generalized Bayes: Optimization Geometry Mismatch in Switching Surrogates for Chaotic Dynamicscited by 2
  9. 2024Modeling Nonlinear Oscillator Networks Using Physics-Informed Hybrid Reservoir Computingcited by 1
  10. 2025KPFlow: An Operator Perspective on Dynamic Collapse Under Gradient Descent Training of Recurrent Networkscited by 1
  11. 2025ChaosNexus: A Foundation Model for ODE-based Chaotic System Forecasting with Hierarchical Multi-scale Awarenesscited by 1
  12. 2026Transformers for dynamical systems learn transfer operators in-contextcited by 1
  13. 2026The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modelingcited by 1
  14. 2025What Neuroscience Can Teach AI About Learning in Continuously Changing Environmentscited by 0
  15. 2026Storage and selection of multiple chaotic attractors in minimal reservoir computerscited by 0
  16. 2026Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstructioncited by 0
  17. 2026OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systemscited by 0
  18. 2026Topological Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 0
  19. 2026A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systemscited by 0
  20. 2026Extrapolating the emergence of Hamiltonian chaos with random-feature Hamiltonian neural networkscited by 0
  21. 2026Information-theoretic receding-horizon active learning of nonlinear dynamical systemscited by 0